Feature Align-HFL: A Feature Alignment Based Client-Specific Model For Heterogeneous Federated Learning

Abstract

Federated Learning (FL) enables decentralized model training while ensuring data privacy between clients and the server, and is applicable in various domains such as healthcare, finance, and edge computing. Traditional FL approaches assume homogeneous model architectures across clients and servers, limiting their applicability in real-world scenarios where clients use diverse models. To address this, we introduce Feature Align-HFL, a novel framework designed for Heterogeneous Federated Learning (HFL) that aligns dissimilar models for effective knowledge sharing. Specifically, we demonstrate its application by aligning CNNs and U-Nets for a common classification task. By employing cosine similarity, Feature Align-HFL identifies and aligns common features across heterogeneous models. Experiments on the CIFAR-10 dataset show that CNN and adapted U-Net models achieved 90 percent and 86 percent accuracy, respectively, with feature map similarities reaching up to 96 percent. These results highlight the significant transferable knowledge between dissimilar architectures, indicating that Feature Align-HFL can effectively support knowledge sharing in heterogeneous FL settings.