Amazing technological breakthrough possible @S-Logix

Office Address

  • #5, First Floor, 4th Street Dr. Subbarayan Nagar Kodambakkam, Chennai-600 024 Landmark : Samiyar Madam
  • +91- 81240 01111

Social List

Layer-wised Model Aggregation for Personalized Federated Learning - 2022

Layer-Wised Model Aggregation For Personalized Federated Learning

Research Paper on Layer-Wised Model Aggregation For Personalized Federated Learning

Research Area:  Machine Learning


Personalized Federated Learning (pFL) not only can capture the common priors from broad range of distributed data, but also support customized models for heterogeneous clients. Researches over the past few years have applied the weighted aggregation manner to produce personalized models, where the weights are determined by calibrating the distance of the entire model parameters or loss values, and have yet to consider the layer-level impacts to the aggregation process, leading to lagged model convergence and inadequate personalization over non-IID datasets. In this paper, we propose a novel pFL training framework dubbed Layer-wised Personalized Federated learning (pFedLA) that can discern the importance of each layer from different clients, and thus is able to optimize the personalized model aggregation for clients with heterogeneous data. Specifically, we employ a dedicated hypernetwork per client on the server side, which is trained to identify the mutual contribution factors at layer granularity. Meanwhile, a parameterized mechanism is introduced to update the layer-wised aggregation weights to progressively exploit the inter-user similarity and realize accurate model personalization. Extensive experiments are conducted over different models and learning tasks, and we show that the proposed methods achieve significantly higher performance than state-of-the-art pFL methods.

Federated Learning
Personalized Federated Learning (pFL)
Deep Learning
Machine Learning

Author(s) Name:  Xiaosong Ma, Jie Zhang, Song Guo, Wenchao Xu

Journal name:  Computer Science

Conferrence name:  

Publisher name:  arXiv:2205.03993

DOI:  10.48550/arXiv.2205.03993

Volume Information: