Research Area:  Machine Learning

Abstract:

The emerging concern about data privacy and security has motivated the proposal of federated learning, which allows nodes to only synchronize the locally-trained models instead their own original data. Conventional federated learning architecture, inherited from the parameter server design, relies on highly centralized topologies and the assumption of large nodes-to-server bandwidths. However, in real-world federated learning scenarios the network capacities between nodes are highly uniformly distributed and smaller than that in a datacenter. It is of great challenges for conventional federated learning approaches to efficiently utilize network capacities between nodes. In this paper, we propose a model segment level decentralized federated learning to tackle this problem. In particular, we propose a segmented gossip approach, which not only makes full utilization of node-to-node bandwidth, but also has good training convergence. The experimental results show that even the training time can be highly reduced as compared to centralized federated learning.

Keywords:  
Decentralized
Federated Learning
Segmented Gossip Approach
Machine Learning
Deep Learning

Author(s) Name:  Chenghao Hu, Jingyan Jiang, Zhi Wang

Publisher name:  arXiv:1908.07782

DOI:  10.48550/arXiv.1908.07782

Volume Information:  

Paper Link:   https://arxiv.org/abs/1908.07782