Amazing technological breakthrough possible @S-Logix pro@slogix.in

Office Address

  • 2nd Floor, #7a, High School Road, Secretariat Colony Ambattur, Chennai-600053 (Landmark: SRM School) Tamil Nadu, India
  • pro@slogix.in
  • +91- 81240 01111

Social List

Decentralized Federated Learning: Balancing Communication and Computing Costs - 2022

Decentralized Federated Learning: Balancing Communication And Computing Costs

Research Paper on Decentralized Federated Learning: Balancing Communication And Computing Costs

Research Area:  Machine Learning

Abstract:

Decentralized stochastic gradient descent (SGD) is a driving engine for decentralized federated learning (DFL). The performance of decentralized SGD is jointly influenced by inter-node communications and local updates. In this paper, we propose a general DFL framework, which implements both multiple local updates and multiple inter-node communications periodically, to strike a balance between communication efficiency and model consensus. It can provide a general decentralized SGD analytical framework. We establish strong convergence guarantees for the proposed DFL algorithm without the assumption of convex objectives. The convergence rate of DFL can be optimized to achieve the balance of communication and computing costs under constrained resources. For improving communication efficiency of DFL, compressed communication is further introduced to the proposed DFL as a new scheme, named DFL with compressed communication (C-DFL). The proposed C-DFL exhibits linear convergence for strongly convex objectives. Experiment results based on MNIST and CIFAR-10 datasets illustrate the superiority of DFL over traditional decentralized SGD methods and show that C-DFL further enhances communication efficiency.

Keywords:  
Decentralized Federated Learning
stochastic gradient descent
Deep Learning
Machine Learning

Author(s) Name:  Wei Liu; Li Chen; Wenyi Zhang

Journal name:  IEEE Transactions on Signal and Information Processing over Networks

Conferrence name:  

Publisher name:  IEEE

DOI:  10.1109/TSIPN.2022.3151242

Volume Information:  ( Volume: 8), Page(s): 131 - 143