Research Area:  Machine Learning

Abstract:

Transfer Learning (TL) has achieved significant developments in the past few years. However, the majority of work on TL assume implicit access to both the extit{target} and extit{source} datasets, which limits its application in the context of Federated Learning (FL), where target (client) datasets are usually not directly accessible. In this paper, we address the problem of source model selection in TL for federated scenarios. We propose a simple framework, called Selective Federated Transfer Learning (SFTL), to select the best pre-trained models which provide a positive performance gain when their parameters are transferred on to a new task. We leverage the concepts from representation similarity to compare the similarity of the client model and the source models and provide a method which could be augmented to existing FL algorithms to improve both the communication cost and the accuracy of client models.

Keywords:  

Author(s) Name:  Tushar Semwal Haofan Wang Chinnakotla Krishna Teja Reddy

Publisher name:  Center for Open Science

DOI:  10.31219/osf.io/kbhq5

Volume Information:  Volume 2021

Paper Link:   https://osf.io/kbhq5