Research Area:  Machine Learning

Abstract:

Recently, sequential transfer learning emerged as a modern technique for applying the “pretrain then fine-tune” paradigm to leverage existing knowledge to improve the performance of various downstream NLP tasks, with no exception of sentiment analysis. Previous pieces of literature mostly focus on reviewing the application of various deep learning models to sentiment analysis. However, supervised deep learning methods are known to be data hungry, but insufficient training data in practice may cause the application to be impractical. To this end, sequential transfer learning provided a solution to alleviate the training bottleneck issues of data scarcity and facilitate sentiment analysis application. This study aims to discuss the background of sequential transfer learning, review the evolution of pretrained models, extend the literature with the application of sequential transfer learning to different sentiment analysis tasks (aspect-based sentiment analysis, multimodal sentiment analysis, sarcasm detection, cross-domain sentiment classification, multilingual sentiment analysis, emotion detection) and suggest future research directions on model compression, effective knowledge adaptation techniques, neutrality detection and ambivalence handling tasks.

Keywords:  
Sentiment analysis
Deep learning
Word embedding
Pretrained models
Transfer learning
Natural language processing

Author(s) Name:  Jireh Yi-Le Chan, Khean Thye Bea, Steven Mun Hong Leow, Seuk Wai Phoong & Wai Khuen Cheng

Publisher name:  Springer

DOI:  10.1007/s10462-022-10183-8

Volume Information:  volume 56, pages: 749–780

Paper Link:   https://link.springer.com/article/10.1007/s10462-022-10183-8