Amazing technological breakthrough possible @S-Logix

Office Address

  • #5, First Floor, 4th Street Dr. Subbarayan Nagar Kodambakkam, Chennai-600 024 Landmark : Samiyar Madam
  • +91- 81240 01111

Social List

Pre-trained models for natural language processing:A survey - 2020

Pre-Trained Models For Natural Language Processing:A Survey

Research Area:  Machine Learning


Recently, the emergence of pre-trained models (PTMs) has brought natural language processing (NLP) to a new era. In this survey, we provide a comprehensive review of PTMs for NLP. We first briefly introduce language representation learning and its research progress. Then we systematically categorize existing PTMs based on a taxonomy from four different perspectives. Next, we describe how to adapt the knowledge of PTMs to downstream tasks. Finally, we outline some potential directions of PTMs for future research. This survey is purposed to be a hands-on guide for understanding, using, and developing PTMs for various NLP tasks.


Author(s) Name:  XiPeng Qiu, TianXiang Sun, YiGe Xu, YunFan Shao, Ning Dai & XuanJing Huang

Journal name:  Science China Technological Sciences

Conferrence name:  

Publisher name:  Springer

DOI:  10.1007/s11431-020-1647-3

Volume Information:  volume 63, pages1872–1897 (2020)