Amazing technological breakthrough possible @S-Logix

Office Address

  • #5, First Floor, 4th Street Dr. Subbarayan Nagar Kodambakkam, Chennai-600 024 Landmark : Samiyar Madam
  • +91- 81240 01111

Social List

Cross-Lingual Ability of Multilingual BERT:An Empirical Study - 2019

Cross-Lingual Ability Of Multilingual Bert:An Empirical Study

Research Area:  Machine Learning


Recent work has exhibited the surprising cross-lingual abilities of multilingual BERT (M-BERT) -- surprising since it is trained without any cross-lingual objective and with no aligned data. In this work, we provide a comprehensive study of the contribution of different components in M-BERT to its cross-lingual ability. We study the impact of linguistic properties of the languages, the architecture of the model, and the learning objectives. The experimental study is done in the context of three typologically different languages -- Spanish, Hindi, and Russian -- and using two conceptually different NLP tasks, textual entailment and named entity recognition. Among our key conclusions is the fact that the lexical overlap between languages plays a negligible role in the cross-lingual success, while the depth of the network is an integral part of it.


Author(s) Name:  Karthikeyan K, Zihan Wang, Stephen Mayhew, Dan Roth

Journal name:  Computer Science

Conferrence name:  

Publisher name:  arXiv:1912.07840

DOI:  10.48550/arXiv.1912.07840

Volume Information: