Amazing technological breakthrough possible @S-Logix

Office Address

  • 2nd Floor, #7a, High School Road, Secretariat Colony Ambattur, Chennai-600053 (Landmark: SRM School) Tamil Nadu, India
  • +91- 81240 01111

Social List

SciBERT:A Pretrained Language Model for Scientific Text - 2019

Scibert:A Pretrained Language Model For Scientific Text

Research Area:  Machine Learning


Obtaining large-scale annotated data for NLP tasks in the scientific domain is challenging and expensive. We release SciBERT, a pretrained language model based on BERT (Devlin et al., 2018) to address the lack of high-quality, large-scale labeled scientific data. SciBERT leverages unsupervised pretraining on a large multi-domain corpus of scientific publications to improve performance on downstream scientific NLP tasks. We evaluate on a suite of tasks including sequence tagging, sentence classification and dependency parsing, with datasets from a variety of scientific domains. We demonstrate statistically significant improvements over BERT and achieve new state-of-the-art results on several of these tasks.


Author(s) Name:  Iz Beltagy, Kyle Lo, Arman Cohan

Journal name:  Computer Science

Conferrence name:  

Publisher name:  arXiv:1903.10676

DOI:  10.48550/arXiv.1903.10676

Volume Information: