Amazing technological breakthrough possible @S-Logix pro@slogix.in

Office Address

  • #5, First Floor, 4th Street Dr. Subbarayan Nagar Kodambakkam, Chennai-600 024 Landmark : Samiyar Madam
  • pro@slogix.in
  • +91- 81240 01111

Social List

BERT Rediscovers the Classical NLP Pipeline - 2019

Research Area:  Machine Learning

Abstract:

Pre-trained text encoders have rapidly advanced the state of the art on many NLP tasks. We focus on one such model, BERT, and aim to quantify where linguistic information is captured within the network. We find that the model represents the steps of the traditional NLP pipeline in an interpretable and localizable way, and that the regions responsible for each step appear in the expected sequence: POS tagging, parsing, NER, semantic roles, then coreference. Qualitative analysis reveals that the model can and often does adjust this pipeline dynamically, revising lower-level decisions on the basis of disambiguating information from higher-level representations.

Author(s) Name:  Ian Tenney, Dipanjan Das, Ellie Pavlick

Journal name:  Computer Science

Conferrence name:  

Publisher name:  arXiv:1905.05950

DOI:  10.48550/arXiv.1905.05950

Volume Information: