Research breakthrough possible @S-Logix pro@slogix.in

Office Address

  • 2nd Floor, #7a, High School Road, Secretariat Colony Ambattur, Chennai-600053 (Landmark: SRM School) Tamil Nadu, India
  • pro@slogix.in
  • +91- 81240 01111

Social List

BLEURT:Learning Robust Metrics for Text Generation - 2020

Bleurt:Learning Robust Metrics For Text Generation

Research Area:  Machine Learning

Abstract:

Text generation has made significant advances in the last few years. Yet, evaluation metrics have lagged behind, as the most popular choices (e.g., BLEU and ROUGE) may correlate poorly with human judgments. We propose BLEURT, a learned evaluation metric based on BERT that can model human judgments with a few thousand possibly biased training examples. A key aspect of our approach is a novel pre-training scheme that uses millions of synthetic examples to help the model generalize. BLEURT provides state-of-the-art results on the last three years of the WMT Metrics shared task and the WebNLG Competition dataset. In contrast to a vanilla BERT-based approach, it yields superior results even when the training data is scarce and out-of-distribution.

Keywords:  

Author(s) Name:  Thibault Sellam, Dipanjan Das, Ankur P. Parikh

Journal name:  Computer Science

Conferrence name:  

Publisher name:  arXiv:2004.04696

DOI:  10.48550/arXiv.2004.04696

Volume Information: