Amazing technological breakthrough possible @S-Logix

Office Address

  • #5, First Floor, 4th Street Dr. Subbarayan Nagar Kodambakkam, Chennai-600 024 Landmark : Samiyar Madam
  • +91- 81240 01111

Social List

An Introductory Survey on Attention Mechanisms in NLP Problems - 2019

An Introductory Survey On Attention Mechanisms In Nlp Problems

Research Area:  Machine Learning


First derived from human intuition, later adapted to machine translation for automatic token alignment, attention mechanism, a simple method that can be used for encoding sequence data based on the importance score each element is assigned, has been widely applied to and attained significant improvement in various tasks in natural language processing, including sentiment classification, text summarization, question answering, dependency parsing, etc. In this paper, we survey through recent works and conduct an introductory summary of the attention mechanism in different NLP problems, aiming to provide our readers with basic knowledge on this widely used method, discuss its different variants for different tasks, explore its association with other techniques in machine learning, and examine methods for evaluating its performance.


Author(s) Name:  Dichao HuEmail author

Journal name:  

Conferrence name:  Proceedings of SAI Intelligent Systems Conference

Publisher name:  Springer

DOI:  10.1007/978-3-030-29513-4_31

Volume Information:  pp 432-448