Research Area:  Machine Learning

Abstract:

Attention Model has now become an important concept in neural networks that has been researched within diverse application domains. This survey provides a structured and comprehensive overview of the developments in modeling attention. In particular, we propose a taxonomy that groups existing techniques into coherent categories. We review salient neural architectures in which attention has been incorporated and discuss applications in which modeling attention has shown a significant impact. We also describe how attention has been used to improve the interpretability of neural networks. Finally, we discuss some future research directions in attention. We hope this survey will provide a succinct introduction to attention models and guide practitioners while developing approaches for their applications.

Keywords:  

Author(s) Name:  Sneha Chaudhari , Varun Mithal , Gungor Polatkan , Rohan Ramanath

Publisher name:  ACM

DOI:  10.1145/3465055

Volume Information:  Volume 12,Issue 5,October 2021, Article No.: 53,pp 1–32

Paper Link:   https://dl.acm.org/doi/abs/10.1145/3465055