Natural language processing supports computational linguistics along with deep learning models to analyze and represent the human language automatically. Deep learning models for NLP provide superior performance even with massive data but require less linguistic expertise to train and operate. Challenges in NLP are Phrasing Ambiguities, Misspellings handling, Words with Multiple Meanings, Phrases with Multiple Intentions, Uncertainty and False Positives Domain-specific and Low-resource languages.

Attention mechanisms in NLP are employed to address those challenges. The attention mechanism is one of the emerging development for natural language processing that utilizes deep learning architectures. Generally, the attention mechanism in deep learning is the implement the action of specifically concentrating on some relevant task while diminishing others. The attention mechanism for NLP uses neural architectures to dynamically highlight the relevant sequence of textual elements of the input information.

The most representative categories are basic multi-dimensional attention, hierarchical attention, self-attention, memory-based attention, and task-specific attention. Application of deep learning for NLP are Text Classification, Language Modeling, Speech Recognition, Caption Generation, Machine Translation, Document Summarization, and Question Answering. Future directions of attention mechanism in NLP are Neural-Symbolic Learning and Reasoning, Attention for Deep Networks Investigation, Unsupervised Learning With AttentionAttention for Outlier Detection, and Sample Weighing.