Research Area:  Machine Learning

Abstract:

The current generation of neural network-based natural language processing models excels at learning from large amounts of labelled data. Given these capabilities, natural language processing is increasingly applied to new tasks, new domains, and new languages.Current models, however, are sensitive to noise and adversarial examples and prone too verfitting. This brittleness, together with the cost of attention, challenges the supervised learning paradigm.Transfer learning allows us to leverage knowledge acquired from related data in order to improve performance on a target task.Implicit transfer learning in the form of pretrained word representations has been a common component in natural language processing. In this dissertation, we argue that more explicit transfer learning is key to deal with the dearth of training data and to improve downstream performance of natural language processing models. We show experimental results transferring knowledge from related domains, tasks, and languages that support this hypothesis.

Name of the Researcher:  Sebastian Ruder

Name of the Supervisor(s):   John G. Breslin, Parsa Ghaffari

Year of Completion:  2019

University:  NATIONAL UNIVERSITY OF IRELAND

Thesis Link:   Home Page Url