Research Area:  Machine Learning

Abstract:

Inferring commonsense knowledge is a key challenge in machine learning. Due to the sparsity of training data, previous work has shown that supervised methods for commonsense knowledge mining underperform when evaluated on novel data. In this work, we develop a method for generating commonsense knowledge using a large, pre-trained bidirectional language model. By transforming relational triples into masked sentences, we can use this model to rank a tripls validity by the estimated pointwise mutual information between the two entities. Since we do not update the weights of the bidirectional model, our approach is not biased by the coverage of any one commonsense knowledge base. Though we do worse on a held-out test set than models explicitly trained on a corresponding training set, our approach outperforms these methods when mining commonsense knowledge from new sources, suggesting that our unsupervised technique generalizes better than current supervised approaches.

Keywords:  
Commonsense Knowledge Mining
Pretrained Models
Supervised methods
Unsupervised technique
Supervised approaches

Author(s) Name:  Joe Davison, Joshua Feldman, Alexander Rush

Publisher name:  ACL

DOI:  10.18653/v1/D19-1109

Volume Information:  

Paper Link:   https://aclanthology.org/D19-1109/