Research Area:  Machine Learning

Abstract:

In this work, we perform unsupervised learning of representations by maximizing mutual information between an input and the output of a deep neural network encoder. Importantly, we show that structure matters: incorporating knowledge about locality of the input to the objective can greatly influence a representations suitability for downstream tasks. We further control characteristics of the representation by matching to a prior distribution adversarially. Our method, which we call Deep InfoMax (DIM), outperforms a number of popular unsupervised learning methods and competes with fully-supervised learning on several classification tasks. DIM opens new avenues for unsupervised learning of representations and is an important step towards flexible formulations of representation-learning objectives for specific end-goals.

Keywords:  
Deep Learning
Representations
Mutual Information Estimation And Maximization
Machine Learning

Author(s) Name:  R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, Yoshua Bengio

Publisher name:  arxiv

DOI:  10.48550/arXiv.1808.06670

Volume Information:  Volume 2018

Paper Link:   https://arxiv.org/abs/1808.06670