Research Area:  Machine Learning

Abstract:

Many strategies have been put forward for training deep network models, however, stacking of several layers of nonlinearities typically results in poor propagation of gradients and activations. The purpose of this paper is to explore the use of two steps strategy where initial deep learning model is obtained first by unsupervised learning and then optimizing the initial deep learning model by fine tuning. A number of fine tuning algorithms are explored in this work for optimizing deep learning models. This includes proposing a new algorithm where Backpropagation with adaptive gain algorithm is integrated with Dropout technique and the authors evaluate its performance in the fine tuning of the pre trained deep network. The parameters of deep neural networks are first learnt using greedy layer-wise unsupervised pre training. The proposed technique is then used to perform supervised fine tuning of the deep neural network model. Extensive experimental study is performed to evaluate the performance of the proposed fine tuning technique on three benchmark data sets: USPS, Gisette and MNIST. The authors have tested the approach on varying size data sets which include randomly chosen training samples of size 20, 50, 70 and 100 percent from the original data set. Through extensive experimental study, it is concluded that the two steps strategy and the proposed fine tuning technique significantly yield promising results in optimization of deep network models.

Keywords:  
Deep Network Models
Fine Tuning
Machine Learning
Deep Learning

Author(s) Name:  M. Arif Wani, Saduf Afzal

Publisher name:  Emerald Publishing Limited

DOI:  https://doi.org/10.1108/IJICC-06-2017-0070

Volume Information:  Volume 11 Issue 3

Paper Link:   https://www.emerald.com/insight/content/doi/10.1108/IJICC-06-2017-0070/full/html