Research Area:  Machine Learning

Abstract:

During minibatch gradient-based optimization, the contribution of observations to the updating of the deep neural networks (DNN-s) weights for enhancing the discrimination of certain classes can be small, despite the fact that these classes may still have a large generalization error. This happens, for instance, due to overfitting, i.e. to classes whose error in the training set is negligible, or simply when the contributions of the misclassified observations to the updating of the weights associated with these classes cancel out. To alleviate this problem, a new criterion for identifying the so-called “neglected” classes during the training of DNNs, i.e. the classes which stop to optimize early in the training procedure, is proposed. Moreover, based on this criterion a novel cost function is proposed, that extends the cross-entropy loss using subclass partitions for boosting the generalization performance of the neglected classes. In this way, the network is guided to emphasize the extraction of features that are discriminant for the classes that are prone to being neglected during the optimization procedure. The proposed framework can be easily applied to improve the performance of various DNN architectures.

Keywords:  

Author(s) Name:  Nikolaos Gkalelis, Vasileios Mezaris

Publisher name:  ACM

DOI:  https://doi.org/10.1007/978-3-030-37731-1_19

Volume Information:  Pages 227–238

Paper Link:   https://dl.acm.org/doi/abs/10.1007/978-3-030-37731-1_19