Research Area:  Machine Learning

Abstract:

Incremental learning is one of the most important abilities of human beings. In the age of artificial intelligence, it is the key task to make neural network models as powerful as human beings, to achieve the ability to continuously acquire, fine-tune, and accumulate knowledge while simultaneously avoid catastrophic forgetting. In recent years, by virtue of deep neural networks, incremental learning has been attracting a great deal of attention in the field of computer vision. In this paper, we systematically review the current development of incremental learning and give the overall taxonomy of the incremental learning methods. Specifically, three kinds of mainstream methods, i.e., parameter regularization-based approaches, knowledge distillation-based approaches, and dynamic architecture-based approaches, are surveyed, summarized, and discussed in detail. Furthermore, we comprehensively analyze the performance of data-permuted incremental learning, class-incremental learning, and multi-modal incremental learning on widely used datasets, covering a broad of incremental learning scenarios for image classification and semantic segmentation. Lastly, we point out some possible research directions and inspiring suggestions for incremental learning in the field of computer vision.

Keywords:  
Incremental learning
Catastrophic forgetting
Parameter regularization
Knowledge distillation
Dynamic architecture
Computer vision
Machine Learning
Deep Learning

Author(s) Name:  Hao Liu, Yong Zhou, Bing Liu, Jiaqi Zhao, Rui Yao & Zhiwen Shao

Publisher name:  Springer

DOI:  10.1007/s10462-022-10294-2

Volume Information:  

Paper Link:   https://link.springer.com/article/10.1007/s10462-022-10294-2