Research Area:  Machine Learning

Abstract:

In recent years, deep learning models have been more and more widely used in various fields and have become a research hotspot for various tasks in artificial intelligence, but there are significant limitations in non-convex optimization problems. As a model training strategy for non-convex optimization, curriculum learning advocates that models learn in the order of easier to more difficult data, mimicking the basic idea of gradual human learning as they learn curriculum. This strategy has been widely used in the fields of computer vision, natural language processing, and reinforcement learning; it can effectively solve the non-convex optimization problem and improve the generalization ability and convergence speed of models. This paper first introduces the application of curriculum learning at three major levels: data, task, and model, and summarizes the evaluators designed using curriculum learning methods in various domains, including difficulty evaluators, training schedulers, and loss evaluators, which correspond to the three stages of difficulty evaluation, training schedule, and loss evaluation in the application of curriculum learning to model training. We also discuss how to choose an appropriate evaluation system and the differences between terms used in different types of research. Finally, we summarize five methods similar to curriculum learning in the field of machine learning and provide a summary and outlook of the curriculum learning evaluation system.

Keywords:  
Curriculum learning
Difficulty evaluator
Training scheduler
Loss evaluator

Author(s) Name:  Fengchun Liu, Tong Zhang, Chunying Zhang

Publisher name:  MDPI

DOI:  10.3390/electronics12071676

Volume Information:  Volume 12

Paper Link:   https://www.mdpi.com/2079-9292/12/7/1676