Research Area: Machine Learning
Neural architecture search (NAS) has become increasingly popular in the deep learning community recently, mainly because it can provide an opportunity to allow interested users without rich expertise to benefit from the success of deep neural networks (DNNs). However, NAS is still laborious and time-consuming because a large number of performance estimations are required during the search process of NAS, and training DNNs is computationally intensive. To solve this major limitation of NAS, improving the computational efficiency is essential in the design of NAS. However, a systematic overview of computationally efficient NAS (CE-NAS) methods still lacks. To fill this gap, we provide a comprehensive survey of the state-of-the-art on CE-NAS by categorizing the existing work into proxy-based and surrogate-assisted NAS methods, together with a thorough discussion of their design principles and a quantitative comparison of their performances and computational complexities. The remaining challenges and open research questions are also discussed, and promising research topics in this emerging field are suggested.
Keywords:
Neural architecture search (NAS)
One-shot NAS
Surrogate model
Bayesian optimization
Performance predictor
Author(s) Name: Shiqing Liu, Haoyu Zhang,
Journal name: Journal of Automation and Intelligence
Conferrence name:
Publisher name: Elsevier
DOI: 10.1016/j.jai.2022.100002
Volume Information: Volume 1
