Amazing technological breakthrough possible @S-Logix

Office Address

  • #5, First Floor, 4th Street Dr. Subbarayan Nagar Kodambakkam, Chennai-600 024 Landmark : Samiyar Madam
  • +91- 81240 01111

Social List

Dynamic Neural Networks: A Survey - 2021

Dynamic Neural Networks: A Survey

Survey Paper on Dynamic Neural Networks

Research Area:  Machine Learning


Dynamic neural network is an emerging research topic in deep learning. Compared to static models which have fixed computational graphs and parameters at the inference stage, dynamic networks can adapt their structures or parameters to different inputs, leading to notable advantages in terms of accuracy, computational efficiency, adaptiveness, etc. In this survey, we comprehensively review this rapidly developing area by dividing dynamic networks into three main categories: 1) sample-wise dynamic models that process each sample with data-dependent architectures or parameters; 2) spatial-wise dynamic networks that conduct adaptive computation with respect to different spatial locations of image data; and 3) temporal-wise dynamic models that perform adaptive inference along the temporal dimension for sequential data such as videos and texts. The important research problems of dynamic networks, e.g., architecture design, decision making scheme, optimization technique and applications, are reviewed systematically. Finally, we discuss the open problems in this field together with interesting future research directions.

Dynamic Neural Networks
Deep Learning
Machine Learning

Author(s) Name:  Yizeng Han; Gao Huang; Shiji Song; Le Yang; Honghui Wang; Yulin Wang

Journal name:  IEEE Transactions on Pattern Analysis and Machine Intelligence

Conferrence name:  

Publisher name:  IEEE

DOI:  DOI: 10.1109/TPAMI.2021.3117837

Volume Information:  2021