Research Area: Machine Learning
At present, in the research of multimodal human action recognition, the weighted fusion method with fixed weight is widely applied in the decision level fusion of most models. In this way, the weight is usually obtained from the original experience or traversal search, which is inaccurate or has a large amount of calculation, and ignores the different representation ability of various modal data for various classes of action information. With the help of the powerful decision-making ability of deep reinforcement learning, we propose a multimodal decision-making fusion weight allocation network based on deep reinforcement learning. This letter mainly discusses the design of the model, which involves the modeling of reinforcement learning problem in action recognition, the design of neural network and the selection of problem-solving scheme. Experimental results on NTU RGB + D and HMDB51 datasets show the effectiveness of the proposed method.
Keywords:
Reinforcement learning
Resource management
Data models
Signal processing algorithms
Task analysis
Neural networks
Decision making
Author(s) Name: Jiale Guo; Qiang Liu; Enqing Chen
Journal name: IEEE
Conferrence name:
Publisher name: IEEE
DOI: 10.1109/LSP.2021.3128379
Volume Information: Volume 29
