Research Area: Machine Learning
With the continuous development of portable noninvasive human sensor technologies such as brain–computer interfaces (BCI), multimodal emotion recognition has attracted increasing attention in the area of affective computing. This paper primarily discusses the progress of research into multimodal emotion recognition based on BCI and reviews three types of multimodal affective BCI (aBCI): aBCI based on a combination of behavior and brain signals, aBCI based on various hybrid neurophysiology modalities and aBCI based on heterogeneous sensory stimuli. For each type of aBCI, we further review several representative multimodal aBCI systems, including their design principles, paradigms, algorithms, experimental results and corresponding advantages. Finally, we identify several important issues and research directions for multimodal emotion recognition based on BCI.
Keywords:
emotion recognition
multimodal fusion
brain–computer interface
affective computing
Author(s) Name: Zhipeng He,Zina Li,Fuzhou Yang,Lei Wang,Jingcong Li,Chengju Zhou, Jiahui Pan
Journal name: Brain Sciences
Conferrence name:
Publisher name: MDPI
DOI: https://doi.org/10.3390/brainsci10100687
Volume Information: Volume: 10
Paper Link: https://www.mdpi.com/2076-3425/10/10/687
