Research Area: Machine Learning
With the development of visual social networks, the sentiment analysis of images has quickly emerged for opinion mining. Based on the observation that the sentiments conveyed by some images are related to salient objects in them, we propose a scheme for visual sentiment analysis that combines global and local information. First, the sentiment is predicted from the entire images. Second, it is judged whether there are salient objects in an image or not. If there are, sub-images are cropped from the entire image based on the detection window of the salient objects. Moreover, a CNN model is trained for the set of sub-images. Predictions of sentiments from entire images and sub-images are then fused together to obtain the final results. If no salient object is detected in the images, the sentiment predicted directly from entire images is used as the final result. The compared experimental results show that the proposed approach is superior to state-of-the-art algorithms. It also demonstrates that reasonably utilizing the local information could improve the performance for visual sentiment analysis.
Keywords:
Visual sentiment analysis
Salient objects
Local information
Global information
Author(s) Name: Lifang Wu, Mingchao Qi, Meng Jian & Heng Zhang
Journal name: Neural Processing Letters
Conferrence name:
Publisher name: Springer
DOI: 10.1007/s11063-019-10027-7
Volume Information: volume 51, pages: 2063–2075 (2020)
