Mutilmodal Feature Extraction and Attention-based Fusion for Emotion Estimation in Videos

作者：Shu, Tao, Xinke Wang, Ruotong Wang, Chuang Chen, Yixin Zhang, Xiao Sun · 发表于：arXiv (Cornell University) · 年份：2023 · DOI：10.48550/arxiv.2303.10421 · 被引用次数：5 · 研究领域：Emotion and Mood Recognition、Human Pose and Action Recognition、Sentiment Analysis and Opinion Mining

The continuous improvement of human-computer interaction technology makes it possible to compute emotions. In this paper, we introduce our submission to the CVPR 2023 Competition on Affective Behavior Analysis in-the-wild (ABAW). Sentiment analysis in human-computer interaction should, as far as possible Start with multiple dimensions, fill in the single imperfect emotion channel, and finally determine the emotion tendency by fitting multiple results. Therefore, We exploited multimodal features extracted from video of different lengths from the competition dataset, including audio, pose and images. Well-informed emotion representations drive us to propose a Attention-based multimodal framework for emotion estimation. Our system achieves the performance of 0.361 on the validation dataset. The code is available at [https://github.com/xkwangcn/ABAW-5th-RT-IAI].