Tensor Correlation Fusion for Multimodal Physiological Signal Emotion Recognition

作者：Jian Shen, Kexin Zhu, Huakang Liu, Jinwen Wu, Kang Wang, Qunxi Dong · 发表于：IEEE Transactions on Computational Social Systems · 年份：2024 · DOI：10.1109/tcss.2024.3406988 · 被引用次数：19 · 研究领域：Emotion and Mood Recognition、EEG and Brain-Computer Interfaces、Non-Invasive Vital Sign Monitoring

As an essential challenge within the realm of affective computing, emotion recognition assumes a vital role in bestowing computers with a higher level and comprehensive intelligence. Furthermore, it has emerged as a crucial research topic in both human–computer interaction (HCI) and medical rehabilitation related to mental illnesses. However, the related fusion studies for modeling physiological signals in emotion recognition are less based on multimodal coordinated representation and lack the exploration of multimodal physiological signal correlation. In this article, we propose a tensor correlation fusion framework for emotion recognition based on multimodal physiological signals. After extracting effective features from various physiological signals, the coordinated representation module of the framework first simultaneously learns the linear correlation of all input physiological signals based on the covariance tensor. An optimized solution strategy is constructed to obtain the coordinated representation corresponding to each physiological signal. Finally, an emotion recognition module fuses the correlation information of the coordinated representation of different physiological signals as input to the emotion recognition classifier. This framework constructs coordinated representation by introducing a strategy to simultaneously capture the correlation among multiple physiological signals, providing a fresh perspective with a well-defined mathematical foundation for the f...