A Real-Time Emotion-Aware System Based on Wireless Body Area Network for IoMT Applications

作者：Chang Li, Yingchi Mao, Qian Huang, Weiliang Xie, Xiaoming He, Jie Wu · 发表于：IEEE Internet of Things Journal · 年份：2024 · DOI：10.1109/jiot.2024.3458976 · 被引用次数：13 · 研究领域：Emotion and Mood Recognition、EEG and Brain-Computer Interfaces、Wireless Body Area Networks

The Internet of Medical Things (IoMT) stimulates the development of intelligent medical applications. As mental disorders become a global problem, emotion recognition has received widespread attention, as it can contribute to more comprehensive mental health monitoring and psychological assessment. Physiological signal-based emotion-aware monitoring is a particularly promising application due to its noninvasive and objective data collection. Recently, multimodal emotion recognition has been enhanced with wireless body area network (WBAN) access to IoMT, where wireless medical sensors are interconnected and abundant signals are acquired conveniently. However, how to synthesize these multisource physiological signals to facilitate emotion recognition is a challenging problem due to their heterogeneity and interference. To solve this problem, we propose a real-time differential multimodal transformer (Diff-MT), where the main components are the differential hyperinformation extraction (DHE) module, the multimodal global cross-attention encoder (MGCE), and the difference-augmented feature fusion (DFF). Ultimately, we endow the system with emotional awareness and distribute the state to IoMT devices. Extensive experiments demonstrate that the proposed Diff-MT exhibits superior performance compared to existing methods on the WESAD and DEAP datasets and is appropriate for IoMT-based healthcare.