Semi-Supervised Dual-Stream Self-Attentive Adversarial Graph Contrastive Learning for Cross-Subject EEG-Based Emotion Recognition

作者：Weishan Ye, Zhiguo Zhang, Fei Teng, Min Zhang, Jianhong Wang, Dong Ni, Fali Li, Peng Xu, Zhen Liang · 发表于：IEEE Transactions on Affective Computing · 年份：2024 · DOI：10.1109/taffc.2024.3433470 · 被引用次数：45 · 研究领域：Emotion and Mood Recognition、EEG and Brain-Computer Interfaces、Face and Expression Recognition

Electroencephalography (EEG) is an objective tool for emotion recognition with promising applications. However, the scarcity of labeled data remains a major challenge in this field, limiting the widespread use of EEG-based emotion recognition. In this paper, a semi-supervisedDual-streamSelf-attentiveAdversarialGraphContrastive learning framework (termed asDS-AGC) is proposed to tackle the challenge of limited labeled data in cross-subject EEG-based emotion recognition. The DS-AGC framework includes two parallel streams for extracting non-structural and structural EEG features. The non-structural stream incorporates a semi-supervised multi-domain adaptation method to alleviate distribution discrepancy among labeled source domain, unlabeled source domain, and unknown target domain. The structural stream develops a graph contrastive learning method to extract effective graph-based feature representation from multiple EEG channels in a semi-supervised manner. Further, a self-attentive fusion module is developed for feature fusion, sample selection, and emotion recognition, which highlights EEG features more relevant to emotions and data samples in the labeled source domain that are closer to the target domain. Extensive experiments are conducted on four benchmark databases (SEED, SEED-IV, SEED-V, and FACED) using a semi-supervised cross-subject leave-one-subject-out cross-validation evaluation protocol. The results show that the proposed model outperforms existing methods under d...