Reversible Graph Neural Network-based Reaction Distribution Learning for Multiple Appropriate Facial Reactions Generation

作者：Tong Xu, Micol Spitale, Hao Tang, Lu Liu, Hatice Güneş, Siyang Song · 发表于：IEEE Transactions on Affective Computing · 年份：2026 · DOI：10.1109/taffc.2026.3682774 · 被引用次数：5 · 研究领域：Emotion and Mood Recognition、Face recognition and analysis、Face Recognition and Perception

Generating facial reactions in a human-human dyadic interaction is complex and highly dependent on the context since more than one facial reactions can be appropriate for the speaker's behaviour. This has challenged existing machine learning (ML) methods, whose training strategies enforce models to reproduce a specific (not multiple) facial reaction from each input speaker behaviour. This paper proposes the first multiple appropriate facial reaction generation (MAFRG) framework which re-formulates the one-to-many mapping facial reaction generation problem as a one-to-one mapping problem. This means that we approach this problem by considering generating a distribution of listeners' appropriate facial reactions instead of multiple different appropriate facial reactions, i.e., ‘many’ appropriate facial reaction labels are summarised as ‘one’ distribution label during training. Our model consists of a perceptual processor, a cognitive processor, and a motor processor. The motor processor is implemented with a novel Reversible Multi-dimensional Edge Graph Neural Network (REGNN). This allows us to obtain a distribution of appropriate real facial reactions during the training process, enabling the cognitive processor to be trained to predict the appropriate facial reaction distribution. At the inference stage, the REGNN decodes an appropriate facial reaction by using the predicted distribution as input. Experimental results demonstrate that our approach outperforms existing models ...