Ges-QA: A Multidimensional Quality Assessment Dataset for Audio-to-3D Gesture Generation

作者：Zhilin Gao, Yunhao Li, Sijing Wu, Yuqin Cao, Huiyu Duan, Guangtao Zhai · 年份：2025 · DOI：10.1109/vcip67698.2025.11396818 · 被引用次数：1 · 研究领域：Music Technology and Sound Studies、Speech and Audio Processing、Music and Audio Processing

The Audio-to-3D-Gesture (A2G) task exhibits significant potential across domains including virtual reality, computer graphics, and 3D animation production. However, current evaluation metrics, such as Fréchet Gesture Distance or Beat Constancy, fail at reflecting the human preference of the generated 3D gestures. To cope with this problem, exploring human preference and an objective quality assessment metric for AI-generated 3D human gestures is becoming increasingly significant. In this paper, we introduce the Ges-QA dataset, which includes 1,400 samples with multidimensional scores for gesture quality and audio-gesture consistency. Moreover, we collect binary classification labels to determine whether the generated gestures match the emotions of the audio. Equipped with our Ges-QA dataset, we propose a multi-modal transformer-based neural network with 3 branches for video, audio and 3D skeleton modalities, which can score A2G contents in multiple dimensions. Comparative experimental results and ablation studies demonstrate that Ges-QAer yields state-of-the-art performance on our dataset.