MMM: Generative Masked Motion Model

作者：Ekkasit Pinyoanuntapong, Pu Wang, Minwoo Lee, Chen Chen · 发表于：Computer Vision and Pattern Recognition · 年份：2023 · DOI：10.1109/CVPR52733.2024.00153 · 被引用次数：152 · 研究领域：Computer Science

Recent advances in text-to-motion generation using dif-fusion and autoregressive models have shown promising re-sults. However, these models often suffer from a trade-off between real-time performance, high fidelity, and motion editability. To address this gap, we introduce MMM, a novel yet simple motion generation paradigm based on Masked Motion Model. MMM consists of two key components: (1) a motion tokenizer that transforms 3D human motion into a sequence of discrete tokens in latent space, and (2) a conditional masked motion transformer that learns to predict randomly masked motion tokens, conditioned on the pre-computed text tokens. By attending to motion and text to-kens in all directions, MMM explicitly captures inherent dependency among motion tokens and semantic mapping between motion and text tokens. During inference, this al-lows parallel and iterative decoding of multiple motion to-kens that are highly consistent with fine-grained text de-scriptions, therefore simultaneously achieving high-fidelity and high-speed motion generation. In addition, MMM has innate motion editability. By simply placing mask tokens in the place that needs editing, MMM automatically fills the gaps while guaranteeing smooth transitions between editing and non-editing parts. Extensive experiments on the HumanML3D and KIT-ML datasets demonstrate that MMM surpasses current leading methods in generating high-quality motion (evidenced by superior FID scores of 0.08 and 0.429), while offering ad...