Timo: Taming Multimodal Diffusion Transformers for Human Motion Generation
cs.CV
Submitted: 2026-09-25
Updated: 2026-10-10
Project page: https://kyfafyd.wang/projects/timo
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Most existing human motion generation (HMG) methods use cross-attention modules to inject text semantics, but ignore the importance of bidirectional modeling between motion and text tokens, which
Terminology
Abstract
Most existing human motion generation (HMG) methods use cross-attention modules to inject text semantics, but ignore the importance of bidirectional modeling between motion and text tokens, which limits text comprehension. A straightforward idea is introducing multimodal diffusion transformers (MMDiT), which have shown effective joint text--visual modeling in vision generation, into HMG. However, we find that articulated motion is temporally coherent but weakly correlated across joints, in which directly applying an MMDiT with flow matching produces poorly coordinated and jerky motion. In this work, we propose Timo, a novel kinematics-aware MMDiT framework tailored for HMG. Timo combines fully shared multimodal attention for bidirectional text--motion modeling with flow matching, geometric and rotational-kinematics supervision that compares actual rotations and their changes over time, and a two-stage curriculum progressing from broad motion learning to detailed caption alignment. Further, we construct a benchmark of 40, 025 held-out clips from six public datasets spanning diverse actions, assessing six complementary dimensions under a common evaluator and scoring protocol. Our model substantially outperforms state-of-the-art methods in both quantitative and qualitative evaluations. Remarkably, Timo surpasses Kimodo on five of six dimensions, achieving a 40.8 % relative improvement in the average benchmark score. Project page: https://kyfafyd.wang/projects/timo. Demo page: https://timo.kyfafyd.wang.
Sources
- Training Deep Nets with Sublinear Memory Cost
- Classifier-Free Diffusion Guidance
- HunyuanVideo: A Systematic Framework For Large Video Generative Models
- Back to Basics: Let Denoising Generative Models Denoise
- The Quest for Generalizable Motion Generation: Data, Model, and Evaluation
- Embody 3D: A Large-scale Multimodal Motion and Behavior Dataset
- Absolute Coordinates Make Motion Generation Easy
- Kimodo: Scaling Controllable Human Motion Generation
- GLU Variants Improve Transformer
- Holistic-Motion2D: Scalable Whole-body Human Motion Generation in 2D Space
- HY-Motion 1.0: Scaling Flow Matching Models for Text-To-Motion Generation
- Motion Anything: Any to Motion Generation
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models