Movens-Zero: Scaling Humanoid Motion Foundation Models with Large-Scale Human Videos
cs.RO, cs.CV
Submitted: 2026-10-30
Updated: 2026-10-30
Comments: Technical report. 86 pages, 31 figures, 15 tables. Project page: https://movens-zero.github.io
Code: https://github.com/IDEA-Research/Motion-X
Project page: https://movens-zero.github.io
License: http://creativecommons.org/licenses/by/4.0/
The gist: Scaling humanoid motion foundation models is constrained by the scarcity of diverse, robot-executable data and by the difficulty of jointly learning from heterogeneous multimodal inputs while…
Terminology
Abstract
Scaling humanoid motion foundation models is constrained by the scarcity of diverse, robot-executable data and by the difficulty of jointly learning from heterogeneous multimodal inputs while supporting real-time closed-loop control. We present Movens-5M, constructed from raw Internet videos through clip generation, quality filtering, semantic annotation, motion recovery, and physical refinement with humanoid retargeting and execution verification. It contains 15 million human-motion sequences and 5 million physically executable humanoid-motion sequences spanning over 10,000 hours, aligned with text, RGB video, speech, and music. Building on this data, we introduce Movens-Zero, a scalable understanding-generation-execution architecture that supports joint learning across modalities and real-time closed-loop execution. Task-mixed flow-matching training jointly optimizes a single action expert across seven generation tasks, enabling direct motion generation from text, speech, music, and video. Multimodal Real-Time Chunking connects overlapping chunks under continuously updated instructions and robot states for continuous closed-loop execution. We further introduce Movens-Intent, a four-track benchmark with 8,192 expert-verified condition-motion pairs across text, speech, music, and video. Experiments show consistent gains across all four modalities when scaling execution-verified data from 1% to 100% and action-expert capacity from 64M to 911M parameters, while demonstrating that data quality and cross-modal supervision improve generation and execution.
Sources
- Retargeting Matters: General Motion Retargeting for Humanoid Motion Tracking
- Qwen3-VL Technical Report
- OpenT2M: No-frill Motion Generation with Open-source,Large-scale, High-quality Data
- HoloMotion-1 Technical Report
- Gemini: A Family of Highly Capable Multimodal Models
- SnapMoGen: Human Motion Generation from Expressive Texts
- OMG: Omni-Modal Motion Generation for Generalist Humanoid Control
- UniAct: Unified Motion Generation and Action Streaming for Humanoid Robots
- Adam: A Method for Stochastic Optimization
- PHUMA: Physically Reliable Humanoid Locomotion Dataset
- AnyMo: Scaling Any-Modality Conditional Motion Generation with Masked Modeling
- From Language to Locomotion: Retargeting-free Humanoid Control via Motion Latent Guidance
- VersatileMotion: A Unified Framework for Motion Synthesis and Comprehension
- SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Control
- Learning from Massive Human Videos for Universal Humanoid Pose Control
- Humanoid-GPT: Scaling Data and Structure for Zero-Shot Motion Tracking
- Kimodo: Scaling Controllable Human Motion Generation
- World-Grounded Human Motion Recovery via Gravity-View Coordinates
- GPC: Large-Scale Generative Pretraining for Transferable Motor Control
- OmniXtreme: Breaking the Generality Barrier in High-Dynamic Humanoid Control
Related papers
- Rethinking Actions: Continuity-Aware Priors for World-Model Planning
- MOSAIC: Modular Obstacle and Spatial-Aware Framework for Manipulation In Cluttered Scenes
- SAGAS: Semantic-Aware Graph-Assisted Stitching for Offline Temporal Logic Planning
- DexSeed: Expanding Human Hand-Object Interactions into Robot Demonstrations for Dexterous Policy Learning
- LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
- CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation