MimicX: Policy-in-the-Loop Supervision Refinement for Video-Driven Humanoid Motion Tracking
cs.RO, cs.AI
Submitted: 2026-10-06
Updated: 2026-10-06
Comments: 33 pages, 26 figures, 19 tables, including references and appendices. Project website: https://nebulis-lab.com/MimicX
Code: https://github.com/NEBULIS-Lab/MimicX
Project page: https://nebulis-lab.com/MimicX
License: http://creativecommons.org/licenses/by/4.0/
The gist: Human videos provide rich motion targets for humanoid learning, yet visually plausible references can still produce persistent failures under physics-based execution.
Terminology
Abstract
Human videos provide rich motion targets for humanoid learning, yet visually plausible references can still produce persistent failures under physics-based execution. These failures reveal where training supervision should change. We present MimicX, a policy-in-the-loop framework that uses execution feedback to refine video-driven humanoid motion tracking. Starting from reconstructed and retargeted motion, MimicX localizes difficult transitions and affected body regions, then jointly adapts tracking objectives and the reset curriculum for policy continuation. Repeated rollout verification selects execution-priority improvements subject to tracking guards. Across four core video tasks, MimicX consistently improves tracking accuracy and Robust Execution Horizon relative to the Fixed Reference baseline. Task-averaged results show a 25.7% reduction in body-tracking error and a 255.6% increase in execution horizon. Additional video, motion-reference, and collision-scene studies evaluate the method beyond the core tasks, while MimicX-HLoop accelerates feedback through heterogeneous execution. Overall, MimicX turns policy failure into actionable supervision for deciding what to refine and which refinement to retain.
Sources
- Retargeting Matters: General Motion Retargeting for Humanoid Motion Tracking
- Towards Adaptable Humanoid Control via Adaptive Motion Tracking
- BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusion
- Proximal Policy Optimization Algorithms
- mjlab: A Lightweight Framework for GPU-Accelerated Robot Learning
- MeshMimic: Geometry-Aware Humanoid Motion Learning through 3D Scene Reconstruction
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies