KungfuAthleteBot: learning high-dynamic humanoid motion from video with unified robust recovery
cs.RO
Submitted: 2026-10-02
Updated: 2026-10-06
Project page: https://kungfuathletebot.github.io
License: http://creativecommons.org/licenses/by-sa/4.0/
The gist: Video is an abundant, inexpensive source of human motion data that is rich in extreme athletic behaviors.
Abstract
Video is an abundant, inexpensive source of human motion data that is rich in extreme athletic behaviors. Making it usable for humanoid robots, however, is not a matter of simply retargeting a reconstructed trajectory: video-derived motion is physically inconsistent, devoid of actuation information, and says nothing about failure or recovery. We present KungfuAthleteBot (KAB), a framework that treats learning high-dynamic motion from video as the central problem and resolves each of these three failure modes in turn. (C1) We build the KungfuAthlete dataset from videos of national-level martial artists and introduce a physics-guided parabolic trajectory correction that removes height floating, ground penetration, and high-frequency jitter from reconstructed aerial and landing phases. (C2) Because video carries no force information, strict tracking of a reconstructed trajectory is dynamically infeasible, and error-driven initialization keeps re-launching the policy from infeasible aerial poses. We introduce physics-driven pseudo-low-kinetic-energy (LKE) sampling, our central mechanism for making such references learnable: it biases initialization towards dynamically feasible states, letting the policy discover feasible actuation patterns instead of imitating infeasible ones. (C3) Finally, we introduce a direct training paradigm in which disturbance rejection and fall recovery are learned inside the same policy that tracks the video motion, requiring no recovery reference data and no manual mode switching. On a humanoid robot, KAB learns dynamic skills from video and recovers from arbitrary falls in about 0.7 s, the fastest reported recovery for a unified policy. Ablations on the unified policy confirm the necessity of its components, supporting the view that repairing and compensating video data, rather than only collecting more of it, is what unlocks high-dynamic humanoid skills.
Sources
- Retargeting Matters: General Motion Retargeting for Humanoid Motion Tracking
- HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpit
- GMT: General Motion Tracking for Humanoid Whole-Body Control
- ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills
- Learning Getting-Up Policies for Real-World Humanoid Robots
- Learning Humanoid Standing-up Control across Diverse Postures
- ExBody2: Advanced Expressive Humanoid Whole-Body Control
- FARM: Frame-Accelerated Augmentation and Residual Mixture-of-Experts for Physics-Based High-Dynamic Humanoid Control
- PHUMA: Physically Reliable Humanoid Locomotion Dataset
- BFM-Zero: A Promptable Behavioral Foundation Model for Humanoid Control Using Unsupervised Reinforcement Learning
- BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusion
- SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Control
- AMASS: Archive of Motion Capture as Surface Shapes
- SafeFall: Learning Protective Control for Humanoid Robots
- Agility Meets Stability: Versatile Humanoid Control with Heterogeneous Data
- Learning Sim-to-Real Humanoid Locomotion in 15 Minutes
- World-Grounded Human Motion Recovery via Gravity-View Coordinates
- ZEST: Zero-shot Embodied Skill Transfer for Athletic Robot Control
- Learning to Get Up Across Morphologies: Zero-Shot Recovery with a Unified Humanoid Policy
- RobotDancing: Residual-Action Reinforcement Learning Enables Robust Long-Horizon Humanoid Motion Tracking
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies