Praxis: Distilling Physical Interaction Priors from Egocentric Videos for Generalizable Whole-Body Manipulation
cs.RO
Submitted: 2026-09-25
Updated: 2026-09-25
Code: https://github.com/teal024/FoundationPose-plus-plus
Project page: https://edem-ai.github.io/Praxis
License: http://creativecommons.org/licenses/by/4.0/
The gist: Mobile humanoid manipulation requires both reaching a usable workspace and preserving precise hand-object interactions as object poses and contact conditions change.
Terminology
Abstract
Mobile humanoid manipulation requires both reaching a usable workspace and preserving precise hand-object interactions as object poses and contact conditions change. Learning these behaviors from limited task-specific data remains challenging. To bridge this gap, we introduce Praxis, a whole-body manipulation framework that combines physical interaction priors from one-shot egocentric video demonstrations with closed-loop posture calibration and online perception. The framework coordinates three stages: vision-language-guided navigation toward target objects, closed-loop posture calibration to align the arm-hand workspace, and dexterous manipulation with synchronized upper- and lower-body control. Online visual feedback re-grounds demonstrated interaction geometry under new object poses and scene configurations, while tactile feedback adapts hand motions to actual contact conditions. Each manipulation skill is specified by one human demonstration, without task-specific manipulation-policy retraining. Experiments across five long-horizon manipulation tasks demonstrate spatial, visual, and cross-object generalization, as well as recovery from external physical disturbances across all three stages.
Sources
- $\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization
- GR00T N1: An Open Foundation Model for Generalist Humanoid Robots
- Gemini Robotics: Bringing AI into the Physical World
- Humanoid Locomotion and Manipulation: Current Progress and Challenges in Control, Planning, and Learning
- TrajBooster: Boosting Humanoid Whole-Body Manipulation via Trajectory-Centric Learning
- HDMI: Learning Interactive Humanoid Whole-Body Control from Human Videos
- KungfuBot: Physics-Based Humanoid Whole-Body Control for Learning Highly-Dynamic Skills
- Emergence of Human to Robot Transfer in Vision-Language-Action Models
- WoCoCo: Learning Whole-Body Humanoid Control with Sequential Contacts
- FALCON: Learning Force-Adaptive Humanoid Loco-Manipulation
- HumanPlus: Humanoid Shadowing and Imitation from Humans
- OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning
- Real-Time Verification of Embodied Reasoning for Generative Skill Acquisition
- Learning Precise Affordances from Egocentric Videos for Robotic Manipulation
- HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpit
- Proximal Policy Optimization Algorithms
- MediaPipe Hands: On-device Real-time Hand Tracking
- LISA: Reasoning Segmentation via Large Language Model
- WholeBodyVLA: Towards Unified Latent VLA for Whole-Body Loco-Manipulation Control
- HMC: Learning Heterogeneous Meta-Control for Contact-Rich Loco-Manipulation
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies