Morphometric Imitation: From Morphology and Contact Aware Hand Retargeting to Sim-to-Real Visuomotor Policy
cs.RO
Submitted: 2026-09-23
Updated: 2026-09-25
Comments: 24 pages, 10 figures
License: http://creativecommons.org/licenses/by/4.0/
The gist: Human hand-object interactions (HOIs) provide a rich source of demonstrations for dexterous manipulation, but learning directly from them presents challenges in bridging morphology gaps, ensuring
Terminology
Abstract
Human hand-object interactions (HOIs) provide a rich source of demonstrations for dexterous manipulation, but learning directly from them presents challenges in bridging morphology gaps, ensuring dynamical feasibility, and sim-to-real deployment. We present Morphometric Imitation, a three-stage framework that transforms reconstructed HOIs into zero-shot sim-to-real visuomotor policies. First, morphometric optimization (MMO) kinematically retargets human motion across hand morphologies while preserving demonstrated contacts. Second, residual reinforcement learning (RL) refines the kinematic reference using object pose and contact information from the human motion to produce dynamically feasible robot demonstrations. Third, these demonstrations are distilled into visuomotor policies. Across three robot hands and ten HOIs, MMO outperforms five baselines in contact F1, improving on the strongest ones by 8 to 28 points, while improving the success rate of downstream dynamic retargeting by as much as 35 points. On a Sharpa hand, the visuomotor policies achieve 89.3% zero-shot success in 300 real-world trials on 30 objects spanning 10 categories. Project page: https://morphometricimitation.github.io
Sources
- EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data
- AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System
- TopoRetarget: Interaction-Preserving Retargeting for Dexterous Manipulation
- A Minimalist Retargeting-Guided Reinforcement Learning Recipe for Dexterous Manipulation
- SPIDER: Scalable Physics-Informed Dexterous Retargeting
- Object-Centric Dexterous Manipulation from Human Motion Data
- Hand-Object Interaction Pretraining from Videos
- Crossing the Human-Robot Embodiment Gap with Sim-to-Real RL using One Human Demonstration
- DexMachina: Functional Retargeting for Bimanual Dexterous Manipulation
- SimToolReal: An Object-Centric Policy for Zero-Shot Dexterous Tool Manipulation
- Geometric Retargeting: A Principled, Ultrafast Neural Hand Retargeting Algorithm
- DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation
- Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations
- Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learning
- Open-TeleVision: Teleoperation with Immersive Active Visual Feedback
- Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on Youtube
- ResPilot: Teleoperated Finger Gaiting via Gaussian Process Residual Learning
- Visual Imitation Enables Contextual Humanoid Control
- Embodied Hands: Modeling and Capturing Hands and Bodies Together
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies