METIS: Multi-Source Egocentric Training for Integrated Dexterous Vision-Language-Action Model
cs.RO, cs.CV
Submitted: 2025-11-21
Updated: 2026-10-13
Project page: https://aureleopku.github.io/METIS
License: http://creativecommons.org/licenses/by/4.0/
The gist: Building a generalist robot that can perceive, reason, and act across diverse tasks remains an open challenge, especially for dexterous manipulation.
Terminology
Abstract
Building a generalist robot that can perceive, reason, and act across diverse tasks remains an open challenge, especially for dexterous manipulation. A major bottleneck lies in the scarcity of large-scale, action-annotated data for dexterous skills, as teleoperation is difficult and costly. Human data, with its vast scale and diverse manipulation behaviors, provides rich priors for learning robotic actions. While prior works have explored leveraging human demonstrations, they are often constrained by limited scenarios and a large visual gap between human and robots. To eliminate these limitations, we propose METIS, a vision-language-action (VLA) model for dexterous manipulation pretrained on multi-source egocentric datasets. We first construct EgoAtlas, which integrates large-scale human and robotic data from multiple sources, all unified under a consistent action space. We further extract motion-aware dynamics, a compact and discretized motion representation, which provides efficient and expressive supervision for VLA training. Built upon them, METIS integrates reasoning and acting into a unified framework, enabling effective deployment to downstream dexterous manipulation tasks. Our method demonstrates exceptional dexterous manipulation capabilities, achieving highest average success rate in six real-world tasks. Experimental results also highlight the superior generalization and robustness to out-of-distribution scenarios. These findings emphasize METIS as a promising step toward a generalist model for dexterous manipulation.
Sources
- H-RDT: Human Manipulation Enhanced Bimanual Robotic Manipulation
- $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
- AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems
- UniVLA: Learning to Act Anywhere with Task-centric Latent Actions
- CordViP: Correspondence-based Visuomotor Policy for Dexterous Manipulation in Real-World
- EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video
- An Embodied Generalist Agent in 3D World
- DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset
- OpenVLA: An Open-Source Vision-Language-Action Model
- Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success
- RAM: Retrieval-Based Affordance Transfer for Generalizable Zero-Shot Robotic Manipulation
- Vision-Language Foundation Models as Effective Robot Imitators
- Learning Geometry-Aware Nonprehensile Pushing and Pulling with Dexterous Hands
- Leveraging Pretrained Latent Representations for Few-Shot Imitation Learning on a Dexterous Robotic Hand
- OneTwoVLA: A Unified Vision-Language-Action Model with Adaptive Reasoning
- HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model
- Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videos
- DINOv2: Learning Robust Visual Features without Supervision
- Humanoid Policy ~ Human Policy
- Bimanual Dexterity for Complex Tasks
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies