EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Data
cs.RO, cs.AI
Submitted: 2026-07-08
Updated: 2026-10-13
Code: https://github.com/rail-berkeley/oculus_reader
License: http://creativecommons.org/licenses/by/4.0/
The gist: Egocentric human data offers scalable supervision for robot manipulation.
Terminology
Abstract
Egocentric human data offers scalable supervision for robot manipulation. However, behavior cloning entangles transferable content like objects, scenes, and task semantics, with non-transferable factors like human morphology, head motion, and behavioral style. We study whether World Action Models (WAMs) provide a better training signal by requiring policies to predict not only actions, but also how the scene evolves. The central question is what world representation best enables human-to-robot transfer. We hypothesize that an effective world target should abstract appearance, capture agent-invariant physical effects, and separate camera motion from environment change. We introduce EgoWAM, a controlled human-robot co-training framework that fixes the policy backbone, action head, and data mixture while varying only the world prediction target, comparing Pixel, DINO, and 3D motion flow. Across three real-world bimanual tasks, WAM co-training scales more effectively with in-the-wild egocentric human data than behavior cloning. Pixel-based prediction transfers weakly, while DINO and 3D flow yield substantial gains: DINO improves out-of-distribution object and scene generalization by up to 4x, and 3D flow improves in-domain performance by 20-30%. More details: https://gatech-rl2.github.io/egowam.github.io
Sources
- EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video
- Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
- Emergence of Human to Robot Transfer in Vision-Language-Action Models
- EgoMimic: Scaling Imitation Learning via Egocentric Video
- Humanoid Policy ~ Human Policy
- EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data
- EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the World
- World Action Models are Zero-shot Policies
- Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning
- Unified Video Action Model
- Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets
- Diffusion Transformers with Representation Autoencoders
- SpatialTrackerV2: 3D Point Tracking Made Easy
- Track4World: Feedforward World-centric Dense 3D Tracking of All Pixels
- Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformers
- R3M: A Universal Visual Representation for Robot Manipulation
- Pre-training Auto-regressive Robotic Models with 4D Representations
- Whole-Body Conditioned Egocentric Video Prediction
- EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videos
- H-RDT: Human Manipulation Enhanced Bimanual Robotic Manipulation
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies