Native Action-Prior Learning from Videos for World Action Models
cs.CV, cs.RO
Submitted: 2026-10-02
Updated: 2026-10-02
Comments: Project Page: https://zhaochongan.github.io/projects/NAVA-WAM
Project page: https://zhaochongan.github.io/projects/NAVA-WAM
License: http://creativecommons.org/licenses/by/4.0/
The gist: World action models integrate future visual dynamics with robot action prediction, but their scalability remains limited by the need for action-annotated robot trajectories.
Abstract
World action models integrate future visual dynamics with robot action prediction, but their scalability remains limited by the need for action-annotated robot trajectories. Observation-only videos contain rich evidence about interaction dynamics, but existing approaches typically use them either to pretrain visual representations that must later be adapted for control, or to infer latent actions that are subsequently grounded to robot commands. We present NAVA-WAM, which introduces native action-prior learning by directly pretraining the action policy from observation-only videos, avoiding indirect representation-to-control transfer or a separate latent-action model. Our training consists of two stages. First, we pretrain on observation-only videos, where future-video flow-matching supervision over visual transitions is propagated through transition-structured joint attention to optimize the Action-DiT and learn action-relevant priors. Second, we use action-labeled demonstrations to post-train the Action-DiT for robot control through joint video--action flow matching, while asymmetric attention decouples the visual branch from iterative action denoising and enables efficient action-only inference. Extensive experiments show that NAVA-WAM consistently outperforms prior approaches under both in-distribution and out-of-distribution settings, while demonstrating strong action-label efficiency and effective real-robot generalization. These results establish native action-prior learning as an effective approach to directly pretrain action policies from observation-only videos, providing a scalable path beyond action-labeled robot data.
Sources
- V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning
- Motus2: A Self-Evolving General World Model for Dexterous Manipulation
- GR00T N1: An Open Foundation Model for Generalist Humanoid Robots
- $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
- AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems
- LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies
- MV-WAM: Manifold-Aware World Action Model with Value Augmentation
- RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation
- FlowWAM: Optical Flow as a Unified Action Representation for World Action Models
- LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models
- Gemini Robotics: Bringing AI into the Physical World
- World Models for Learning Dexterous Hand-Object Interactions from Human Videos
- BagelVLA: Enhancing Long-Horizon Manipulation via Interleaved Vision-Language-Action Generation
- DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset
- OpenVLA: An Open-Source Vision-Language-Action Model
- Masquerade: Learning from In-the-wild Human Videos using Data-Editing
- Causal World Modeling for Robot Control
- Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models
- JEPA-WAM: Learning Vision-Language-Action Policies with Joint-Embedding World Modeling
- Being-H0.7: A Latent World-Action Model from Egocentric Videos
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models