OpenWAM: An Open Framework for Composable World-Action Models
cs.RO, cs.CV
Submitted: 2026-10-06
Updated: 2026-10-06
Comments: 18 pages, 5 figures, 14 tables. Project page: https://openwam.stanford.edu ; Code: https://github.com/OpenWAM/OpenWAM ; Code and project page released June 4, 2026. Equal contribution: Heng Yu, David D. Yuan, Juze Zhang
Code: https://github.com/OpenWAM/OpenWAM
Project page: https://openwam.stanford.edu
License: http://creativecommons.org/licenses/by/4.0/
The gist: World-action models (WAMs) couple future prediction with robot control, yet existing systems often vary the video backbone, interaction structure, supervision, and inference procedure simultaneously,
Abstract
World-action models (WAMs) couple future prediction with robot control, yet existing systems often vary the video backbone, interaction structure, supervision, and inference procedure simultaneously, making their design choices difficult to compare. We introduce OPENWAM, an open world-action modeling framework built around a common causal robot-video foundation and configurable video-action interaction. Starting from Wan2.2-5B, we perform causal robot-video pretraining on over 10,000 hours of video, then integrate an action expert through a shared Mixture-of-Transformers architecture that supports joint, video-then-action, action-then-video, and decoupled generation. OPENWAM achieves high success rates on four LIBERO suites and real-world bimanual tasks; robot-video training with causal adaptation improves VTA success on LIBERO-Long from 68.4% to 97.8%. The same configurable architecture naturally extends to inverse and forward dynamics, allowing us to study how counterfactual transitions improve independently trained dynamics models beyond demonstrations alone. When only the video predictor is adapted to a new task, a frozen local-context inverse dynamics model trained on counterfactual data and demonstrations achieves 84.0% mean success across four held-out LIBERO-90 tasks, compared with 47.0% for a full-context inverse model and 21.5% for a local-context model trained only on demonstrations. For forward dynamics, counterfactual supervision reduces RGB prediction error by 34.5% and raises outcome identification from 21.1% to 71.3% among 16 same-state outcomes. OPENWAM provides a common testbed for comparing WAM interaction designs and for studying dynamics learning from video data beyond successful demonstrations.
Sources
- Cosmos 3: Omnimodal World Models for Physical AI
- Motus2: A Self-Evolving General World Model for Dexterous Manipulation
- GR00T N1: An Open Foundation Model for Generalist Humanoid Robots
- $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
- RT-1: Robotics Transformer for Real-World Control at Scale
- RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
- AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems
- WorldVLA: Towards Autoregressive Action World Model
- GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation
- Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots
- AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic Manipulation
- Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control
- Unified 4D World Action Modeling from Video Priors with Asynchronous Denoising
- Dream to Control: Learning Behaviors by Latent Imagination
- Mastering Diverse Domains through World Models
- Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representations
- NoiseGate: Learning Per-Latent Timestep Schedules as Information Gating in World Action Models
- Streaming-WAM: Action-Conditioned World-Action Model for Asynchronous Robot Manipulation
- Fail2Progress: Learning from Real-World Robot Failures with Stein Variational Inference
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies