World Action Modeling with Progressive Visual Planning
cs.AI, cs.CV, cs.RO
Submitted: 2026-10-01
Updated: 2026-10-01
Comments: Project Page: https://sii-ferenas.github.io/ProWAM-page
Code: https://github.com/robocasa-benchmark/leaderboard
Project page: https://sii-ferenas.github.io/ProWAM-page
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: World action models (WAMs) have emerged as a promising paradigm for robotic control by jointly predicting future visual dynamics and actions from an initial observation and instruction.
Abstract
World action models (WAMs) have emerged as a promising paradigm for robotic control by jointly predicting future visual dynamics and actions from an initial observation and instruction. However, existing WAMs struggle with long-horizon prediction, as generating dense video rollouts is highly inefficient. Some recent WAMs address this by predicting a single future frame without generating the full video, but this approach neglects how to progress toward the goal. We present ProWAM, a progressive world action model that jointly predicts actions and an ordered sequence of sparse visual sub-goals, providing explicit visual guidance to anchor action generation throughout task execution. This design scales naturally, as sub-goal prediction can be learned from large-scale action-free videos, allowing the video backbone to offload complex visual planning from the action policy. For efficient action generation, ProWAM executes a single video-backbone forward pass to cache sparse sub-goal features, eliminating iterative full-video generation and requiring only lightweight action denoising during replanning. Across extensive evaluations, ProWAM achieves superior out-of-distribution robustness. On simulation benchmarks, it sets new state-of-the-art results on LIBERO-Plus (85.8%) and randomized RoboTwin (75.7%), outperforming the strongest baseline with relative gains of up to +35.9%. On RoboCasa365, ProWAM achieves a 48.1% success rate and 18.2% on the challenging Composite-Unseen split, ranking 4th overall. Crucially, in zero-shot real-world experiments, ProWAM achieves 70.0% success, outperforming the strongest baseline by +15.0 (from 55.0% to 70.0%, a +27.3% relative gain) in novel scenes. These results demonstrate the value of progress-indexed visual foresight for closed-loop control. Our program is in https://sii-ferenas.github.io/ProWAM-page.
Sources
- ${\pi}_{0.7}$: a Steerable Generalist Robotic Foundation Model with Emergent Capabilities
- Qwen3-VL Technical Report
- AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems
- MV-WAM: Manifold-Aware World Action Model with Value Augmentation
- RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation
- VISTA: Enhancing Visual Conditioning via Track-Following Preference Optimization in Vision-Language-Action Models
- ABot-PhysWorld: Interactive World Foundation Model for Robotic Manipulation with Physics Alignment
- LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models
- VLA-Reasoner: Empowering Vision-Language-Action Models with Reasoning via Online Monte Carlo Tree Search
- BagelVLA: Enhancing Long-Horizon Manipulation via Interleaved Vision-Language-Action Generation
- RLDX-1 Technical Report
- Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning
- Causal World Modeling for Robot Control
- Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models
- Flow Matching for Generative Modeling
- Scaling World Model for Hierarchical Manipulation Policies
- Being-H0.5: Scaling Human-Centric Robot Learning for Cross-Embodiment Generalization
- RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots
- Improving Pre-Trained Vision-Language-Action Policies with Model-Based Search
- mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAs
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems