The Right Future for Action: Learning Action-Relevant Predictive States in World Action Models
cs.CV
Submitted: 2026-09-20
Updated: 2026-09-20
Comments: 19 pages, 5 figures, 8 tables
License: http://creativecommons.org/licenses/by/4.0/
The gist: Generation-free world action models (WAMs) retain future-video prediction during training but act from internal video features at inference, leaving unclear what these features should preserve for
Terminology
Abstract
Generation-free world action models (WAMs) retain future-video prediction during training but act from internal video features at inference, leaving unclear what these features should preserve for control. Our representation diagnostics show that representations with more predictable future changes need not make linear action decoding easier. Observed future changes provide additional action information beyond the present, and linearly readable action information is spatially concentrated. These findings motivate Action-Relevant Predictive States (ARPS), a compact predictive interface between the video and action experts. ARPS uses a horizon-conditioned state predictor to aggregate intermediate video features into a compact state that supplies all visual context to the action expert. Future-representation supervision trains different parts of this state to predict visual representations at different future times, together with their changes relative to the present. At inference, the supervision branch is removed, and the action expert only uses the learned predictive state computed from current observations. Controlled ablations show that future supervision substantially improves generalization under distribution shift. ARPS achieves 99.2% success on LIBERO and transfers to LIBERO-Plus without adaptation, reaching 87.3% and exceeding Fast-WAM by 39.2 percentage points.
Sources
- V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning
- LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies
- DreamX-World 1.0: A General-Purpose Interactive World Model
- DreamX-Phi 1.0: Action-Conditioned Video World Model for Robotic Manipulation
- Compressor-VLA: Instruction-Guided Visual Token Compression for Efficient Robotic Manipulation
- DC-WAM: Dynamic-Centric Visual Supervision and Reasoning for World-Action Models
- Learning Explicit Physical Parameter Control and Benchmarking for Video Generation
- Being-H0.7: A Latent World-Action Model from Egocentric Videos
- Making Foresight Actionable: Repurposing Representation Alignment in World Action Models
- World Tokens: Enhancing Embodied Policies with Training-Time World Modeling
- RepWAM: World Action Modeling with Representation Visual-Action Tokenizers
- LiLa-WAM: Lightweight Latent Reasoning World-Action Model for Robotic Manipulation
- ABot-M0: VLA Foundation Model for Robotic Manipulation with Action Manifold Learning
- GigaWorld-Policy: An Efficient Action-Centered World--Action Model
- World Action Models are Zero-shot Policies
- Fast-WAM: Do World Action Models Need Test-time Future Imagination?
- Enfold: Folding World Model Imagination into Predictive Representations for Ultra-Efficient Embodied Control
- Do World Action Models Generalize Better than VLAs? A Robustness Study
- Faster-WAM: Efficient Inference-Time Future Conditioning for Robust World Action Models
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models