Elastic-WAM: Adaptive Spatiotemporal Imagination for Efficient Robot Control
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: We're looking at the paper: "Elastic-WAM: Adaptive Spatiotemporal Imagination for Efficient Robot Control".
Paper summary: Rosa: So, to recap, this paper introduces "Elastic-WAM: Adaptive Spatiotemporal Imagination for Efficient Robot Control." The central idea is that existing world action models improve robot control by predicting future visual dynamics, but the iterative video denoising used in those models causes a lot of inference overhead.
Dev: They address this by proposing a unified spatiotemporal refinement field. This field governs three things: when to stop world refinement, how far to advance each denoising step, and where to refresh future video regions.
Rosa: The method parameterizes this field using a shared temporal basis and spatial amplitudes, which they learn through a three-stage action-grounded supervised procedure.
Taro: They learn the temporal profile by comparing an immediate world commit with continuations of different denoising distances from the same recurrent state s.
Dev: Then they learn spatial priorities by comparing dense and sparse future video updates at the same denoising state, figuring out which regions are most relevant to action generation.
Rosa: And finally, they learn the overall demand by learning a global offset that controls the total refinement demand across all recurrent prefixes.
Conclusion: Rosa: So, looking at "Elastic-WAM: Adaptive Spatiotemporal Imagination for Efficient Robot Control," it’s clear that this method focuses on making the world model more adaptive to the current situation.
Dev: The authors show that by jointly adapting temporal progression and spatial allocation, they can capture stagedependent visual changes better than previous methods, which helps with task-state discrimination.
Taro: What this means for us is that we can expect better control performance in complex manipulation tasks, even when the environment shifts unexpectedly.
Rosa: It suggests that achieving state-of-the-art control performance is now possible with reduced inference latency because you’re only refining what the action really needs to see.
Dev: The title points to it being an adaptive imagination system, which means the robot isn't just blindly looking at the future; it's intelligently deciding where and how much computation is necessary for each step.
Haitian Li, Tianfu Li, Xuxi Zhao, Zizheng Liu, Yangcheng Yu, Yiwen Hou, Zeyu Xiao, Mutian Xu, žžžžHaonan Chen, ◆Lin Shao
National University of Singapore · RoboScience
cs.RO, cs.CV
Submitted: 2026-11-10
Updated: 2026-11-10
Comments: 14 pages. Project page: https://lihaitian.com/Elastic-WAM/
Project page: https://lihaitian.com/Elastic-WAM
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 99/100
The gist: World-Action Models improve robot control by predicting future visual dynamics, but iterative video denoising incurs substantial inference overhead Page 1 Our empirical observations reveal that…
Key concepts
- Unified Spatiotemporal Field
- This field models the total demand for world refinement across both time and space. It dictates when to stop refining, how far to advance each denoising step, and which future video regions need refreshing. It captures the overall remaining refinement demand and its distribution along time.
- Action-Grounded Supervision
- The training signal is built through three supervised stages that optimize the field parameters. This includes learning the temporal profile (how far to advance), spatial priorities (which regions are important), and overall demand (when to stop). This process ensures the model learns how to allocate computation effectively for action generation.
- Adaptive Sparse World Refinement
- Instead of dense refinement, this process starts with a dense step followed by adaptive sparse refinement guided by the learned field. Only specific future video tokens are recomputed in subsequent steps, while others reuse cached information. This selective approach reduces inference latency significantly compared to full joint models.
- Selective Refinement
- This technique involves choosing which future video regions to refine based on learned spatial scores, rather than random selection or refining everything. It preserves useful action information while avoiding redundant world updates, leading to better control and lower latency.
Terminology
Summary
World-Action Models improve robot control by predicting future visual dynamics, but iterative video denoising incurs substantial inference overhead Page 1 Our empirical observations reveal that preferred refinement depths vary across tasks and states, while different future regions exhibit varying relevance to action generation Page 3 Motivated by these findings, we propose Elastic-WAM, which adaptively allocates world refinement computation across temporal denoising time and spatial future space Page 3 Evaluations across diverse simulation benchmarks and real-world long-horizon manipulation demonstrate state-of-the-art control performance with reduced inference latency Page 3. p. 0
Pruning redundant reasoning tokens
Elastic-WAM models the demand for world refinement through a unified spatiotemporal field, as illustrated in Fig. 1 (b) Page 3. p. 1 This field governs when to stop world refinement, how far to advance each denoising step, and which regions to refresh Page 3. p. 1 The total mass of this field captures the overall remaining refinement demand, its distribution along z characterizes preferred denoising progression, and the local amplitudes ag(s) indicate how demand varies across future video regions Page 3. p. 3
How the training signal is built
The field is parameterized using a shared temporal basis B0(z s) and a region-specific amplitude ag(s) Page 3. The learning process involves three stages of action-grounded supervision, progressively optimizing the temporal profile, relative spatial priorities, and overall refinement demand Page 3. p. 1
The stages are:
-
Temporal Profile (HOW FAR): This is learned by comparing immediate World Commit with continuations of different denoising distances from the same recurrent state s Page 3. p. 4
-
Spatial Priorities (WHERE): This is learned by comparing dense and sparse future video updates at the same denoising state Page 3. p. 4
-
Overall Demand (WHEN): This involves learning the global offset b0(s) to control the overall refinement demand D(s) Page 3. p. 4
Where the gains come from
The inference process starts with a dense joint denoising step, followed by adaptive sparse world refinement guided by the learned field Page 3. The selected exact-token mask determines which future video queries are recomputed in the subsequent step Page 3. p. 4 During sparse joint updates, observation and action queries remain dense, while the selected future video tokens preserve their cached hidden states, keys, values, and last-refresh velocities Page 6. p. 5 This allows for reduced inference latency by 32. p. 12% compared with Joint-WAM on RoboCasa-GR1 Page 8. p. 1
The method demonstrates state-of-the-art control performance among methods without large-scale embodied pretraining, while reducing inference latency by 32.2% compared with Joint-WAM on RoboCasa-GR1 Page 3. p. 1 Elastic-WAM achieves the highest success rate on RoboCasa-GR1 (Table 4) Page 8 and consistently outperforms Joint-WAM across object, position, semantic, and task variations on LIBERO-Pro Page 7. p. 6
The adaptive spatiotemporal refinement helps Elastic-WAM capture stagedependent visual changes more effectively, thereby facilitating task-state discrimination and stage-appropriate action generation Page 9. Furthermore, in real-world long-horizon manipulation, Elastic-WAM achieves the highest full-task SR, while consistently attaining the best task PS across all evaluation settings Page 8. p. 6 This suggests that adaptive spatiotemporal refinement may also benefit long-horizon task execution Page 9. p. 7
The unified spatiotemporal field determines WHEN to stop world refinement, HOW FAR to advance denoising, and WHERE to refresh future video regions Page 3. This unified field is parameterized with a shared temporal basis and spatial amplitudes Page 4. p. 1 The learning framework uses an action-grounded supervised procedure to optimize these parameters Page 3. p. 1
The spatial selection outperforms random selection under the same token budget, as well as all-active refinement with a larger budget and FirstKV Page 8. The qualitative analysis shows that Elastic-WAM succeeds where Joint-WAM fails, suggesting that selective refinement can preserve useful action information while avoiding redundant world updates Page 9. p. 6 This indicates the benefit of selectively refreshing action-relevant future regions rather than relying on random allocation, broader refinement, or cached representations alone Page 8. p. 6
The temporal profile is supervised by comparing immediate World Commit with continuations of different denoising distances from the same recurrent state s Page 3. The stopping supervision is derived from evaluating immediate World Commit against the expected error of continuing under the learned temporal policy Page 3. p. 4
The learning of the global offset b0(s) is supervised by an objective that constrains cumulative stopping probabilities and commit timing across recurrent prefixes Page 3. This ensures that the cumulative World Commit threshold is reached appropriately Page 3. p. 5 The resulting framework enables efficient inference through recurrent sparse refinement while preserving the full logical attention context Page 8. p. 7
The learned spatial scores u rel g(s) are supervised by comparing dense and sparse future video updates at the same denoising state Page 3. This optimization is achieved using action losses, video losses, and budget regularization terms Page 4. p. 4 The WAM remains frozen while gradients propagate through its computations to optimize the spatial scores Page 3. p. 4
The adaptive temporal control and token-refresh budget provide complementary benefits, demonstrating that adaptive stopping improves upon the WHERE-only baseline, while further adapting denoising progression yields the best overall performance Page 8. The default budget K = 96 achieves the highest overall success rate, with smaller budgets remaining competitive and larger budgets yielding lower performance Page 8. p. 6 The results highlight the benefit of selectively continuing future refinement can provide more effective conditioning for precision-sensitive actions Page 13. p. 6
The final stage optimizes the global offset b0(s) using hazard predictions from the frozen temporal teacher to construct target continuation masses C k Page 3. This mapping is done through the aggregated regional amplitudes Page 4. p. 4 The objective Lfinal supervises cumulative stopping probabilities and commit timing across recurrent prefixes Page 3. p. 4
Elastic-WAM achieves the highest full-task SR, while consistently attaining the best task PS across all evaluation settings under real-world conditions Page 8. This superior performance on LIBERO Long suite and higher task progress in real-world experiments suggest that adaptive spatiotemporal refinement may also benefit long-horizon task execution Page 9. p. 7 The results highlight the benefit of selectively refining action-relevant future representations, improving control in complex manipulation while avoiding redundant world updates Page 8. p. 6 p. 6 p. 6 p. 6 p. 6 p. 6
The setup involves building Elastic-WAM upon FastWAM-Joint, which jointly refines future video representations and robot actions through iterative denoising Page 12. The model processes observation, future-video, and action tokens, allowing future visual representations to provide conditioning for action generation Page 12. p. 1
The framework enables efficient inference through recurrent sparse refinement while preserving the full logical attention context Page 8. This efficiency improvement is enabled by adaptive temporal refinement, selective future-video token updates, and cache reuse during action generation Page 8. p. 1 p. 1 Unselected future video tokens preserve their cached hidden states, keys, values, and last-refresh velocities Page 8. p. 5 Once the cumulative World Commit threshold is reached or a safety condition is triggered, Elastic-WAM stops world refinement and directly reuses the current video representation and layerwise cache for subsequent action denoising Page 8. p. 5
Elastic-WAM introduces a unified spatiotemporal field to model refinement demand, where geometry-aware local curves describe the refinement needs of different regions Page 3. The global aggregation Q(z s) represents the spatially aggregated refinement demand Page 4. p. 3 This field is learned through three stages of action-grounded supervision Page 3. p. 1
The evaluation across simulation benchmarks and real-world long-horizon manipulation demonstrates state-of-the-art control performance with reduced inference latency Page 3. The results highlight the benefit of selectively refining action-relevant future representations, improving control in complex manipulation while avoiding redundant world updates Page 8. This demonstrates that state-of-the-art control performance is achievable with reduced inference latency Page 8. p. 1
The pipeline of Elastic-WAM involves a dense joint refinement step, followed by adaptive sparse world refinement with state-dependent denoising progression and spatial token selection Page 3. This process repeats with updated refinement decisions at each step Page 8. p. 4 The pipeline is guided by the learned refinement field Page 3. p. 4
The results show that Elastic-WAM achieves the highest average success rate on LIBERO (Table 1) Page 8, and demonstrates superior robustness across all perturbation categories on LIBERO-Plus Page 8. These improvements suggest that adapting refinement depth and spatial allocation to the current state provides more effective future representations for action generation under diverse distribution shifts Page 8. p. 6 The results highlight the benefit of selectively refining action-relevant future representations, improving control in complex manipulation while avoiding redundant world updates Page 8.
The evaluation on RoboCasa-GR1 shows that Elastic-WAM achieves the highest success rate (Table 4) Page 8 and reduces inference latency by 32.2% compared with Joint-WAM Page 8. p. 1 This efficiency improvement is enabled by adaptive temporal refinement, selective future-video token updates, and cache reuse during action generation Page 8. The results highlight the benefit of selectively refining action-relevant future representations, improving control in complex manipulation while avoiding redundant world updates Page 8.
The qualitative analysis shows that Elastic-WAM achieves successful control through selective refinement while preserving the full attention context Page 9. This allows different regions to undergo different levels of refinement while preserving the full logical attention context Page 8. p. 6 The results demonstrate state-of-the-art control performance with reduced inference latency Page 8. p. 1 The results highlight the benefit of selectively refining action-relevant future representations, improving control in complex manipulation while avoiding redundant world updates Page 8.
The results indicate that joint adaptation of temporal progression and spatial allocation helps Elastic-WAM capture stagedependent visual changes more effectively, thereby facilitating task-state discrimination and stage-appropriate action generation Page 9. This interpretation is consistent with its superior performance on the LIBERO Long suite (Table 1) Page 8 and higher task progress in real-world experiments (Table 7) Page 9. p. 7 This suggests that adaptive spatiotemporal refinement may also benefit long-horizon task execution Page 8. p. 7
The results show that Elastic-WAM achieves the highest full-task SR, while consistently attaining the best task PS across all evaluation settings under real-world conditions Page 8. This superior performance on LIBERO-Pro and LIBERO-Plus suggests that adapting refinement depth and spatial allocation to the current state provides more effective future representations for action generation under diverse distribution shifts Page 8. p. 6 The results highlight the benefit of selectively refining action-relevant future representations, improving control in complex manipulation while avoiding redundant world updates Page 8.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Adaptive Allocation of World Refinement Computation: Elastic-WAM models demand for world refinement through a unified spatiotemporal field that governs
when to stop world refinement, how far to advance each denoising step, and which regions to refresh.
This allows the system to adapt computation across temporal denoising time and spatial future space based on task state. -
Action-Grounded Supervised Learning Framework: The system develops an
action-grounded supervised learning framework and recurrent sparse inference strategy, enabling adaptive refinement with a frozen WAM backbone.
This enables adaptation while preserving the underlying action generation mechanism of the pretrained model. -
Recurrent Sparse Inference Strategy: At inference, Elastic-WAM uses the learned field to
guide recurrent sparse world refinement to improve task success while reducing inference overhead.
This guides subsequent sparse world refinement and continues action denoising after world refinement stops, resulting in reduced inference latency by 32.2% compared to Joint-WAM. -
Precision-Sensitive Manipulation: The system can successfully execute long-horizon, precision-sensitive tasks on complex manipulation benchmarks like RoboCasa-GR1, where it
successfully executes the manipulation
even when methods like Fast-WAM fail to complete final placement steps. -
Robustness under Distribution Shifts: Elastic-WAM demonstrates superior robustness on LIBERO-Plus and LIBEROPro, showing
superior robustness across all perturbation categories, with particularly notable gains under camera viewpoint changes, robot initialization changes, and sensor noise.
This suggests adaptation to the current state provides more effective future representations for action generation under diverse distribution shifts. -
Task-State Discrimination: By jointly adapting temporal progression and spatial allocation of future refinement, Elastic-WAM is hypothesized to
capture stagedependent visual changes more effectively, thereby facilitating task-state discrimination and stage-appropriate action generation.
This benefit is consistent with its superior performance on the LIBERO Long suite.
Sources
- Cosmos 3: Omnimodal World Models for Physical AI
- Flash-WAM: Modality-Aware Distillation for World Action Models
- GR00T N1: An Open Foundation Model for Generalist Humanoid Robots
- $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
- RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
- UniVLA: Learning to Act Anywhere with Task-centric Latent Actions
- WorldVLA: Towards Autoregressive Action World Model
- ObjectWorld: Scaling Object-Centric World Models with A Large-Scale 3D Full-Object Flow Dataset
- MVG-WAM: Multiple View Geometry-Aware World-Action Modeling for Robotic Manipulation
- StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing
- LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models
- Unified 4D World Action Modeling from Video Priors with Asynchronous Denoising
- ELAN4D: Embodiment-Centric 4D Supervision for Vision-Language-Action Models via Plug-and-Play Adaptation
- OpenVLA: An Open-Source Vision-Language-Action Model
- Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success
- Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning
- Efficient-WAM: A 1B-Parameter World-Action Model with Low-Cost Future Imagination
- Causal World Modeling for Robot Control
- SLIP-VLA: Single-Step Latent Imagination for Policy Learning in Vision-Language-Action Models
- Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference
Related papers
- Rethinking Actions: Continuity-Aware Priors for World-Model Planning
- MOSAIC: Modular Obstacle and Spatial-Aware Framework for Manipulation In Cluttered Scenes
- SAGAS: Semantic-Aware Graph-Assisted Stitching for Offline Temporal Logic Planning
- DexSeed: Expanding Human Hand-Object Interactions into Robot Demonstrations for Dexterous Policy Learning
- LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
- CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation