STRIKE: Learning Visual State Transitions for Physical World Modeling
cs.CV
Submitted: 2026-10-07
Updated: 2026-10-07
Code: https://github.com/Wan-Video/Wan2.2
Project page: https://strike-applied.github.io/Abstract
License: http://creativecommons.org/licenses/by/4.0/
The gist: Physical world modeling requires predicting how interactions change a scene, not merely generating coherent motion.
Abstract
Physical world modeling requires predicting how interactions change a scene, not merely generating coherent motion. We propose STRIKE, a framework that separates visual state transition learning from dense video generation. We construct event-aligned supervision by extracting observed states from training videos and pairing them with transition descriptions and temporal offsets. An image-based transition model learns to predict the next scene configuration from the current image, a local transition specification, and elapsed time. At inference, a pretrained vision-language planner predicts time transition specifications, and recursive application of the learned transition model produces a sequence of future visual states. A separately trained dynamic model then generates the complete rollout conditioned on these states and their temporal locations. Experiments on Physics-IQ Verified, PhyGenBench, Pisa-Experiments, and RoboTwin2.0 show improvements of STRIKE over the corresponding video-backbone baselines in benchmark measures of physical consistency and manipulation-video fidelity. These results support learned visual state transitions as an effective intermediate representation for physical world modeling.
Sources
- World Simulation with Video Foundation Models for Physical AI
- Qwen3-VL Technical Report
- RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation
- Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control
- LTX-Video: Realtime Video Latent Diffusion
- SKIP: Sparse Keyframe Interpolation Paradigm for Efficient Embodied World Models
- LaMo: Self-Supervised Latent Motion Priors for Physical Realism in Video Generation
- KeyWorld: Key Frame Reasoning Enables Effective and Efficient World Models
- Cosmos 3: Omnimodal World Models for Physical AI
- Open-Sora 2.0: Training a Commercial-Level Video Generation Model in $200k
- Physics-IQ Verified
- WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Models
- RoFormer: Enhanced Transformer with Rotary Position Embedding
- DreamWorld: Unified World Modeling in Video Generation
- Wan: Open and Advanced Large-Scale Video Generative Models
- Qwen-Image Technical Report
- Motion Forcing: A Decoupled Framework for Robust Video Generation in Motion Dynamics
- CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer
- CausalMotion: Structured Physical Reasoning as Keyframe and Trajectory Guidance for Training-Free Video Generation
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models