Kepler4D: Controllable Future Video Generation via 4D Scene State Evolution
cs.CV
Submitted: 2026-10-02
Updated: 2026-10-02
Comments: Project page: https://brack-wang.github.io/kepler4d/
Code: https://github.com/ModelTC/LightX2V
Project page: https://brack-wang.github.io/kepler4d
License: http://creativecommons.org/licenses/by/4.0/
The gist: Video world models aim to preserve scene structure and predict how dynamic objects evolve beyond visual observations.
Abstract
Video world models aim to preserve scene structure and predict how dynamic objects evolve beyond visual observations. We present Kepler4D, a framework for future video generation through explicit 4D scene state evolution. Given a monocular video, Kepler4D constructs a shared 3D representation of background geometry, object motion histories, coarse spatial supports, and semantic context. Chain-of-Motion summarizes observed motion and uses a vision-language model to select structured speed and heading decisions and decide whether to bound object-center height from below. A deterministic rollout converts these decisions into future object trajectories for inspection and editing before synthesis. We render the evolving proxies into geometric controls for a pretrained video generator, separating coarse object motion from the synthesis of appearance and articulation. Experiments on real-world videos demonstrate that Kepler4D enables controllable object motion and plausible future rollout while preserving scene consistency.
Sources
- Qwen2.5-VL Technical Report
- DeepVerse: 4D Autoregressive Video Generation as a World Model
- Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Models
- VGGT-Long: Chunk it, Loop it, Align it -- Pushing VGGT's Limits on Kilometer-scale Long RGB Sequences
- LIST3R: Long-sequence Instance-aware 3D Reconstruction
- GAIA-1: A Generative World Model for Autonomous Driving
- The 2017 DAVIS Challenge on Video Object Segmentation
- Wan: Open and Advanced Large-Scale Video Generative Models
- RayMap3R: Inference-Time RayMap for Dynamic 3D Reconstruction
- CogniMap3D: Cognitive 3D Mapping and Rapid Retrieval
- EvoWorld: Evolving Panoramic World Generation with Explicit 3D Memory
- From Seeing to Predicting: A Vision-Language Framework for Trajectory Forecasting and Controlled Video Generation
- Learning Interactive Real-World Simulators
- OptiWorld: Optimal Control for Video World Generation under Physical Constraints
- MolmoMotion: Forecasting Point Trajectories in 3D with Language Instruction
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models