minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Models
cs.CV
Submitted: 2026-05-28
Updated: 2026-09-22
Code: https://github.com/shengshu-ai/minWM
License: http://creativecommons.org/licenses/by/4.0/
The gist: Recent video diffusion foundation models have achieved remarkable progress in high-quality video generation, yet turning them into real-time interactive video world models remains challenging.
Terminology
Abstract
Recent video diffusion foundation models have achieved remarkable progress in high-quality video generation, yet turning them into real-time interactive video world models remains challenging. Interactive world models require controllable, causal, and low-latency rollout, which in practice demands a full pipeline spanning data construction, controllable fine-tuning, autoregressive training, few-step distillation, and streaming inference. In this work, we present minWM, a full-stack open-source framework for building real-time interactive video world models. minWM provides an end-to-end pipeline that converts existing bidirectional T2V/TI2V video foundation models into camera-controllable few-step autoregressive world models. Specifically, minWM first fine-tunes a bidirectional video diffusion model with camera control, and then applies the Causal Forcing / Causal Forcing++ pipeline, including AR diffusion training, causal ODE or causal consistency distillation, and asymmetric DMD, to distill it into a few-step autoregressive generator for low-latency rollout. The framework is modular and architecture-extensible: we instantiate it on representative open backbones, including Wan2.1-T2V-1.3B and HY1.5-TI2V-8B, covering both cross-attention-based condition injection and MMDiT-style architectures. minWM also supports adapting existing video world models, such as HY-WorldPlay, to new data distributions, training recipes, and latency targets. Beyond releasing runnable scripts, checkpoints, documentation, and inference code, we provide practical ablations on camera trajectory quality, controllability training steps, and minimal batch-size requirements. We hope minWM serves as a reproducible and extensible recipe for building and adapting real-time interactive video world models. Project Page: [https://github.com/shengshu-ai/minWM](https://github.com/shengshu-ai/minWM)
Sources
- Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Models
- CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer
- Open-Sora Plan: Open-Source Large Video Generation Model
- Open-Sora: Democratizing Efficient Video Production for All
- Wan: Open and Advanced Large-Scale Video Generative Models
- HunyuanVideo: A Systematic Framework For Large Video Generative Models
- WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling
- Hunyuan-GameCraft-2: Instruction-following Interactive Game World Model
- Yume-1.5: A Text-Controlled Interactive World Generation Model
- Vidarc: Embodied Video Diffusion Model for Closed-loop Control
- Live Avatar: Streaming Real-time Audio-Driven Avatar Generation with Infinite Length
- StreamAvatar: Streaming Diffusion Models for Real-Time Interactive Human Avatars
- RELIC: Interactive Video World Model with Long-Horizon Memory
- Yan: Foundational Interactive Video Generation
- PAN: A World Model for General, Actionable, and Long-Horizon World Simulation
- Matrix-game 2.0: An open-source, real-time, and streaming interactive world model
- MotionStream: Real-Time Video Generation with Interactive Motion Controls
- Diffusion Adversarial Post-Training for One-Step Video Generation
- Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion
- Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models