Wizard: Manipulate Any World in Real Time via Agentic Control
cs.CV
Submitted: 2026-10-29
Updated: 2026-11-17
Comments: Project Page: https://wyzardwu.github.io/wizard
Code: https://github.com/fyyakaxyy/AnimationGPT
Project page: https://wyzardwu.github.io/wizard
License: http://creativecommons.org/licenses/by/4.0/
The gist: Recent advances in video world models have enabled the generation of increasingly realistic and interactive virtual environments, opening new possibilities for player-customizable gameplay.
Abstract
Recent advances in video world models have enabled the generation of increasingly realistic and interactive virtual environments, opening new possibilities for player-customizable gameplay. Agentic frameworks further advance this direction by decoupling explicit game logic and controllable world representations from visual generation. However, existing approaches support only simple game settings and predefined motion patterns, generalize poorly to unseen environments and characters, and offer limited real-time interactivity. In this paper, we introduce Wizard, an agentic game world model that enables real-time, precise control over character motions while generalizing zero-shot across novel scenes, characters, and actions. To achieve this, we construct a diverse dataset spanning 750 scenes and 178 characters, incorporating skeleton and geometry information as structured representations. We leverage these representations as intermediate proxies to condition a video generation model, together with an adaptive masking mechanism that improves alignment between the conditioning signals and generated video rollouts. Furthermore, we introduce an agentic framework that explicitly maintains and updates game states, allowing players to freely configure their virtual worlds and exercise fine-grained control over character motions while preserving visual quality and gameplay consistency. By combining agentic control with structured proxy-based generation, our framework unlocks new possibilities for player-driven game creation with real-time, generalizable, and mechanically consistent world simulation.
Sources
- WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling
- Matrix-game 2.0: An open-source, real-time, and streaming interactive world model
- Infinite Worlds with Versatile Interactions
- Multics: Generative Multi-Player World Model with Hierarchical Communication
- WorldCrafter: Consistent Video World Model with Implicit 3D-aware Memory
- SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer
- WorldCast: Distributed Multiplayer World Models
- StatePlay: State-Aware Game World Models for Mechanics-Consistent Generation
- ReactiveGWM: Flexible Control and NPC Reactivity in Game World Models
- SCOPE: Simulating Cross-game Operations in Playable Environments for FPS World Models
- Incantation: Natural Language as the Action Interface for Multi-Entity Video World Models
- H3-World: Turning Language Understanding into World Control
- GameDirector: Decoupling Gameplay Logic from Rendering for Player-Configurable Game World Models
- Code World Model: Coding Agent as World Brain
- AnyGameRenderer: Entity-Consistent Generative Rendering Across Games and an Entity-Level Benchmark
- Programmable World Model
- Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generation
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs