EvoWorld: Evolving Panoramic World Generation with Explicit 3D Memory
cs.CV
Submitted: 2025-10-01
Updated: 2026-11-04
Comments: Code available at: https://github.com/JiahaoPlus/EvoWorld
Code: https://github.com/NVIDIA/Cosmos
License: http://creativecommons.org/licenses/by/4.0/
The gist: Humans possess a remarkable ability to mentally explore and replay 3D environments they have previously experienced.
Terminology
Abstract
Humans possess a remarkable ability to mentally explore and replay 3D environments they have previously experienced. Inspired by this mental process, we present EvoWorld: a world model that bridges panoramic video generation with evolving 3D memory to enable spatially consistent long-horizon exploration. Given a single panoramic image as input, EvoWorld first generates future video frames by leveraging a video generator with fine-grained view control, then evolves the scene's 3D reconstruction using a feedforward plug-and-play transformer, and finally synthesizes futures by conditioning on geometric reprojections from this evolving explicit 3D memory. Unlike prior state-of-the-arts that synthesize videos only, our key insight lies in exploiting this evolving 3D reconstruction as explicit spatial guidance for the video generation process, projecting the reconstructed geometry onto target viewpoints to provide rich spatial cues that significantly enhance both visual realism and geometric consistency. To evaluate long-range exploration capabilities, we introduce the first comprehensive benchmark spanning synthetic outdoor environments, Habitat indoor scenes, and challenging real-world scenarios, with particular emphasis on loop-closure detection and spatial coherence over extended trajectories. Extensive experiments demonstrate that our evolving 3D memory substantially improves visual fidelity and maintains spatial scene coherence compared to existing approaches, representing a significant advance toward long-horizon spatially consistent world modeling.
Sources
- Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets
- Cosmos World Foundation Model Platform for Physical AI
- LTX-Video: Realtime Video Latent Diffusion
- Wan: Open and Advanced Large-Scale Video Generative Models
- CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer
- ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis
- Open3D: A Modern Library for 3D Data Processing
- Towards Accurate Generative Models of Video: A New Metric & Challenges
- GPT-4o System Card
- Simple and Effective Synthesis of Indoor 3D Scenes
- PanoWan: Lifting Diffusion Video Generation Models to 360{\deg} with Latitude/Longitude-aware Mechanisms
- ModelScope Text-to-Video Technical Report
- History-Guided Video Diffusion
- AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning
- VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation
- World Models
- Video In-context Learning: Autoregressive Transformers are Zero-Shot Video Imitators
- Video as the New Language for Real-World Decision Making
- GAIA-1: A Generative World Model for Autonomous Driving
- DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving
Related papers
- Efficient All-in-One Weather Restoration using Spectral Harmonization
- Peak-End-Net: A Peak-End Rule Inspired Framework for Generalizable Video Aesthetic Assessment
- Sensor2Sensor: Cross-Embodiment Sensor Conversion for Autonomous Driving
- WithEveryone: Unified Planning and Identity Grounding for Group Image Generation
- SciFigure2Code: An AI-Reconstructed Benchmark for Scientific Figure-to-Code
- RepFusion: Leveraging Multimodal Priors for Denoising in Representation Space