Position: Next-Generation Game Engines Should Be Built on Interactive Generative Video

summary

The gist

Interactive Generative Video (IGV) is proposed as the foundation for Generative Game Engines (GGE), enabling unlimited novel content generation in next-generation gaming because GGE leverages IGV’s…

In short

The paper proposes Interactive Generative Video (IGV) as the foundation for Generative Game Engines (GGE). IGV combines video synthesis with user control, memory, physics understanding, and causal reasoning. This framework aims to revolutionize game creation by enabling unlimited novel content generation through AI-driven systems.

Key concepts

Interactive Generative Video (IGV)
IGV is a new approach to video generation that allows users to directly control the output. It integrates video synthesis with user input, memory of past scenes, simulation of physical rules, and causal reasoning. Essentially, it builds an explorable virtual world through generated videos that responds dynamically to player actions.
Generative Game Engine (GGE)
A GGE is a game engine powered by IGV technology. It uses the video generation capabilities to create infinite new content for games. GGE aims to lower technical barriers and boost creativity by allowing developers to generate complex, novel game worlds automatically.
Physics-aware World Modeling
This capability allows AI models used in video generation to understand and simulate real-world physical laws, such as gravity, collision, and acceleration. It ensures that the generated virtual environments behave realistically according to established physical rules.
Maturity Roadmap (L0-L4)
This is a five-level plan guiding the development of GGE. Level 0 is manual content creation, moving through Levels 1 (AI-assisted assets), 2 (physics-compliant generation), 3 (causal reasoning), and finally Level 4, which aims for a self-evolving world ecosystem.

Terminology used across episodes

This episode discusses

The paper

Position: Next-Generation Game Engines Should Be Built on Interactive Generative Video · Read on arXiv

Jiwen Yu, Yiran Qin, Haoxuan Che, Quande Liu, Xintao Wang, Pengfei Wan, Di Zhang, Xihui Liu

HKU University of Hong Kong, HKUST University of Science and Technology, Kuaishou Technology Company

Modern game development faces significant challenges in creativity and cost due to predetermined content in traditional game engines. Recent breakthroughs in video generation models, capable of synthesizing realistic and interactive virtual environments, present an opportunity to revolutionize game creation. In this position paper, we propose Interactive Generative Video (IGV) as the foundation for Generative Game Engines (GGE), enabling unlimited novel content generation in next-generation gaming. GGE leverages IGV's unique strengths in unlimited high-quality content synthesis, physics-aware world modeling, user-controlled interactivity, long-term memory capabilities, and causal reasoning. We present a comprehensive framework detailing GGE's core modules and a hierarchical maturity roadmap (L0-L4) to guide its evolution. Our work charts a new course for game development in the AI era, envisioning a future where AI-powered generative systems fundamentally reshape how games are created and experienced.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Position: Next-Generation Game Engines Should Be Built on Interactive Generative Video".

Jane: Here's the gist of the paper. Interactive Generative Video (IGV) is proposed as the foundation for Generative Game Engines (GGE), enabling unlimited novel content generation in next-generation gaming because GGE leverages IGV’s unique strengths in unlimited high-quality content synthesis, physics-aware world modeling, user-controlled interactivity, long-term memory capabilities, and causal reasoning.

Tom: The Core Concept of IGV and GGE IGV is a new paradigm that extends video generation capabilities with interactive features because it centers around video generation while incorporating four key characteristics: user control over the generated content, memory of video context, understanding and simulation of physical rules, and causal reasoning intelligence.

Jane: IGV effectively constructs an explorable and interactive virtual world through video generation, functioning similarly to a simulator.

Tom: The authors argue that Interactive Generative Video (IGV) serves as the core technology for Generative Game Engine (GGE).

Paper summary: Tom: So, this paper is titled "Position: Next-Generation Game Engines Should Be Built on Interactive Generative Video". It's talking about how we can fix the problems with traditional game engines that use predetermined content.

Jane: Right, so the main idea here is they propose something called Interactive Generative Video, or IGV, as the base for what they call Generative Game Engines, or GGE. They argue this will let us generate unlimited new content for gaming in the next generation.

Lu: Exactly. The paper claims that GGE uses IGV's unique strengths: it has unlimited high-quality content synthesis, physics-aware world modeling, user-controlled interactivity, long-term memory capabilities, and causal reasoning.

Meng: So it’s not just making cool videos anymore; it’s about building a system that understands physics and can learn from past interactions to create something new every single time.

Lalam: I read the summary they gave us: IGV is the foundation for GGE, enabling unlimited novel content generation in next-generation gaming because GGE leverages IGV’s unique strengths in unlimited high-quality content synthesis, physics-aware world modeling, user-controlled interactivity, long-term memory capabilities, and causal reasoning.

Tom: It sounds like they're trying to solve the creativity and cost issues that come with having fixed content in old engines. What makes this approach different from just using a standard video generator for a game?

Jane: Well, they explain that IGV is a new paradigm because it extends video generation with interactive features. It centers around generating video while incorporating four things: user control over the generated content, memory of the video context, understanding and simulating physical rules, and causal reasoning intelligence.

Lu: So they see IGV as constructing an explorable and interactive virtual world that functions like a simulator by using video generation to do that. They say it serves as the core technology for GGE >

Meng: What I'm interested in is how this translates to actual game development work. They talk about GGE reducing technical barriers while boosting productivity and creativity through infinite AI-driven content generation. That sounds like a big claim, so I want to hear more about what actually makes it work practically.

Tom: Okay, so they lay out a framework with six core modules: Generation, Control, Memory, Dynamics, Intelligence, and GamePlay. They also have this maturity roadmap from Level zero to Level four that shows how the technology can evolve over time.

Paper summary: Jane: That roadmap is pretty clear: Level zero is no AI-assisted assets generation where everything must be manually designed first. Then you move to Level one with AI-assisted assets, and then in Level two you get Physics-Compliant Interactive World Generation where the engine generates content based on real-time player interactions <ref:2503.17359#pg1>.

Lu: Moving from Level two to Level three involves Causal-Reasoning World Simulation, which adds long-term cause and effect relationships beyond just the immediate physics <ref:2503.17359#pg1>. And at Level four you get a Self-Evolving World Ecosystem where the systems develop new rules on their own <ref:2503.17359#pg3>.

Meng: From my side, I’m looking at Level two and three because that’s where real interactivity comes in, but I worry about the 'Intelligence Module'—how sure is this reasoning going to be when it starts generating things we haven't even thought of <ref:2503.17359#pg1>?

Lalam: The paper says the Intelligence Module includes reasoning for long-term causal inference based on initial conditions, and self-evolution for virtual worlds to continuously develop new knowledge, rules, and behaviors through emergent properties. I'm confident in that direction because the literature suggests this is where true generative potential lies.

Tom: That’s interesting about the self-evolution aspect. If a world can generate its own rules, that means we don't have to pre-script every single interaction or reward structure for every possible scenario.

Jane: And they explain that IGV offers four key advantages that position it for GGE implementation: powerful generalizable generative capabilities, physics-aware world modeling, user-controlled generation for interactive experiences, and leveraging vast video data for training.

Lu: The paper points to GameFactory one hundred twenty-one, which demonstrates the ability to transfer skills to unprecedented scenarios by generating action-controllable videos in open-domain settings <ref:2503.17359#pg2>. That suggests a level of generalizability that’s pretty impressive.

Meng: But what about the practical deployment? How do you handle physics modeling when it starts generating things that aren't just reacting to gravity but are actually breaking and colliding in a physically consistent way at scale?

Jane: The paper also touches on economic concerns, suggesting costs can be reduced through technological advancements. They mention efficient autoregressive video generation, like CausVid one hundred nineteen, which achieves real-time frame generation through distribution matching distillation or DMD one hundred eighteen <ref:2503.17359#pg2>.

Tom: So the authors are pointing toward specific technical solutions for the computational costs. They also discuss data collection costs being mitigated as more open-source datasets like GameGenX thirteen become available, even though initial training has a cost.

Lu: And they address licensing costs by suggesting generative models will lower the barrier for developers to create their own new intellectual properties, creating a better ecosystem between developers and gaming companies.

Paper summary: Meng: I still see security as a big thing here. They mention that safety measures can be implemented through value alignment using Reinforcement Learning from Human Feedback, or RLHF during post-training phase, and real-time harmful content detection using Vision Language Models to analyze generated content quickly >

Jane: So the paper is proposing this whole GGE structure—the Generation, Control, Memory, Dynamics, Intelligence modules—to build this interactive foundation. It’s a big picture concept for how games will be made.

Tom: And it ends by showing a workflow integration across four phases: Pre-production where you design prompts and prototypes; Production where you fine-tune models with targeted game data; Testing for content screening and performance; and Post-launch for updates based on player behavior.

Lu: The feasibility of this whole thing depends on a few things: robust base IGV models that support efficient control, fine-tuning, and fast inference, accessible data sharing mechanisms that enable legal access to high-quality training data, and enough AI computing resources.

Meng: I'm leaning toward the model capability requirement. If the base IGV models don't handle complex physics and long-term reasoning well during inference at scale, this whole workflow falls apart quickly on real traffic >

Jane: So to wrap up this discussion of "Position: Next-Generation Game Engines Should Be Built on Interactive Generative Video," the authors are arguing that IGV is the core technology for GGE, which aims to reduce technical barriers and boost productivity through infinite AI-driven content generation.

Tom: It’s a roadmap from manual design to self-evolving ecosystems in Level four <ref:2503.17359#pg3>. The implication is that we move away from building fixed games toward building systems that can generate their own experiences based on player input and learned rules.

Jane: It suggests a future where everyone can become a game designer just by providing design instructions to video generation models, which then generate limitless explorable virtual worlds >

Lu: The paper is really pushing the idea that GGE isn't just about making pretty assets; it’s about creating systems with physics, memory, and reasoning built in from the start.

Meng: For me, it means we need to focus less on painstakingly hand-crafting every level and more on designing the core rules and constraints for the system that will handle the rest of the content generation >

Tom: That’s a pretty concrete shift. We're moving from being content creators to being system architects who design how an infinite set of content can be generated interactively.

Conclusion: Tom: So we’ve gone through the modules and the roadmap, and now we gotta talk about what this whole paper is actually saying in simple terms.

Jane: Yeah, so they're arguing that Interactive Generative Video, or IGV, should be the foundation for what they call Generative Game Engines. It’s basically their big proposal for how to build games in the AI era.

Lu: They’re framing it as a way to fix the problems with traditional game engines having fixed content by letting them generate unlimited novel content through this system.

Meng: From my side, I’m thinking about what that means for deployment. It sounds like a huge leap from building one specific game to building a whole system that can constantly create new stuff on its own.

Tom: Exactly, and they put this framework out—Generation, Control, Memory, Dynamics, Intelligence—and the maturity roadmap L0 to L4 shows how we get there over time.

Jane: It’s about moving from starting with zero AI assets at Level zero to an ecosystem where the world actually evolves on its own at Level four. That’s a massive scope.

Lu: The core idea is that IGV gives us these specific capabilities like physics-aware modeling and causal reasoning, which lets the engine simulate a world instead of just rendering pre-made scenes.

Tom: And they’re not ignoring the practical stuff either—they discuss how to manage those high computational costs through things like efficient video generation methods.

Jane: They also lay out some serious ethical considerations, talking about copyright and safety, which is important because this technology is so powerful.

Meng: It seems like the main thing here is shifting the focus from a content creator mindset to a system architect mindset who designs the rules for how infinite content gets generated interactively.

Tom: That’s right—we’re moving from hand-crafting every level to designing a generative system that handles the rest of the experience.

Jane: And that opens up some huge questions about what our role as game developers actually becomes in this new landscape.

Lu: It’s a big shift, and it leads us directly into how you actually get those models to work reliably when you put them into a real production environment.

More episodes

← Home