Einstein World Models

arXiv:2606.26969 · cs.AI, cs.CL, cs.CV · Submitted 2026-06-25 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Einstein World Models".

Jane: Here's the gist of the paper. Does intelligence require the ability to reason about phenomena beyond direct experience?

Tom: Einstein World Models are a blueprint for LLM-based reasoning systems that place visual-temporal rollouts inside the reasoning trace, allowing them to reason in ways that text alone may not support well.

Jane: Pruning redundant reasoning tokens Einstein World Models (EWMs) are designed to enable language models to utilize visualising counterfactual events as a mechanism for complex thought.

Tom: The core idea is that some questions require a model to visualise how a described scene unfolds, and that this may be poorly supported by language alone.

Title and authors: Tom: So we're looking at this paper called "Einstein World Models," and it's basically asking if intelligence needs the ability to reason about things you haven't directly experienced.

Jane: It seems like the core idea is that sometimes complex thought just can't be captured through text alone, and they are exploring whether visualizing hypothetical events can help language models do that better.

Lu: The authors are looking at how we can train language models to use these visualization mechanisms in a way that actually helps their reasoning abilities.

Meng: It sounds like they want to give the AI a way to 'see' what it's thinking about when it makes a complex decision, instead of just guessing based on words.

Lalam: It positions this as an extension of existing tool use capabilities, but focusing specifically on visual thought experiments rather than just web search or code execution.

The paper's summary: Tom: So what they propose is these Einstein World Models, where the language model doesn't just generate text; it can call a world-module to produce short video rollouts of the scene it's considering.

Jane: That rollout isn't supposed to be the final answer, but rather an inspectable hypothesis that the model can look at later in its reasoning process.

Lu: It treats this rollout as a measurable component of the reasoning trace, which is really about externalizing those latent assumptions so they become visible.

Meng: So instead of just having hidden internal thoughts, you get a video artifact you can actually examine and study without seeing the model's weights or its hidden activations.

Lalam: It means the system can reason in ways that text alone might not support well, by letting it visualize counterfactual events under consideration.

The paper's improvements: Tom: They have a few ways they improve things here, and one big thing is treating those generated rollouts as inspectable artifacts instead of final answers or just pre-given inputs.

Jane: That externalization makes the reasoning process available for inspection and debugging, which seems really useful for understanding how the model arrives at its conclusions.

Lu: They also focus on selective world-module use, which means they want the model to learn when it's actually useful to invoke that world-module in the first place.

Meng: And they have a way to reward this usage during training using a GRPO style objective, where you optimize the final answer correctness along with an optional reward for calling that world-module.

Lalam: This setup means traces where a visual thought experiment proves useful enough to justify calling the world-module get higher advantages during training because they are better at solving the target tasks.

Conclusion: Tom: So, to wrap up, the main idea of "Einstein World Models" is that some questions require a model to visualize how a scene unfolds, and text alone might not support that thinking well.

Jane: They achieve this by letting the language model call a world-module to produce short video rollouts as inspectable hypotheses within its reasoning trace before giving an answer.

Lu: The path forward really depends on better world-module curation and developing better datasets so we can train models on when and how to invoke these thought experiments.

Meng: And for us in production, the feasibility relies on the fact that language models already have a lot of the world knowledge they need to query a world-module effectively.

Lalam: Ultimately, this paper points toward language models that reason not only through words and symbols but also through these externalized visual walk-throughs.

Munachiso Samuel Nwadike, Zangir Iklassov, Ali Mekky, Zayd M. Kawakibi Zuhri, Kentaro Inui

MBZUAI AI Research Institute of Japan Institute of Advanced Technology Tohoku University

cs.AI, cs.CL, cs.CV

Submitted: 2026-06-25

Updated: 2026-11-10

Comments: 12 pages (9 without references), 2 figures, 1 algorithm

Project page: https://thinking-with-video.github.io

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 84/100

The gist: Does intelligence require the ability to reason about phenomena beyond direct experience? Einstein World Models are a blueprint for LLM-based reasoning systems that place visual-temporal rollouts…

Key concepts

Einstein World Models (EWMs)
A system where an LLM acts as the reasoner but can selectively pause its text generation to call a 'world-module.' This module generates a short video rollout of a scene, which is then added back into the reasoning trace. This lets the model use visual thought experiments to solve problems that are hard to describe with words alone.
World-Module
A component called by the LLM that produces visual data, such as a video rollout. These modules can be Renderers (generating video from text) or Simulators (allowing the reasoner to intervene in a visualized world). The quality of the resulting visual hypothesis depends on the underlying module's physical intuition.
Rollouts as Inspectable Hypotheses
The generated video rollouts are treated as intermediate artifacts, not final answers. Because they are externalized, they can be shared and studied without needing to access the model's internal weights. This externalization makes the model's reasoning process transparent and allows researchers to inspect *why* a certain path was chosen.
Ensembling
A technique where multiple LLM reasoners, each attached to a different world-module, exchange their generated rollouts and critiques before giving a final answer. This comparison reveals disagreements between different visual hypotheses, helping the system understand what aspects of the reasoning need further inspection.

Terminology

Summary

Does intelligence require the ability to reason about phenomena beyond direct experience? p. 0

Where the paper says it

nwadike@mbzuai. p. 0

Where the paper says it

Kawakibi Zuhri1 , Kentaro Inui1,2,3 1MBZUAI 2RIKEN AIP, Japan 3Tohoku University munachiso. nwadike@mbzuai.

ae Abstract Does intelligence require the ability to reason about phenomena beyond direct experience?

Einstein World Models are a blueprint for LLM-based reasoning systems that place visual-temporal rollouts inside the reasoning trace, allowing them to reason in ways that text alone may not support well. p. 0
Where the paper says it

Motivated by this question, we propose Einstein World Models. EWMs are a blueprint for LLM-based reasoning systems that place visual-temporal rollouts inside the reasoning trace, allowing them to reason in ways that text alone may not support well.

Pruning redundant reasoning tokens

Einstein World Models (EWMs) are designed to enable language models to utilize visualising counterfactual events as a mechanism for complex thought. The core idea is that some questions require a model to visualise how a described scene unfolds, and that this may be poorly supported by language alone. p. 7

Where the paper says it

Epilogue This paper proposed Einstein World Models, for treating visual thought experiments as a tool-use behaviour in LLM reasoning. The central idea is that some questions require a model to visualise how a described scene unfolds, and that this may be poorly supported by language alone.

EWMs keep the LLM as the reasoner, but allow it to call a world-module, render a short visual-temporal rollout, and return that rollout to the reasoning trace before answering. p. 7
Where the paper says it

The central idea is that some questions require a model to visualise how a described scene unfolds, and that this may be poorly supported by language alone. EWMs keep the LLM as the reasoner, but allow it to call a world-module, render a short visual-temporal rollout, and return that rollout to the reasoning trace before answering.

p. 7
Where the paper says it

The central idea is that some questions require a model to visualise how a described scene unfolds, and that this may be poorly supported by language alone. EWMs keep the LLM as the reasoner, but allow it to call a world-module, render a short visual-temporal rollout, and return that rollout to the reasoning trace before answering.

The returned video rollout is not assumed to be a perfect simulation, but rather a visual hypothesis about how a described situation may unfold. p. 7
Where the paper says it

EWMs keep the LLM as the reasoner, but allow it to call a world-module, render a short visual-temporal rollout, and return that rollout to the reasoning trace before answering. A video rollout is not assumed to be a perfect simulation, but rather a visual hypothesis about how a described situation may unfold.

How the training signal is built

The training process involves two main stages: supervised fine-tuning and reinforcement learning. At the initial stage, the reasoner undergoes supervised fine-tuning on EWM trace formats with standard next-token cross-entropy, masking returned rollout observations from the loss as detailed in the Supplementary Notes, part A. p. 3

Where the paper says it

The rendered frames remain available for inspection, while the visual tokens provide the representation consumed by the reasoner. 4 Training At the initial stage, the reasoner undergoes supervised fine-tuning on EWM trace formats with standard next-token cross-entropy, masking returned

Consistent with standard practice in recent reasoning-model training (Guo et al. p. 4
Where the paper says it

rollout observations from the loss as detailed in the Supplementary Notes, part A. Consistent with standard practice in recent reasoning-model training (Guo et al.

, 2025), we then recommend following SFT with RLVR-style training over complete EWM trajectories, since target tasks provide verifiable final answers even when intermediate visual thought experiments remain unlabeled. p. 4
Where the paper says it

Consistent with standard practice in recent reasoning-model training (Guo et al. , 2025), we then recommend following SFT with RLVR-style training over complete EWM trajectories, since target tasks provide verifiable final answers even when intermediate visual thought experiments remain unlabeled.

The EWM reward is defined as rM(T, y⋆) = r(ˆy, y⋆) + rW(T), where rW(T) denotes the optional implementation-dependent reward for world-module calling behavior in the trace. p. 4
Where the paper says it

rM(T , y⋆ ) = r(ˆy, y⋆ ) + rW(T ). (2) Here, rW(T ) denotes the optional implementationdependent reward for world-module calling behavior in the trace.

A GRPO-style implementation can optimise this reward with a clipped surrogate to optimize the group-relative advantage Ai = ri − r¯sr + ϵadv, where r¯ and sr are the mean and standard deviation of the ri within τ1:G. p. 4
Where the paper says it

Let ri = rM(τi , y⋆ ). The group-relative advantage of rollout i is Ai = ri − r¯ sr + ϵadv , where r¯ and sr are the mean and standard deviation of the ri within τ1:G.

Where the gains come from

The gains stem from treating generated rollouts as inspectable intermediate artifacts rather than final answers or pre-given inputs. Externalised thought makes latent assumptions visible (Nwadike et al. p. 3

Where the paper says it

In Einstein World Models, the reasoner must learn not only when to visualise, but how to query for the visualised scene that will support later reasoning. 2 Rollouts as Inspectable Hypotheses Externalised thought makes latent assumptions visible (Nwadike et al.

, 2026). This allows the system to render that scene as an examinable video artifact, which can be shared and studied without access to model weights or hidden activations. p. 3
Where the paper says it

Instead of leaving the model’s visualised scene implicit, the system renders that scene as an examinable video artifact, for instance as a sequence of frames. Because this rollout is externalised, it can be shared and studied without access to model weights or hidden activations.

The EWM trace update is defined as Tt+1 = Tt ⊕ (st, if W is not queried, [qt, vt], if W is queried). p. 3
Where the paper says it

If a query qt is generated, the world-module returns a visualtemporal rollout vt ∼ W(qt). The trace update is therefore Tt+1 = Tt ⊕ ( st , if W is not queried, [qt , vt ], if W is queried.

Traces where a visual thought experiment proves useful enough to justify its world-module usage receive higher advantages during training. p. 4
Where the paper says it

3 converts this tracelevel reward into group-relative advantages over complete EWM trajectories. Traces where a visual thought experiment proves useful enough to justify its world-module usage receive higher advantages,

World-Module Selection

The selection of the world-module is crucial for the success of EWMs, as it dictates the quality and utility of the visual hypotheses generated. World-modules may come in several forms, including Renderers, Simulators, and Planners. p. 5

Where the paper says it

3 19: end for 3 World-Module Selection Since RLVR optimises the reasoner’s use of the world-module rather than the world-module itself, the question becomes how to select a module that produces useful visual-temporal thought experiments. 1 Architecture World-modules may come in several forms.

Renderers are the default world-modules in EWMs and include text-to-video generators or image-to-video generators used within a text-to-image pipeline. p. 5
Where the paper says it

In EWMs, renderers are the default world-modules. Text-to-video generators, or image-to-video generators used within a text-to-image pipeline, are of particular interest.

p. 5
Where the paper says it

In EWMs, renderers are the default world-modules. Text-to-video generators, or image-to-video generators used within a text-to-image pipeline, are of particular interest.

Simulators allow the reasoner to intervene in a visualised world and observe what follows, such as Genie-style systems. p. 5
Where the paper says it

Simulators: allow the reasoner to intervene in a visualised world and observe what follows. Interactive world models such as Genie-style systems provide one example of this interface (Bruce et al.

The quality of the rollout depends on the underlying video model's physical intuition, with diffusion models being strong candidates due to their intuitive-physics quality.

Rollouts as Inspectable Hypotheses

The concept of rollouts being inspectable is central to EWMs, distinguishing them from passive visual prediction or action-conditioned prediction. Because this rollout is externalised, it can be shared and studied without access to model weights or hidden activations. p. 3

Where the paper says it

Instead of leaving the model’s visualised scene implicit, the system renders that scene as an examinable video artifact, for instance as a sequence of frames. Because this rollout is externalised, it can be shared and studied without access to model weights or hidden activations.

The distinction between the inspectability of the world-module’s rollout, and the physical plausibility of the rollout is key; a rollout is not considered more plausible solely because it is inspectable. p. 3
Where the paper says it

Key here is the distinction between the inspectability of the world-module’s rollout, and the physical plausibility of the rollout. A rollout is not considered more plausible solely because it is inspectable.

The reasoner must learn not only when to visualise, but how to query for the visualised scene that will support later reasoning. p. 3
Where the paper says it

As with web search, the query helps determine what information is returned. In Einstein World Models, the reasoner must learn not only when to visualise, but how to query for the visualised scene that will support later reasoning.

Related Work

Einstein World Models build upon prior work like Chain-of-thought and Whiteboard-of-Thought by changing the artifact from a static drawing to a short video rollout.<ref:2606.26969#pg2> Einstein World Models seek to provide an analogous capability for language models by treating visualised visual episodes as intermediate reasoning artifacts. p. 6

Where the paper says it

Humans often reason about such variables through visualisation. Einstein World Models seek to provide an analogous capability for language models by treating visualised visual episodes as intermediate reasoning artifacts.

<ref:2606.26969#pg3> The paper distinguishes itself by asking whether an LLM can use video generation as a thought experiment tool, rather than replacing the LLM with a video generator. p. 6
Where the paper says it

Rather than asking whether the video generator can serve as the reasoner, EWMs ask whether an LLM can use video generation as a thought experiment tool. In other words, rather than replacing the LLM with a video generator, we ask how frontier LLMs can decide when visualisation is useful, selectively invoke the world-module, and integrate the resulting rollout back into reasoning.

<ref:2606.26969#pg4> The work is distinct from Visualization-of-Thought (VoT), which focuses on controlled 2D grid-world tasks rather than real-world visualtemporal thought experiments. p. 6
Where the paper says it

(2024) propose Visualization-ofThought (VoT), a related method for spatial reasoning in LLMs. However, VoT focuses on controlled 2D grid-world tasks rather than real-world visualtemporal thought experiments.

<ref:2606.26969#pg5> The paper proposes a mechanism for selective visualtemporal thought experiments instantiated by tooluse behavior in LLM reasoning. p. 1
Where the paper says it

Equally central is the ability to integrate these rollouts back into the LLM’s reasoning trace, as discussed in Section 2. Our analysis of this element forms one of three contributions: First, we propose Einstein World Models as a mechanism for selective visualtemporal thought experiments instantiated by tooluse behavior in LLM reasoning.

<ref:2606.26969#pg2>

Future Work: A Call for Datasets

A central bottleneck for EWMs is data, as few existing datasets explicitly target the behaviour required by the model.<ref:2606.26969#pg2> The missing setting is where an LLM can perform a visual thought experiment in its reasoning trace before outputting final answer tokens. p. 7

Where the paper says it

The missing setting is neither ordinary text reasoning, nor visual question answering over an already provided image. It is a setting in which an LLM is fully capable of taking in text alone as input (even without the aid of visual inputs, when requested), and can perform a visual thought experiment in its reasoning trace before outputting final answer tokens.

<ref:2606.26969#pg2> SimpleBench is one of the rare public datasets pointing in this direction, but it is limited to a little over 200 questions. p. 7
Where the paper says it

SimpleBench is one of the rare public datasets pointing in this direction. However, the full dataset contains only a little over 200 questions, while its public release exposes only 10, making it useful as an illustration, rather than as a training corpus.

<ref:2606.26969#pg3> The paper makes an open call for datasets where the problem begins as language, and the model must decide for itself whether to generate a visualisation. p. 7
Where the paper says it

By contrast, Einstein World Models need datasets where the problem begins as language, and the model must decide, for itself, whether to generate a visualisation. We therefore make an open call for datasets in this setting.

Ensembling

Ensembling extends beyond repeated calls to a single module to describe the comparison of visual hypotheses generated by different world-modules. Because rollouts are externalised, these assumptions can be compared rather than left hidden, allowing different modules to favor different inductive biases. p. 6

Where the paper says it

Because rollouts are externalised, these assumptions can be compared rather than left hidden. The problem is therefore not only to select a strong world-module, but to select modules whose inductive biases are usefully different.

In ensembling, different LLM reasoners attached to different world-modules exchange rollouts and critiques before answering. p. 6
Where the paper says it

The problem is therefore not only to select a strong world-module, but to select modules whose inductive biases are usefully different. In ensembling, different LLM reasoners, attached to different world-modules, exchange rollouts and critiques before answering.

This disagreement reveals what must be inspected next in the reasoning process. p. 6
Where the paper says it

In ensembling, different LLM reasoners, attached to different world-modules, exchange rollouts and critiques before answering. Each rollout proposes a different interpretation of the scene, and disagreement reveals what must be inspected next.

Relation to Other World Models

Einstein World Models shift the emphasis from the notion of a world model to that of a world-module. An EWM is not itself a learned dynamics model, simulator, or video generator; it is a reasoning system in which the LLM remains the reasoner and calls the world-module. p. 8

Where the paper says it

An EWM is not itself a learned dynamics model, simulator, or video generator. It is a reasoning system in which the LLM remains the reasoner and calls the world-module when a visual-temporal thought experiment may support its reasoning trace.

The world-module is treated only as a component of a reasoning system rather than a full 3D simulator. p. 8
Where the paper says it

The world-module is the component most closely aligned with what is traditionally the predictive component of a world model. It need not be a full 3D simulator, and is treated only as a component of a reasoning system.

This distinction shifts what must be learned, focusing on how a reasoner should selectively invoke visualisation to facilitate reasoning. p. 8
Where the paper says it

This distinction also shifts what must be learned. Whereas many world-model architectures focus on modelling environmental dynamics, EWMs focus on how a reasoner should selectively invoke visualisation to facilitate reasoning

Ensemble and World-Module Quality

The paper suggests that the problem is not only to select a strong world-module, but to select modules whose inductive biases are usefully different for ensembling. A generated video rollout may only be as useful as the physical intuitions sustained by its underlying video model. p. 5

Where the paper says it

2 World-Module Quality Einstein’s visual thought experiments were disciplined by strong physical intuition. A generated video rollout may therefore only be as useful as the physical intuitions sustained by its underlying video model.

The paper also requires rollout faithfulness, meaning an unfaithful rollout may not reveal the computation that produced the answer. p. 6
Where the paper says it

depends on. An unfaithful rollout, much like in traditional chain-of-thought, may not reveal the computation that produced the answer (Turpin et al.

Ensembling allows for comparing visual hypotheses generated by different world-modules, such as one favoring visual realism and another physical consistency. p. 6
Where the paper says it

Here, ensembling extends beyond repeated calls to a single module to describe the comparison of visual hypotheses generated by different worldmodules, much as different humans may visualise the same problem differently. One module may favour visual realism, another physical consistency, and another temporal continuity.

Feasibility

The feasibility of this approach relies on the fact that LLMs already contain much of the world knowledge needed to query a world-module effectively. The unidirectional nature of autoregression is surprisingly robust in practice, and it is reflected in models that combine autoregressive prediction with latent diffusion sampling. p. 4

Where the paper says it

The unidirectional nature of autoregression is surprisingly robust in practice (Nwadike et al. It is also reflected in models that combine autoregressive prediction with latent diffusion sampling (Parker-Holder et al.

Targeted post-training, rather than fresh pretraining from scratch, is suggested given the existing structured representations LLMs can acquire from language alone. p. 4
Where the paper says it

The LLM already contains much of the world knowledge needed to query a worldmodule effectively. Thus, the remaining task is not necessarily fresh pretraining from scratch, but targeted post-training.

The paper points toward language models that reason not only through words and symbols, but through externalised visual walk-throughs. p. 7
Where the paper says it

The path forward thus requires both better world-module curation, as well as better datasets to allow for training LLMs on when and how thought experiments should be invoked. In this sense, Einstein World Models point toward language models that reason not only through words and symbols, but through externalised visual walk-throughs.

p. 7
Where the paper says it

The path forward thus requires both better world-module curation, as well as better datasets to allow for training LLMs on when and how thought experiments should be invoked. In this sense, Einstein World Models point toward language models that reason not only through words and symbols, but through externalised visual walk-throughs.

Implications for Agentic Systems

An EWM can be incorporated into an agentic system, but it is not necessarily an agent in itself. Such systems may qualify as Einstein World Models when they use a world-module to generate visualtemporal rollouts as intermediate reasoning artifacts. p. 8

Where the paper says it

However, although an EWM can be incorporated into an agentic system, it is not necessarily an agent in itself. Such systems may qualify as Einstein World Models when they use a world-module to generate visualtemporal rollouts as intermediate reasoning artifacts, but not merely by virtue of being agents.

An EWM need not have the broader infrastructure of a general-purpose agent for executing user-facing actions, such as managing software repositories or querying databases. p. 8
Where the paper says it

Existing agentic systems can call tools, but they do not by default perform externalised visual thought experiments in the sense proposed here. Conversely, an Einstein World Model need not have the broader infrastructure of a general-purpose agent for executing user-facing actions, such as managing software repositories or querying databases.

It may be implemented as a language reasoner that calls a world-module only to imagine a described scene, inspect the resulting rollout, and incorporate it into its reasoning trace. p. 8
Where the paper says it

Conversely, an Einstein World Model need not have the broader infrastructure of a general-purpose agent for executing user-facing actions, such as managing software repositories or querying databases. It may be implemented as a language reasoner that calls a world-module only to imagine a described scene, inspect the resulting rollout, and incorporate it into its reasoning trace.

The paper discusses the relationship between Einstein World Models and contemporary agentic LLM systems in the Supplementary Notes, Part B. p. 6
Where the paper says it

This differs from the setting of interest here, where thought experiments are used to support text-based reasoning problems and the model must decide, as part of its own reasoning process, when visualtemporal rollouts are useful. We discuss the relationship between Einstein World Models and contemporary agentic LLM systems in the Supplementary Notes, Part B.

<ref:2606.26969#pg5>

Final Summary

The central idea is that some questions require a model to visualise how a described scene unfolds, and that this may be poorly supported by language alone. EWMs keep the LLM as the reasoner, but allow it to call a world-module, render a short visual-temporal rollout, and return that rollout to the reasoning trace before answering. A video rollout is not assumed to be a perfect simulation, but rather a visual hypothesis about how a described situation may unfold. p. 7

Where the paper says it

EWMs keep the LLM as the reasoner, but allow it to call a world-module, render a short visual-temporal rollout, and return that rollout to the reasoning trace before answering. A video rollout is not assumed to be a perfect simulation, but rather a visual hypothesis about how a described situation may unfold.

Its distinctive usefulness comes from being externalised, making the model’s reasoning process available for inspection and debugging. p. 7
Where the paper says it

A video rollout is not assumed to be a perfect simulation, but rather a visual hypothesis about how a described situation may unfold. Its distinctive usefulness comes from being externalised, making the model’s reasoning process available for inspection and debugging.

The path forward requires both better world-module curation, as well as better datasets to allow for training LLMs on when and how thought experiments should be invoked. p. 7
Where the paper says it

Its distinctive usefulness comes from being externalised, making the model’s reasoning process available for inspection and debugging. The path forward thus requires both better world-module curation, as well as better datasets to allow for training LLMs on when and how thought experiments should be invoked.

The paper points toward language models that reason not only through words and symbols, but through externalised visual walk-throughs.

--- Page 1 ---

Einstein World Models

Munachiso Samuel Nwadike1,2

, Zangir Iklassov1

, Ali Mekky1

Zayd M. Kawakibi Zuhri1

, Kentaro Inui1,2,3

1MBZUAI 2RIKEN AIP, Japan 3Tohoku University

munachiso.nwadike@mbzuai.ac.ae

--- Page 2 ---

Abstract

Does intelligence require the ability to reason about phenomena beyond direct experience? It is natural to suspect that some complex thought cannot be captured through language alone. p. 0

Where the paper says it

ae Abstract Does intelligence require the ability to reason about phenomena beyond direct experience? It is natural to suspect that some complex thought cannot be captured through language alone.

However, of particular concern to this work, is whether visualising counterfactual events can complement language as a mechanism for complex thought. p. 0
Where the paper says it

It is natural to suspect that some complex thought cannot be captured through language alone. However, of particular concern to this work, is whether visualising counterfactual events can complement language as a mechanism for complex thought.

We ask whether LLMs can be trained to utilise such visualisation mechanisms, in a way that benefits their reasoning abilities. p. 0
Where the paper says it

However, of particular concern to this work, is whether visualising counterfactual events can complement language as a mechanism for complex thought. We ask whether LLMs can be trained to utilise such visualisation mechanisms, in a way that benefits their reasoning abilities.

Motivated by this question, we propose Einstein World Models. p. 0
Where the paper says it

We ask whether LLMs can be trained to utilise such visualisation mechanisms, in a way that benefits their reasoning abilities. Motivated by this question, we propose Einstein World Models.

EWMs are a blueprint for LLM-based reasoning systems that place visual-temporal rollouts inside the reasoning trace, allowing them to reason in ways that text alone may not support well. p. 0
Where the paper says it

Motivated by this question, we propose Einstein World Models. EWMs are a blueprint for LLM-based reasoning systems that place visual-temporal rollouts inside the reasoning trace, allowing them to reason in ways that text alone may not support well.

In an EWM, the LLM calls a world-module (not to be confused with a world model), to produce short rollouts of scenes under consideration. p. 0
Where the paper says it

EWMs are a blueprint for LLM-based reasoning systems that place visual-temporal rollouts inside the reasoning trace, allowing them to reason in ways that text alone may not support well. In an EWM, the LLM calls a world-module (not to be confused with a world model), to produce short rollouts of scenes under consideration.

The returned rollout is treated not as the answer, but as an inspectable hypothesis that can support later reasoning. p. 0
Where the paper says it

In an EWM, the LLM calls a world-module (not to be confused with a world model), to produce short rollouts of scenes under consideration. The returned rollout is treated not as the answer, but as an inspectable hypothesis that can support later reasoning.

Einstein World Models extend the capability of LLMs for tool calling (such as web search or code execution), into the domain of visual thought experiments. p. 0
Where the paper says it

The returned rollout is treated not as the answer, but as an inspectable hypothesis that can support later reasoning. Einstein World Models extend the capability of LLMs for tool calling (such as web search or code execution), into the domain of visual thought experiments.

--- Page 3 ---

Prologue

A call for datasets becomes meaningful after the desirable capability has been specified. This culminates in Section 5. p. 0

Where the paper says it

Prologue A call for datasets becomes meaningful after the desirable capability has been specified. This culminates in Section 5.

The predominance of this work therefore aims to motivate a learnable format for visual thought experimentation, proposing the architecture and training objectives necessitating their training data. p. 0
Where the paper says it

This culminates in Section 5. The predominance of this work therefore aims to motivate a learnable format for visual thought experimentation, proposing the architecture and training objectives necessitating their training data.

--- Page 4 ---

1 Introduction

Scientific invention often makes abstract ideas tractable by turning them into thought experiments. In a thought experiment, we visualise a scene, let it unfold, and notice what changes. p. 0

Where the paper says it

1 Introduction Scientific invention often makes abstract ideas tractable by turning them into thought experiments. In a thought experiment, we visualise a scene, let it unfold, and notice what changes.

Einstein’s recollection of special relativity begins with precisely such a thought experiment, later popularised through the image of Bern’s famous clock tower. p. 0
Where the paper says it

In a thought experiment, we visualise a scene, let it unfold, and notice what changes. Einstein’s recollection of special relativity begins with precisely such a thought experiment, later popularised through the image of Bern’s famous clock tower.

In his Autobiographical Notes (Einstein, 1949), he recalls imagining what it would be like to chase a beam of light. p. 0
Where the paper says it

Einstein’s recollection of special relativity begins with precisely such a thought experiment, later popularised through the image of Bern’s famous clock tower. In his Autobiographical Notes (Einstein, 1949), he recalls imagining what it would be like to chase a beam of light.

If he could accelerate until he matched the speed of the beam, would it eventually appear to hang motionless be side him? p. 0
Where the paper says it

In his Autobiographical Notes (Einstein, 1949), he recalls imagining what it would be like to chase a beam of light. If he could accelerate until he matched the speed of the beam, would it eventually appear to hang motionless beFigure 1: Einstein World Models (proposed) build upon traditional LLM reasoning traces.

In those notes, Einstein writes that such a stationary light wave seemed impossible both empirically and according to Maxwell’s equations. p. 1
Where the paper says it

In those notes, Einstein writes that such a stationary light wave seemed impossible both empirically and according to Maxwell’s equations. His simple thought experiment had transformed an abstract tension between electrodynamics and intuitions about motion, into a concrete scene upon which reasoning could be built.

His simple thought experiment had transformed an abstract tension between electrodynamics and intuitions about motion, into a concrete scene upon which reasoning could be built. p. 1
Where the paper says it

In those notes, Einstein writes that such a stationary light wave seemed impossible both empirically and according to Maxwell’s equations. His simple thought experiment had transformed an abstract tension between electrodynamics and intuitions about motion, into a concrete scene upon which reasoning could be built.

--- Page 5 ---

2 Einstein World Models

2.1 Overview

The objectives of Einstein World Models are twofold. First, we wish to imbue language models with the ability to construct visualisations when answering questions that require physical intuition or scene-level visualisation. p. 1

Where the paper says it

1 Overview The objectives of Einstein World Models are twofold. First, we wish to imbue language models with the ability to construct visualisations when answering questions that require physical intuition or scene-level visualisation.

Second, we wish to obtain a window into those visualisations. p. 1
Where the paper says it

First, we wish to imbue language models with the ability to construct visualisations when answering questions that require physical intuition or scene-level visualisation. Second, we wish to obtain a window into those visualisations.

An EWM rollout, as illustrated in Figure 1, is in effect, a visible hypothesis about how a described scene might unfold. p. 1
Where the paper says it

Second, we wish to obtain a window into those visualisations. An EWM rollout, as illustrated in Figure 1, is in effect, a visible hypothesis about how a described scene might unfold.

--- Page 6 ---

Figure 1: Einstein World Models (proposed) build upon traditional LLM reasoning traces. However, in addition to generating tokens across N autoregressive steps, the model may, at a sparse set of M intermediate steps, invoke a callable world-module. p. 0

Where the paper says it

If he could accelerate until he matched the speed of the beam, would it eventually appear to hang motionless beFigure 1: Einstein World Models (proposed) build upon traditional LLM reasoning traces. However, in addition to generating tokens across N autoregressive steps, the model may, at a sparse set of M intermediate steps, invoke a callable world-module.

The returned visual-temporal rollout becomes part of the trace as an inspectable hypothesis. p. 0
Where the paper says it

However, in addition to generating tokens across N autoregressive steps, the model may, at a sparse set of M intermediate steps, invoke a callable world-module. The returned visual-temporal rollout becomes part of the trace as an inspectable hypothesis.

--- Page 7 ---

Figure 2: If Einstein were travelling away from the Bern clock tower at the speed of light, would the clock’s hands appear frozen in time? This figure compares how different reasoning paradigms would support such a thought experiment. p. 2

Where the paper says it

” EWM Thoughts Answer Question Question Thoughts Answer Answer Answer Question Figure 2: If Einstein were travelling away from the Bern clock tower at the speed of light, would the clock’s hands appear frozen in time? This figure compares how different reasoning paradigms would support such a thought experiment.

Bottom right: In Einstein World Models, by contrast, the LLM (the reasoner) preserves autoregressive text generation while taking responsibility for deciding when to invoke a world-module M times, how to query it, and how to incorporate each returned rollout into subsequent reasoning. p. 2
Where the paper says it

Bottom left: In VL-JEPA-style world models, predictive visual representations are learned from observed visual inputs, but the visual prediction itself is not selectively invoked as an intermediate reasoning artifact. Bottom right: In Einstein World Models, by contrast, the LLM (the reasoner) preserves autoregressive text generation while taking responsibility for deciding when to invoke a world-module M times, how to query it, and how to incorporate each returned rollout into subsequent reasoning.

--- Page 8 ---

2.3 Inference

At inference time, an Einstein World Model is specified by two core components πθ and W. Here, πθ denotes the Einstein reasoner, a trainable LLM policy parameterised by θ, and W denotes the world-module. p. 3

Where the paper says it

3 Inference At inference time, an Einstein World Model is specified by two core components πθ and W. Here, πθ denotes the Einstein reasoner, a trainable LLM policy parameterised by θ, and W denotes the world-module.

If queried, W generates these video rollouts, and returns them to πθ for further autoregression. p. 3
Where the paper says it

πθ generates a reasoning trace and may query W for visual-temporal rollouts. If queried, W generates these video rollouts, and returns them to πθ for further autoregression.

--- Page 9 ---

2.4 Training

At the initial stage, the reasoner undergoes supervised fine-tuning on EWM trace formats with standard next-token cross-entropy, masking returned rollout observations from the loss as detailed in the Supplementary Notes, part A. We then recommend following SFT with RLVR-style training over complete EWM trajectories, since target tasks provide verifiable final answers even when intermediate visual thought experiments remain unlabeled. p. 4

Where the paper says it

Consistent with standard practice in recent reasoning-model training (Guo et al. , 2025), we then recommend following SFT with RLVR-style training over complete EWM trajectories, since target tasks provide verifiable final answers even when intermediate visual thought experiments remain unlabeled.

The EWM training objective is JE(θ) = Ex, τ1:G 1/G X G i=1 1 L g i X Li t=1 1it min(ρitAi, clipϵ(ρit)Ai) − βDKL(πθ∥πref). p. 4
Where the paper says it

Define L g i = PLi t=1 1it, and, for generated tokens, ρit = πθ(zit | τi,<t) πold(zit | τi,<t) . Writing clipϵ (r) = clip(r, 1 − ϵ, 1 + ϵ), the EWM training objective is JE(θ) = Ex, τ1:G " 1 G X G i=1 1 L g i X Li t=1 1it min(ρitAi , clipϵ (ρit)Ai) − βDKL(πθ∥πref) # .

p. 4
Where the paper says it

Define L g i = PLi t=1 1it, and, for generated tokens, ρit = πθ(zit | τi,<t) πold(zit | τi,<t) . Writing clipϵ (r) = clip(r, 1 − ϵ, 1 + ϵ), the EWM training objective is JE(θ) = Ex, τ1:G " 1 G X G i=1 1 L g i X Li t=1 1it min(ρitAi , clipϵ (ρit)Ai) − βDKL(πθ∥πref) # .

--- Page 9 ---

A GRPO-style implementation (Shao et al., 2024) can optimise this reward with a clipped surrogate. p. 4

Where the paper says it

A GRPO-style implementation (Shao et al. , 2024) can optimise this reward with a clipped surrogate.

The EWM training objective is JE(θ) = Ex, τ1:G 1/G X G i=1 1 L g i X Li t=1 1it min(ρitAi, clipϵ(ρit)Ai) − βDKL(πθ∥πref).

--- Page 9 ---

Algorithm 1 Proposed EWM RLVR protocol

Require: Dataset D, reasoner πθ, reference policy πref, world-module W, group size G 1: for each GRPO update round do 2: Set frozen old policy πold ← πθ 3: Sample training batch from D 4: for each (x, y⋆) in the batch do 5: Sample G trajectories τ1:G using πold with access to W 6: for each trajectory τi do 7: Generate,, or segments 8: if the world-module is invoked then 9: Call W with the query in 10: Append rollout as a segment 11: end if

--- Page 9 ---

3 World-Module Selection

Since RLVR optimises the reasoner’s use of the world-module rather than the world-module itself, the question becomes how to select a module that produces useful visual thought experiments. World-modules may come in several forms: Renderers, Simulators, and Planners. p. 5

Where the paper says it

3 19: end for 3 World-Module Selection Since RLVR optimises the reasoner’s use of the world-module rather than the world-module itself, the question becomes how to select a module that produces useful visual-temporal thought experiments. 1 Architecture World-modules may come in several forms.

Renderers are the default world-modules in EWMs and include text-to-video generators or image-to-video generators used within a text-to-image pipeline. Diffusion models remain strong candidates because their intuitive-physics quality can be measured through human-verifiable likelihood estimates derived from the denoising objective of the diffusion model. p. 5
Where the paper says it

A generated video rollout may therefore only be as useful as the physical intuitions sustained by its underlying video model. Diffusion models remain strong candidates because their intuitive-physics quality can be measured through human-verifiable likelihood estimates derived from the denoising objective of the diffusion model.

--- Page 9 ---

World-Module Quality

Einstein’s visual thought experiments were disciplined by strong physical intuition, meaning a generated video rollout may only be as useful as the physical intuitions sustained by its underlying video model. Einstein World Models also require rollout faithfulness, exposing information that the reasoner actually uses and that the final answer … p. 5

Where the paper says it

Einstein World Models also require rollout faithfulness. A faithful rollout exposes information that the reasoner actually uses and that the final answer

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Externalized reasoning trace components allow LLMs to reason in ways text alone may not support well by invoking a callable world-module to produce short rollouts of scenes under consideration.

  2. The system can be trained to utilize visual-temporal thought experiment traces by employing a GRPO-style training objective that optimizes an EWM reward combining final-answer correctness with an optional additional reward term for world-module usage.

  3. The improved AI system can perform selective world-module use, as the RLVR protocol converts tracelevel rewards into group relative advantages, so Traces where a visual thought experiment proves useful enough to justify its world-module usage receive higher advantages.

  4. The AI system can distinguish between different types of world-modules, such as Renderers and Simulators, allowing it to choose modules whose inductive biases are usefully different for ensembling and comparison of visual hypotheses.

  5. The system can incorporate diverse visual hypotheses by employing ensembling where different LLM reasoners, attached to different world-modules, exchange rollouts and critiques before answering.

Abstract

Does intelligence require the ability to reason about phenomena beyond direct experience? It is natural to suspect that some complex thought cannot be captured through language alone. However, of particular concern to this work, is whether visualising counterfactual events can complement language as a mechanism for complex thought. We ask whether LLMs can be trained to utilise such visualisation mechanisms, in a way that benefits their reasoning abilities. Motivated by this question, we propose Einstein World Models. EWMs are a blueprint for LLM-based reasoning systems that place visual-temporal rollouts inside the reasoning trace, allowing them to reason in ways that text alone may not support well. In an EWM, the LLM calls a world-module (not to be confused with a world model), to produce short rollouts of scenes under consideration. The returned rollout is treated not as the answer, but as an inspectable hypothesis that can support later reasoning. Einstein World Models extend the capability of LLMs for tool calling (such as web search or code execution), into the domain of visual thought experiments.

Sources

Related papers