Einstein World Models
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "Einstein World Models".
Jane: Here's the gist of the paper. Does intelligence require the ability to reason about phenomena beyond direct experience?
Tom: Einstein World Models are a blueprint for LLM-based reasoning systems that place visual-temporal rollouts inside the reasoning trace, allowing them to reason in ways that text alone may not support well.
Jane: Pruning redundant reasoning tokens Einstein World Models (EWMs) are designed to enable language models to utilize visualising counterfactual events as a mechanism for complex thought.
Tom: The core idea is that some questions require a model to visualise how a described scene unfolds, and that this may be poorly supported by language alone.
Title and authors: Tom: So we're looking at this paper called "Einstein World Models," and it's basically asking if intelligence needs the ability to reason about things you haven't directly experienced.
Jane: It seems like the core idea is that sometimes complex thought just can't be captured through text alone, and they are exploring whether visualizing hypothetical events can help language models do that better.
Lu: The authors are looking at how we can train language models to use these visualization mechanisms in a way that actually helps their reasoning abilities.
Meng: It sounds like they want to give the AI a way to 'see' what it's thinking about when it makes a complex decision, instead of just guessing based on words.
Lalam: It positions this as an extension of existing tool use capabilities, but focusing specifically on visual thought experiments rather than just web search or code execution.
The paper's summary: Tom: So what they propose is these Einstein World Models, where the language model doesn't just generate text; it can call a world-module to produce short video rollouts of the scene it's considering.
Jane: That rollout isn't supposed to be the final answer, but rather an inspectable hypothesis that the model can look at later in its reasoning process.
Lu: It treats this rollout as a measurable component of the reasoning trace, which is really about externalizing those latent assumptions so they become visible.
Meng: So instead of just having hidden internal thoughts, you get a video artifact you can actually examine and study without seeing the model's weights or its hidden activations.
Lalam: It means the system can reason in ways that text alone might not support well, by letting it visualize counterfactual events under consideration.
The paper's improvements: Tom: They have a few ways they improve things here, and one big thing is treating those generated rollouts as inspectable artifacts instead of final answers or just pre-given inputs.
Jane: That externalization makes the reasoning process available for inspection and debugging, which seems really useful for understanding how the model arrives at its conclusions.
Lu: They also focus on selective world-module use, which means they want the model to learn when it's actually useful to invoke that world-module in the first place.
Meng: And they have a way to reward this usage during training using a GRPO style objective, where you optimize the final answer correctness along with an optional reward for calling that world-module.
Lalam: This setup means traces where a visual thought experiment proves useful enough to justify calling the world-module get higher advantages during training because they are better at solving the target tasks.
Conclusion: Tom: So, to wrap up, the main idea of "Einstein World Models" is that some questions require a model to visualize how a scene unfolds, and text alone might not support that thinking well.
Jane: They achieve this by letting the language model call a world-module to produce short video rollouts as inspectable hypotheses within its reasoning trace before giving an answer.
Lu: The path forward really depends on better world-module curation and developing better datasets so we can train models on when and how to invoke these thought experiments.
Meng: And for us in production, the feasibility relies on the fact that language models already have a lot of the world knowledge they need to query a world-module effectively.
Lalam: Ultimately, this paper points toward language models that reason not only through words and symbols but also through these externalized visual walk-throughs.
Munachiso Samuel Nwadike, Zangir Iklassov, Ali Mekky, Zayd M. Kawakibi Zuhri, Kentaro Inui
MBZUAI AI Research Institute of Japan Institute of Advanced Technology Tohoku University
cs.AI, cs.CL, cs.CV
Submitted: 2026-06-25
Updated: 2026-11-10
Comments: 12 pages (9 without references), 2 figures, 1 algorithm
Project page: https://thinking-with-video.github.io
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 84/100
The gist: Does intelligence require the ability to reason about phenomena beyond direct experience? Einstein World Models are a blueprint for LLM-based reasoning systems that place visual-temporal rollouts…
Key concepts
- Einstein World Models (EWMs)
- A system where an LLM acts as the reasoner but can selectively pause its text generation to call a 'world-module.' This module generates a short video rollout of a scene, which is then added back into the reasoning trace. This lets the model use visual thought experiments to solve problems that are hard to describe with words alone.
- World-Module
- A component called by the LLM that produces visual data, such as a video rollout. These modules can be Renderers (generating video from text) or Simulators (allowing the reasoner to intervene in a visualized world). The quality of the resulting visual hypothesis depends on the underlying module's physical intuition.
- Rollouts as Inspectable Hypotheses
- The generated video rollouts are treated as intermediate artifacts, not final answers. Because they are externalized, they can be shared and studied without needing to access the model's internal weights. This externalization makes the model's reasoning process transparent and allows researchers to inspect *why* a certain path was chosen.
- Ensembling
- A technique where multiple LLM reasoners, each attached to a different world-module, exchange their generated rollouts and critiques before giving a final answer. This comparison reveals disagreements between different visual hypotheses, helping the system understand what aspects of the reasoning need further inspection.
Terminology
Summary
Does intelligence require the ability to reason about phenomena beyond direct experience? p. 0 nwadike@mbzuai. p. 0 Kawakibi Zuhri1
, Kentaro Inui1,2,3
1MBZUAI 2RIKEN AIP, Japan 3Tohoku University
munachiso. nwadike@mbzuai. Motivated by this question,
we propose Einstein World Models. EWMs
are a blueprint for LLM-based reasoning systems that place visual-temporal rollouts inside
the reasoning trace, allowing them to reason in
ways that text alone may not support well.Where the paper says it
Where the paper says it
Where the paper says it
Pruning redundant reasoning tokens
Einstein World Models (EWMs) are designed to enable language models to utilize visualising counterfactual events as a mechanism for complex thought. The core idea is that some questions require a model to visualise how a described scene unfolds, and that this may be poorly supported by language alone. p. 7 Epilogue
This paper proposed Einstein World Models, for
treating visual thought experiments as a tool-use
behaviour in LLM reasoning. The central idea is
that some questions require a model to visualise
how a described scene unfolds, and that this may
be poorly supported by language alone. The central idea is
that some questions require a model to visualise
how a described scene unfolds, and that this may
be poorly supported by language alone. EWMs
keep the LLM as the reasoner, but allow it to call a
world-module, render a short visual-temporal rollout, and return that rollout to the reasoning trace
before answering. The central idea is
that some questions require a model to visualise
how a described scene unfolds, and that this may
be poorly supported by language alone. EWMs
keep the LLM as the reasoner, but allow it to call a
world-module, render a short visual-temporal rollout, and return that rollout to the reasoning trace
before answering. EWMs
keep the LLM as the reasoner, but allow it to call a
world-module, render a short visual-temporal rollout, and return that rollout to the reasoning trace
before answering. A video rollout is not assumed
to be a perfect simulation, but rather a visual hypothesis about how a described situation may unfold.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
How the training signal is built
The training process involves two main stages: supervised fine-tuning and reinforcement learning. At the initial stage, the reasoner undergoes supervised fine-tuning on EWM trace formats with standard next-token cross-entropy, masking returned rollout observations from the loss as detailed in the Supplementary Notes, part A. p. 3 The
rendered frames remain available for inspection,
while the visual tokens provide the representation
consumed by the reasoner. 4 Training
At the initial stage, the reasoner undergoes supervised fine-tuning on EWM trace formats with standard next-token cross-entropy, masking returned rollout observations from the loss as detailed in the
Supplementary Notes, part A. Consistent with standard practice in recent
reasoning-model training (Guo et al. Consistent with standard practice in recent
reasoning-model training (Guo et al. , 2025), we
then recommend following SFT with RLVR-style
training over complete EWM trajectories, since
target tasks provide verifiable final answers even
when intermediate visual thought experiments remain unlabeled. rM(T , y⋆
) = r(ˆy, y⋆
) + rW(T ). (2)
Here, rW(T ) denotes the optional implementationdependent reward for world-module calling behavior in the trace. Let ri = rM(τi
, y⋆
). The group-relative advantage of rollout i is
Ai =
ri − r¯
sr + ϵadv
,
where r¯ and sr are the mean and standard deviation
of the ri within τ1:G.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the gains come from
The gains stem from treating generated rollouts as inspectable intermediate artifacts rather than final answers or pre-given inputs. Externalised thought makes latent assumptions visible (Nwadike et al. p. 3 In Einstein World Models, the reasoner must learn
not only when to visualise, but how to query for the
visualised scene that will support later reasoning. 2 Rollouts as Inspectable Hypotheses
Externalised thought makes latent assumptions visible (Nwadike et al. Instead of leaving the model’s
visualised scene implicit, the system renders that
scene as an examinable video artifact, for instance
as a sequence of frames. Because this rollout is externalised, it can be
shared and studied without access to model weights
or hidden activations. If a query
qt
is generated, the world-module returns a visualtemporal rollout
vt ∼ W(qt). The trace update is therefore
Tt+1 = Tt ⊕
(
st
, if W is not queried,
[qt
, vt
], if W is queried. 3 converts this tracelevel reward into group-relative advantages over
complete EWM trajectories. Traces where a visual
thought experiment proves useful enough to justify
its world-module usage receive higher advantages,Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
World-Module Selection
The selection of the world-module is crucial for the success of EWMs, as it dictates the quality and utility of the visual hypotheses generated. World-modules may come in several forms, including Renderers, Simulators, and Planners. p. 5 3
19: end for
3 World-Module Selection
Since RLVR optimises the reasoner’s use of the
world-module rather than the world-module itself,
the question becomes how to select a module that
produces useful visual-temporal thought experiments. 1 Architecture
World-modules may come in several forms. In EWMs, renderers are the
default world-modules. Text-to-video generators, or image-to-video generators used within a
text-to-image pipeline, are of particular interest. In EWMs, renderers are the
default world-modules. Text-to-video generators, or image-to-video generators used within a
text-to-image pipeline, are of particular interest. Simulators: allow the reasoner to intervene in
a visualised world and observe what follows. Interactive world models such as Genie-style
systems provide one example of this interface
(Bruce et al.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Rollouts as Inspectable Hypotheses
The concept of rollouts being inspectable is central to EWMs, distinguishing them from passive visual prediction or action-conditioned prediction. Because this rollout is externalised, it can be shared and studied without access to model weights or hidden activations. p. 3 Instead of leaving the model’s
visualised scene implicit, the system renders that
scene as an examinable video artifact, for instance
as a sequence of frames. Because this rollout is externalised, it can be
shared and studied without access to model weights
or hidden activations. Key here is the distinction between the inspectability of the world-module’s rollout, and the
physical plausibility of the rollout. A rollout is
not considered more plausible solely because it
is inspectable. As with web search, the
query helps determine what information is returned. In Einstein World Models, the reasoner must learn
not only when to visualise, but how to query for the
visualised scene that will support later reasoning.Where the paper says it
Where the paper says it
Where the paper says it
Related Work
Einstein World Models build upon prior work like Chain-of-thought and Whiteboard-of-Thought by changing the artifact from a static drawing to a short video rollout.<ref:2606.26969#pg2> Einstein World Models seek to provide an analogous capability for language models by treating visualised visual episodes as intermediate reasoning artifacts. p. 6 Humans often reason
about such variables through visualisation. Einstein World Models seek to provide an analogous
capability for language models by treating visualised visual episodes as intermediate reasoning
artifacts. Rather than asking
whether the video generator can serve as the reasoner, EWMs ask whether an LLM can use video
generation as a thought experiment tool. In other
words, rather than replacing the LLM with a video
generator, we ask how frontier LLMs can decide
when visualisation is useful, selectively invoke the
world-module, and integrate the resulting rollout
back into reasoning. (2024) propose Visualization-ofThought (VoT), a related method for spatial reasoning in LLMs. However, VoT focuses on controlled
2D grid-world tasks rather than real-world visualtemporal thought experiments. Equally central is the ability to integrate these rollouts back into the LLM’s
reasoning trace, as discussed in Section 2. Our analysis of this element forms one of
three contributions: First, we propose Einstein
World Models as a mechanism for selective visualtemporal thought experiments instantiated by tooluse behavior in LLM reasoning.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Future Work: A Call for Datasets
A central bottleneck for EWMs is data, as few existing datasets explicitly target the behaviour required by the model.<ref:2606.26969#pg2> The missing setting is where an LLM can perform a visual thought experiment in its reasoning trace before outputting final answer tokens. p. 7 The
missing setting is neither ordinary text reasoning,
nor visual question answering over an already provided image. It is a setting in which an LLM is
fully capable of taking in text alone as input (even
without the aid of visual inputs, when requested),
and can perform a visual thought experiment in
its reasoning trace before outputting final answer
tokens. SimpleBench is one of the rare public datasets
pointing in this direction. However, the full dataset
contains only a little over 200 questions, while its
public release exposes only 10, making it useful as
an illustration, rather than as a training corpus. By contrast, Einstein
World Models need datasets where the problem begins as language, and the model must decide, for
itself, whether to generate a visualisation. We therefore make an open call for datasets in
this setting.Where the paper says it
Where the paper says it
Where the paper says it
Ensembling
Ensembling extends beyond repeated calls to a single module to describe the comparison of visual hypotheses generated by different world-modules. Because rollouts are externalised, these assumptions can be compared rather than left hidden, allowing different modules to favor different inductive biases. p. 6 Because rollouts
are externalised, these assumptions can be compared rather than left hidden. The problem is therefore not only to select a strong world-module, but to
select modules whose inductive biases are usefully
different. The problem is therefore not only to select a strong world-module, but to
select modules whose inductive biases are usefully
different. In ensembling, different LLM reasoners, attached to different world-modules, exchange rollouts and critiques before answering. In ensembling, different LLM reasoners, attached to different world-modules, exchange rollouts and critiques before answering. Each rollout
proposes a different interpretation of the scene, and
disagreement reveals what must be inspected next.Where the paper says it
Where the paper says it
Where the paper says it
Relation to Other World Models
Einstein World Models shift the emphasis from the notion of a world model to that of a world-module. An EWM is not itself a learned dynamics model, simulator, or video generator; it is a reasoning system in which the LLM remains the reasoner and calls the world-module. p. 8 An EWM is not itself a learned dynamics
model, simulator, or video generator. It is a reasoning system in which the LLM remains the reasoner
and calls the world-module when a visual-temporal
thought experiment may support its reasoning trace. The world-module is the component most closely
aligned with what is traditionally the predictive
component of a world model. It need not be a full
3D simulator, and is treated only as a component
of a reasoning system. This distinction also shifts what must be learned. Whereas many world-model architectures focus on
modelling environmental dynamics, EWMs focus
on how a reasoner should selectively invoke visualisation to facilitate reasoningWhere the paper says it
Where the paper says it
Where the paper says it
Ensemble and World-Module Quality
The paper suggests that the problem is not only to select a strong world-module, but to select modules whose inductive biases are usefully different for ensembling. A generated video rollout may only be as useful as the physical intuitions sustained by its underlying video model. p. 5 2 World-Module Quality
Einstein’s visual thought experiments were disciplined by strong physical intuition. A generated
video rollout may therefore only be as useful as
the physical intuitions sustained by its underlying
video model. depends on. An unfaithful rollout, much like in traditional chain-of-thought, may not reveal the computation that produced the answer (Turpin et al. Here, ensembling extends beyond repeated calls
to a single module to describe the comparison of
visual hypotheses generated by different worldmodules, much as different humans may visualise
the same problem differently. One module may
favour visual realism, another physical consistency,
and another temporal continuity.Where the paper says it
Where the paper says it
Where the paper says it
Feasibility
The feasibility of this approach relies on the fact that LLMs already contain much of the world knowledge needed to query a world-module effectively. The unidirectional nature of autoregression is surprisingly robust in practice, and it is reflected in models that combine autoregressive prediction with latent diffusion sampling. p. 4 The unidirectional nature of autoregression is surprisingly robust in practice (Nwadike
et al. It is also reflected in models that combine autoregressive prediction with latent diffusion
sampling (Parker-Holder et al. The LLM already contains much
of the world knowledge needed to query a worldmodule effectively. Thus, the remaining task is
not necessarily fresh pretraining from scratch, but
targeted post-training. The path
forward thus requires both better world-module
curation, as well as better datasets to allow for
training LLMs on when and how thought experiments should be invoked. In this sense, Einstein
World Models point toward language models that
reason not only through words and symbols, but
through externalised visual walk-throughs. The path
forward thus requires both better world-module
curation, as well as better datasets to allow for
training LLMs on when and how thought experiments should be invoked. In this sense, Einstein
World Models point toward language models that
reason not only through words and symbols, but
through externalised visual walk-throughs.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Implications for Agentic Systems
An EWM can be incorporated into an agentic system, but it is not necessarily an agent in itself. Such systems may qualify as Einstein World Models when they use a world-module to generate visualtemporal rollouts as intermediate reasoning artifacts. p. 8 However, although
an EWM can be incorporated into an agentic system, it is not necessarily an agent in itself. Such
systems may qualify as Einstein World Models
when they use a world-module to generate visualtemporal rollouts as intermediate reasoning artifacts, but not merely by virtue of being agents. Existing agentic systems can call tools, but they do not
by default perform externalised visual thought experiments in the sense proposed here. Conversely,
an Einstein World Model need not have the broader
infrastructure of a general-purpose agent for executing user-facing actions, such as managing software repositories or querying databases. Conversely,
an Einstein World Model need not have the broader
infrastructure of a general-purpose agent for executing user-facing actions, such as managing software repositories or querying databases. It may be
implemented as a language reasoner that calls a
world-module only to imagine a described scene,
inspect the resulting rollout, and incorporate it into
its reasoning trace. This differs from the setting of interest here, where
thought experiments are used to support text-based
reasoning problems and the model must decide,
as part of its own reasoning process, when visualtemporal rollouts are useful. We discuss the relationship between Einstein
World Models and contemporary agentic LLM systems in the Supplementary Notes, Part B.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Final Summary
The central idea is that some questions require a model to visualise how a described scene unfolds, and that this may be poorly supported by language alone. EWMs keep the LLM as the reasoner, but allow it to call a world-module, render a short visual-temporal rollout, and return that rollout to the reasoning trace before answering. A video rollout is not assumed to be a perfect simulation, but rather a visual hypothesis about how a described situation may unfold. p. 7 EWMs
keep the LLM as the reasoner, but allow it to call a
world-module, render a short visual-temporal rollout, and return that rollout to the reasoning trace
before answering. A video rollout is not assumed
to be a perfect simulation, but rather a visual hypothesis about how a described situation may unfold. A video rollout is not assumed
to be a perfect simulation, but rather a visual hypothesis about how a described situation may unfold. Its distinctive usefulness comes from being
externalised, making the model’s reasoning process
available for inspection and debugging. Its distinctive usefulness comes from being
externalised, making the model’s reasoning process
available for inspection and debugging. The path
forward thus requires both better world-module
curation, as well as better datasets to allow for
training LLMs on when and how thought experiments should be invoked.Where the paper says it
Where the paper says it
Where the paper says it
--- Page 1 ---
Einstein World Models
Munachiso Samuel Nwadike1,2
, Zangir Iklassov1
, Ali Mekky1
Zayd M. Kawakibi Zuhri1
, Kentaro Inui1,2,3
1MBZUAI 2RIKEN AIP, Japan 3Tohoku University
munachiso.nwadike@mbzuai.ac.ae
--- Page 2 ---
Abstract
Does intelligence require the ability to reason about phenomena beyond direct experience? It is natural to suspect that some complex thought cannot be captured through language alone. p. 0 ae
Abstract
Does intelligence require the ability to reason about phenomena beyond direct experience? It is natural to suspect that some complex thought cannot be captured through language alone. It is natural to suspect that some complex thought cannot be captured through language alone. However, of particular concern to
this work, is whether visualising counterfactual
events can complement language as a mechanism for complex thought. However, of particular concern to
this work, is whether visualising counterfactual
events can complement language as a mechanism for complex thought. We ask whether
LLMs can be trained to utilise such visualisation mechanisms, in a way that benefits their
reasoning abilities. We ask whether
LLMs can be trained to utilise such visualisation mechanisms, in a way that benefits their
reasoning abilities. Motivated by this question,
we propose Einstein World Models. Motivated by this question,
we propose Einstein World Models. EWMs
are a blueprint for LLM-based reasoning systems that place visual-temporal rollouts inside
the reasoning trace, allowing them to reason in
ways that text alone may not support well. EWMs
are a blueprint for LLM-based reasoning systems that place visual-temporal rollouts inside
the reasoning trace, allowing them to reason in
ways that text alone may not support well. In an
EWM, the LLM calls a world-module (not to be
confused with a world model), to produce short
rollouts of scenes under consideration. In an
EWM, the LLM calls a world-module (not to be
confused with a world model), to produce short
rollouts of scenes under consideration. The returned rollout is treated not as the answer, but
as an inspectable hypothesis that can support
later reasoning. The returned rollout is treated not as the answer, but
as an inspectable hypothesis that can support
later reasoning. Einstein World Models extend
the capability of LLMs for tool calling (such as
web search or code execution), into the domain
of visual thought experiments.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
--- Page 3 ---
Prologue
A call for datasets becomes meaningful after the desirable capability has been specified. This culminates in Section 5. p. 0 Prologue
A call for datasets becomes meaningful after the
desirable capability has been specified. This culminates in Section 5. This culminates in Section 5. The predominance of this
work therefore aims to motivate a learnable format
for visual thought experimentation, proposing the
architecture and training objectives necessitating
their training data.Where the paper says it
Where the paper says it
--- Page 4 ---
1 Introduction
Scientific invention often makes abstract ideas tractable by turning them into thought experiments. In a thought experiment, we visualise a scene, let it unfold, and notice what changes. p. 0 1 Introduction
Scientific invention often makes abstract ideas
tractable by turning them into thought experiments. In a thought experiment, we visualise a scene, let it
unfold, and notice what changes. In a thought experiment, we visualise a scene, let it
unfold, and notice what changes. Einstein’s recollection of special relativity begins with precisely such a thought experiment, later
popularised through the image of Bern’s famous
clock tower. Einstein’s recollection of special relativity begins with precisely such a thought experiment, later
popularised through the image of Bern’s famous
clock tower. In his Autobiographical Notes (Einstein, 1949), he recalls imagining what it would
be like to chase a beam of light. In his Autobiographical Notes (Einstein, 1949), he recalls imagining what it would
be like to chase a beam of light. If he could accelerate until he matched the speed of the beam,
would it eventually appear to hang motionless beFigure 1: Einstein World Models (proposed) build upon traditional LLM reasoning traces. In those notes, Einstein writes that such
a stationary light wave seemed impossible both
empirically and according to Maxwell’s equations. His simple thought experiment had transformed
an abstract tension between electrodynamics and
intuitions about motion, into a concrete scene upon
which reasoning could be built. In those notes, Einstein writes that such
a stationary light wave seemed impossible both
empirically and according to Maxwell’s equations. His simple thought experiment had transformed
an abstract tension between electrodynamics and
intuitions about motion, into a concrete scene upon
which reasoning could be built.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
--- Page 5 ---
2 Einstein World Models
2.1 Overview
The objectives of Einstein World Models are twofold. First, we wish to imbue language models with the ability to construct visualisations when answering questions that require physical intuition or scene-level visualisation. p. 1 1 Overview
The objectives of Einstein World Models are
twofold. First, we wish to imbue language models
with the ability to construct visualisations when
answering questions that require physical intuition
or scene-level visualisation. First, we wish to imbue language models
with the ability to construct visualisations when
answering questions that require physical intuition
or scene-level visualisation. Second, we wish to obtain a window into those visualisations. Second, we wish to obtain a window into those visualisations. An EWM
rollout, as illustrated in Figure 1, is in effect, a visible hypothesis about how a described scene might
unfold.Where the paper says it
Where the paper says it
Where the paper says it
--- Page 6 ---
Figure 1: Einstein World Models (proposed) build upon traditional LLM reasoning traces. However, in addition to generating tokens across N autoregressive steps, the model may, at a sparse set of M intermediate steps, invoke a callable world-module. p. 0 If he could accelerate until he matched the speed of the beam,
would it eventually appear to hang motionless beFigure 1: Einstein World Models (proposed) build upon traditional LLM reasoning traces. However, in addition to
generating tokens across N autoregressive steps, the model may, at a sparse set of M intermediate steps, invoke a
callable world-module. However, in addition to
generating tokens across N autoregressive steps, the model may, at a sparse set of M intermediate steps, invoke a
callable world-module. The returned visual-temporal rollout becomes part of the trace as an inspectable hypothesis.Where the paper says it
Where the paper says it
--- Page 7 ---
Figure 2: If Einstein were travelling away from the Bern clock tower at the speed of light, would the clock’s hands appear frozen in time? This figure compares how different reasoning paradigms would support such a thought experiment. p. 2 ”
EWM
Thoughts
Answer
Question
Question
Thoughts
Answer
Answer
Answer
Question
Figure 2: If Einstein were travelling away from the Bern clock tower at the speed of light, would the clock’s
hands appear frozen in time? This figure compares how different reasoning paradigms would support such a
thought experiment. Bottom left: In VL-JEPA-style world models, predictive visual
representations are learned from observed visual inputs, but the visual prediction itself is not selectively invoked as
an intermediate reasoning artifact. Bottom right: In Einstein World Models, by contrast, the LLM (the reasoner)
preserves autoregressive text generation while taking responsibility for deciding when to invoke a world-module
M times, how to query it, and how to incorporate each returned rollout into subsequent reasoning.Where the paper says it
Where the paper says it
--- Page 8 ---
2.3 Inference
At inference time, an Einstein World Model is specified by two core components πθ and W. Here, πθ denotes the Einstein reasoner, a trainable LLM policy parameterised by θ, and W denotes the world-module. p. 3 3 Inference
At inference time, an Einstein World Model is specified by two core components
πθ and W. Here, πθ denotes the Einstein reasoner, a trainable
LLM policy parameterised by θ, and W denotes
the world-module. πθ generates a reasoning trace
and may query W for visual-temporal rollouts. If
queried, W generates these video rollouts, and returns them to πθ for further autoregression.Where the paper says it
Where the paper says it
--- Page 9 ---
2.4 Training
At the initial stage, the reasoner undergoes supervised fine-tuning on EWM trace formats with standard next-token cross-entropy, masking returned rollout observations from the loss as detailed in the Supplementary Notes, part A. We then recommend following SFT with RLVR-style training over complete EWM trajectories, since target tasks provide verifiable final answers even when intermediate visual thought experiments remain unlabeled. p. 4 Consistent with standard practice in recent
reasoning-model training (Guo et al. , 2025), we
then recommend following SFT with RLVR-style
training over complete EWM trajectories, since
target tasks provide verifiable final answers even
when intermediate visual thought experiments remain unlabeled. Define L
g
i =
PLi
t=1 1it, and, for generated tokens,
ρit =
πθ(zit | τi,<t)
πold(zit | τi,<t)
. Writing clipϵ
(r) = clip(r, 1 − ϵ, 1 + ϵ), the EWM
training objective is
JE(θ) = Ex, τ1:G
"
1
G
X
G
i=1
1
L
g
i
X
Li
t=1
1it
min(ρitAi
, clipϵ
(ρit)Ai)
− βDKL(πθ∥πref)
#
. Define L
g
i =
PLi
t=1 1it, and, for generated tokens,
ρit =
πθ(zit | τi,<t)
πold(zit | τi,<t)
. Writing clipϵ
(r) = clip(r, 1 − ϵ, 1 + ϵ), the EWM
training objective is
JE(θ) = Ex, τ1:G
"
1
G
X
G
i=1
1
L
g
i
X
Li
t=1
1it
min(ρitAi
, clipϵ
(ρit)Ai)
− βDKL(πθ∥πref)
#
.Where the paper says it
Where the paper says it
Where the paper says it
--- Page 9 ---
A GRPO-style implementation (Shao et al., 2024) can optimise this reward with a clipped surrogate. p. 4 A GRPO-style implementation (Shao et al. ,
2024) can optimise this reward with a clipped surrogate.Where the paper says it
--- Page 9 ---
Algorithm 1 Proposed EWM RLVR protocol
Require: Dataset D, reasoner πθ, reference policy πref, world-module W, group size G 1: for each GRPO update round do 2: Set frozen old policy πold ← πθ 3: Sample training batch from D 4: for each (x, y⋆) in the batch do 5: Sample G trajectories τ1:G using πold with access to W 6: for each trajectory τi do 7: Generate,, or segments 8: if the world-module is invoked then 9: Call W with the query in 10: Append rollout as a segment 11: end if
--- Page 9 ---
3 World-Module Selection
Since RLVR optimises the reasoner’s use of the world-module rather than the world-module itself, the question becomes how to select a module that produces useful visual thought experiments. World-modules may come in several forms: Renderers, Simulators, and Planners. p. 5 3
19: end for
3 World-Module Selection
Since RLVR optimises the reasoner’s use of the
world-module rather than the world-module itself,
the question becomes how to select a module that
produces useful visual-temporal thought experiments. 1 Architecture
World-modules may come in several forms. A generated
video rollout may therefore only be as useful as
the physical intuitions sustained by its underlying
video model. Diffusion models remain strong candidates because their intuitive-physics quality can be measured through human-verifiable likelihood estimates derived from the denoising objective of the
diffusion model.Where the paper says it
Where the paper says it
--- Page 9 ---
World-Module Quality
Einstein’s visual thought experiments were disciplined by strong physical intuition, meaning a generated video rollout may only be as useful as the physical intuitions sustained by its underlying video model. Einstein World Models also require rollout faithfulness, exposing information that the reasoner actually uses and that the final answer … p. 5 Einstein World Models also require rollout faithfulness. A faithful rollout exposes information that
the reasoner actually uses and that the final answerWhere the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Externalized reasoning trace components allow LLMs to reason in ways text alone may not support well by invoking a callable world-module to produce short rollouts of scenes under consideration.
-
The system can be trained to utilize visual-temporal thought experiment traces by employing a GRPO-style training objective that optimizes an EWM reward combining final-answer correctness with an optional additional reward term for world-module usage.
-
The improved AI system can perform selective world-module use, as the RLVR protocol converts tracelevel rewards into group relative advantages, so
Traces where a visual thought experiment proves useful enough to justify its world-module usage receive higher advantages.
-
The AI system can distinguish between different types of world-modules, such as Renderers and Simulators, allowing it to choose modules whose inductive biases are usefully different for ensembling and comparison of visual hypotheses.
-
The system can incorporate diverse visual hypotheses by employing ensembling where
different LLM reasoners, attached to different world-modules, exchange rollouts and critiques before answering.
Abstract
Does intelligence require the ability to reason about phenomena beyond direct experience? It is natural to suspect that some complex thought cannot be captured through language alone. However, of particular concern to this work, is whether visualising counterfactual events can complement language as a mechanism for complex thought. We ask whether LLMs can be trained to utilise such visualisation mechanisms, in a way that benefits their reasoning abilities. Motivated by this question, we propose Einstein World Models. EWMs are a blueprint for LLM-based reasoning systems that place visual-temporal rollouts inside the reasoning trace, allowing them to reason in ways that text alone may not support well. In an EWM, the LLM calls a world-module (not to be confused with a world model), to produce short rollouts of scenes under consideration. The returned rollout is treated not as the answer, but as an inspectable hypothesis that can support later reasoning. Einstein World Models extend the capability of LLMs for tool calling (such as web search or code execution), into the domain of visual thought experiments.
Sources
- IntPhys 2: Benchmarking Intuitive Physics Understanding In Complex Synthetic Environments
- Learning When Not to Act: Mitigating Tool Abuse in Agentic Reinforcement Learning
- Thinking with Generated Images
- Intuitive physics understanding emerges from self-supervised pretraining on natural videos
- LTX-Video: Realtime Video Latent Diffusion
- HunyuanVideo: A Systematic Framework For Large Video Generative Models
- Measuring Faithfulness in Chain-of-Thought Reasoning
- LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels
- V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learning
- WebGPT: Browser-assisted question-answering with human feedback
- Causal-JEPA: Learning World Models through Object-Level Latent Masking
- Measuring AI Reasoning: A Guide for Researchers
- IntPhys: A Framework and Benchmark for Visual Intuitive Physics Reasoning
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Agentic Reasoning and Tool Integration for LLMs via Reinforcement Learning
- Thinking with Video: Video Generation as a Promising Multimodal Reasoning Paradigm
- WorldBench: Disambiguating Physics for Diagnostic Evaluation of World Models
- Wan: Open and Advanced Large-Scale Video Generative Models
- Acting Less is Reasoning More! Teaching Model to Act Efficiently
- Mind's Eye of LLMs: Visualization-of-Thought Elicits Spatial Reasoning in Large Language Models
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents