Flying on Imagination: DreamerV3-Style World Models under Environment Distribution Shift for Quadrotor Navigation

arXiv:2606.05015 · cs.RO · Submitted 2026-06-03 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "Flying on Imagination".

Dev: Here's the gist of the paper. World models, learned generative models that predict how an environment evolves, have become a promising tool for sample-efficient robot learning The results show that world model robustness during SSL pretraining is a strong predictor of sim-toreal transfer Pruning redundant reasoning tokens This systematic study investigates how DreamerV3-based world models generalize under environmental variability for vision-based quadrotor navigation The authors conduct a comprehensive robustness analysis regarding how DreamerV3-based world model generalizes and responds to environment variability The investigation proceeds in four stages: SSL pretraining, hyperparameter analysis, RL fine-tuning, and real-world deployment operating purely within world model imagination How the training signal is built The method comprises three stages: (a) data collection under varying levels of environmental randomness, (b) SSL pretraining of DreamerV3 world models with hyperparameter optimization, and (c) RL fine-tuning of the best world model per randomness level Data collection involves four levels of environmental randomness, defined as L1 through L4, where L1 fixes obstacles entirely, L2 introduces categorical diversity, L3 introduces spatial diversity with controlled spread using Sobol sampling, and L4 is the maximally stochastic setting The SSL pretraining phase minimizes a joint objective: Lϕ = Lrec + Lrew + β LKL where the terms reconstruct depth and state, predict reward, and regularize the stochastic state Where the gains come from The analysis of self-supervised pretraining reveals that world model quality is governed far more by architectural and training choices than by the environment it is trained in The discrete latent size ddisc and batch length Lbatch are consistently the most influential parameters, with Lbatch contributing a stable ∆ ≈ 35% across environments, while ddisc is both the strongest and most variable World models trained in more structured environments have better reconstruction in the imagination phase Real-world validation and conclusions

Paper summary: Rosa: So, to summarize this paper, they are investigating how world models, which are essentially learned generative models that predict how an environment evolves, handle changes in the environment. Their main thesis is that while these models look promising for sample-efficient robot learning, we don't fully understand their robustness when the environment shifts.

Dev: They address this by using vision-based quadrotor navigation as their testbed problem. They conduct a systematic study training DreamerV3-based world models under varying levels of environmental randomness and then evaluate them across all those levels through both Self-Supervised Learning pretraining and Reinforcement Learning fine-tuning.

Taro: So, the whole setup involves systematically changing the environment to see how well the model holds up when it has to navigate in an unseen setting. It’s about testing generalization under distribution shift.

Rosa: Right, and what they claim is that world model robustness during SSL pretraining is a strong predictor of sim-to-real transfer performance. They set up four levels of environmental randomness to test this hypothesis thoroughly across the whole pipeline, from initial learning to final fine-tuning.

Dev: What matters is how they structured the training signal and the evaluation across these varied settings. They focused on seeing how these models generalize when they move from a controlled training setup to a real quadrotor operating in unseen environments.

Taro: I'm interested in what that "real quadrotor" deployment looks like, especially when it's operating purely within the world model imagination phase after receiving some real sensory input.

Rosa: They deploy all these world models and their associated navigation policies on a physical quadrotor in environments they haven't seen before. They even include an open-loop run where the model only gets twenty point five seconds of real sensory input before everything is cut off, leaving the system to navigate entirely in imagination over a twelve meter traverse.

Dev: So, the test isn't just about seeing if it can navigate; it’s about testing its ability to function when the world misbehaves outside of simulation or training data. That real-world deployment is key to their conclusion on transferability.

Taro: It sounds like they are pushing the limits by making sure the system has to rely entirely on its predictive capabilities in a real, unpredictable setting rather than just following a pre-programmed path.

Rosa: Exactly. The paper shows that the models that were most robust during SSL pretraining tended to perform better when deployed in those challenging, unseen conditions they set up for testing. That's the core finding connecting pretraining stability to real-world performance.

Conclusion: Dev: Looking at the full scope, this paper, "Flying on Imagination: DreamerV3-Style World Models under Environment Distribution Shift for Quadrotor Navigation," is really about moving beyond just training models in one neat environment. It’s about testing how those learned generative models actually work when the real world throws curveballs.

Rosa: And the authors are looking at vision-based quadrotor navigation, using DreamerV3 as their world model foundation. The implication is that if you can train your world model to be robust during SSL pretraining, you have a much better shot at getting a policy that transfers well to real-world navigation tasks.

Taro: So, for someone who only listens to this show, what does this mean practically? It means we should stop treating environment distribution shift as something we just patch later in the training process.

Dev: Exactly. This suggests that the foundational work in self-supervised pretraining needs to be more focused on building models that inherently capture transferable structures about how environments behave, rather than just optimizing for a single, fixed test case.

Rosa: It implies that architectural choices, like the discrete latent size and batch length we discussed earlier, are not just technical details. They are fundamental decisions about what kind of predictive representation the model is capable of learning across different conditions.

Taro: If we focus on those foundational training signals, maybe we can build systems that handle real-world misbehavior more gracefully when they get deployed and start operating autonomously.

Dev: It’s a call to focus on making the world model itself a more general computational primitive for embodied intelligence, not just a specific solution for one specific navigation problem in one perfect setting. That's what this paper is pointing toward.

Luca Zanatta, Grzegorz Malczyk, Kostas Alexis

Norwegian University of Science and Technology

cs.RO

Submitted: 2026-06-03

Updated: 2026-11-15

Code: https://github.com/ntnu-arl/world-model-nav-generalization

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 88/100

The gist: World models, learned generative models that predict how an environment evolves, have become a promising tool for sample-efficient robot learning The results show that world model robustness during…

Key concepts

World Model
A learned generative model that predicts how an environment will evolve based on current observations. It allows a robot to simulate future states or imagine different scenarios before acting, making learning more sample-efficient.
SSL Pretraining
Self-supervised learning used to train the world model. This involves minimizing a joint objective that includes reconstructing depth, predicting rewards, and regularizing the stochastic state of the environment during training.
Environmental Randomness Levels (L1-L4)
Four distinct levels of environmental variability were tested: L1 fixes obstacles entirely, L2 introduces categorical diversity, L3 uses spatial diversity via Sobol sampling, and L4 is maximally stochastic. This tests how robust the world model is to different degrees of unpredictability.
Latent Size (ddisc) and Batch Length (Lbatch)
These are key architectural hyperparameters influencing world model performance. The discrete latent size (ddisc) is found to be the strongest factor, while batch length (Lbatch) provides a stable performance boost across different environments.

Terminology

Summary

World models, learned generative models that predict how an environment evolves, have become a promising tool for sample-efficient robot learning The results show that world model robustness during SSL pretraining is a strong predictor of sim-toreal transfer

Pruning redundant reasoning tokens

This systematic study investigates how DreamerV3-based world models generalize under environmental variability for vision-based quadrotor navigation The authors conduct a comprehensive robustness analysis regarding how DreamerV3-based world model generalizes and responds to environment variability The investigation proceeds in four stages: SSL pretraining, hyperparameter analysis, RL fine-tuning, and real-world deployment operating purely within world model imagination

How the training signal is built

The method comprises three stages: (a) data collection under varying levels of environmental randomness, (b) SSL pretraining of DreamerV3 world models with hyperparameter optimization, and (c) RL fine-tuning of the best world model per randomness level Data collection involves four levels of environmental randomness, defined as L1 through L4, where L1 fixes obstacles entirely, L2 introduces categorical diversity, L3 introduces spatial diversity with controlled spread using Sobol sampling, and L4 is the maximally stochastic setting The SSL pretraining phase minimizes a joint objective: Lϕ = Lrec + Lrew + β LKL where the terms reconstruct depth and state, predict reward, and regularize the stochastic state

Where the gains come from

The analysis of self-supervised pretraining reveals that world model quality is governed far more by architectural and training choices than by the environment it is trained in The discrete latent size ddisc and batch length Lbatch are consistently the most influential parameters, with Lbatch contributing a stable ∆ ≈ 35% across environments, while ddisc is both the strongest and most variable World models trained in more structured environments have better reconstruction in the imagination phase

Real-world validation and conclusions

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Robustness assessment of world models during self-supervised learning can directly predict sim-to-real transfer success; every model that generalized well in cross-environment SSL validation deployed successfully in the real world. This allows for proactive selection of world model architectures and training configurations before extensive reinforcement learning fine-tuning.

  2. Hyperparameter tuning for the RSSM architecture identifies dominant factors governing quality, specifically the discrete latent size and the training-sequence length as the dominant factors governing world model quality. Using these findings provides practical guidelines for configuring DreamerV3-based systems to maximize generalization across environmental variability.

  3. Policy robustness is enhanced by using cross-environment validation from the SSL phase during RL fine-tuning, isolating control policy impact: The cross-environment evaluation protocol from the SSL phase is applied identically here, testing each policy across all levels of randomness in both the context and imagination phases. This ensures policies are not overfitted to specific simulation layouts.

  4. Real-world deployment can be executed with high reliability by prioritizing models with strong SSL generalization: "Every model that generalized well across environments during SSL pretraining transferred successfully to the real world, while the model that dominated RL simulation evaluation was not able to find the path to the target." This establishes a more reliable predictor of deployability than RL win rate alone.

  5. World models can enable autonomous navigation in unseen environments through open-loop imagination: The open-loop experiment showed that a world model can fly a real quadrotor through a 12 m corridor on imagination alone, without hallucinating a path into an obstacle. This allows the system to navigate complex, novel scenarios by predicting future states based solely on learned dynamics.

Sources

Related papers