Reinformed Dreamer: An Asymmetric World Model Efficiently Trained through Latent Guidance
Gaspard Lambrechts, Adrien Bolland, Daniel Ebi, Damien Ernst
cs.LG, stat.ML
Submitted: 2026-07-28
Comments: 8 pages, 18 pages total, 3 figures
License: http://creativecommons.org/licenses/by-sa/4.0/
The gist: Much like humans benefit from guidance while learning, reinforcement learning algorithms may benefit from additional supervision beyond rewards.
Terminology
Abstract
Much like humans benefit from guidance while learning, reinforcement learning algorithms may benefit from additional supervision beyond rewards. Leveraging additional information during training to learn better representations and behaviors has been the focus of asymmetric reinforcement learning. This learning paradigm has proven effective under partial observability when additional state information is available, but also under full observability when more refined state information is available. Focusing on model-based reinforcement learning, we study the effect of asymmetric learning on observation representations and on privileged information representations. First, we identify a limitation in the privileged information representations learned by an asymmetric model-based algorithm known as the Informed Dreamer. Then, we propose a novel asymmetric representation learning objective using latent guidance, resulting in a new algorithm called the Reinformed Dreamer. Experiments across several benchmarks show a more consistent improvement over Dreamer than previous asymmetric approaches.
Sources
- An Introduction to Centralized Training for Decentralized Execution in Cooperative Multi-Agent Reinforcement Learning
- Learning and Querying Fast Generative Models for Reinforcement Learning
- Neural Predictive Belief Representations
- Memory-based control with recurrent neural networks
- DeepMind Control Suite
- R-MADDPG for Partially Observable Environments and Limited Communication
- On Improving Deep Reinforcement Learning for POMDPs
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks