Regularized Latent Dynamics Prediction is a Strong Baseline For Behavioral Foundation Models
cs.AI, cs.LG, cs.RO
Submitted: 2026-03-16
Updated: 2026-08-25
Comments: ICLR 2026 Update 08/25/2026: (i) Fixed a typo in eq. 7. (ii) Updated lemma 1 to a stronger bound for RLDP
License: http://creativecommons.org/licenses/by/4.0/
The gist: Behavioral Foundation Models (BFMs) produce agents with the capability to adapt to any unknown reward or task.
Terminology
Abstract
Behavioral Foundation Models (BFMs) produce agents with the capability to adapt to any unknown reward or task. These methods, however, are only able to produce near-optimal policies for the reward functions that are in the span of some pre-existing state features, making the choice of state features crucial to the expressivity of the BFM. As a result, BFMs are trained using a variety of complex objectives and require sufficient dataset coverage, to train task-useful spanning features. In this work, we examine the question: are these complex representation learning objectives necessary for zero-shot RL? Specifically, we revisit the objective of self-supervised next-state prediction in latent space for state feature learning, but observe that such an objective alone is prone to increasing state-feature similarity, and subsequently reducing span. We propose an approach, Regularized Latent Dynamics Prediction (RLDP), that adds a simple orthogonality regularization to maintain feature diversity and can match or surpass state-of-the-art complex representation learning methods for zero-shot RL. Furthermore, we empirically show that prior approaches perform poorly in low-coverage scenarios where RLDP still succeeds.
Sources
- Variational Option Discovery Algorithms
- Proto Successor Measure: Representing the Behavior Space of an RL Agent
- Can We Gain More from Orthogonality Regularizations in Training Deep CNNs?
- Successor Features for Transfer in Reinforcement Learning
- Learning Successor States and Goal-Dependent Values: A Mathematical Viewpoint
- OpenAI Gym
- Exploration by Random Network Distillation
- Diversity is All You Need: Learning Skills without a Reward Function
- The Information Geometry of Unsupervised Reinforcement Learning
- Diagnosing Bottlenecks in Deep Q-learning Algorithms
- D4RL: Datasets for Deep Data-Driven Reinforcement Learning
- A Minimalist Approach to Offline Reinforcement Learning
- Off-Policy Deep Reinforcement Learning without Exploration
- Towards General-Purpose Model-Free Reinforcement Learning
- Ego4D: Around the World in 3,000 Hours of Egocentric Video
- Bootstrap your own latent: A new approach to self-supervised Learning
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- Dream to Control: Learning Behaviors by Latent Imagination
- Temporal Difference Learning for Model Predictive Control
- Preventing Dimensional Collapse in Self-Supervised Learning via Orthogonality Regularization
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems