Learning to Plan from Random Exploration
cs.LG, cs.RO, stat.ML
Submitted: 2026-09-29
Updated: 2026-09-29
License: http://creativecommons.org/licenses/by/4.0/
The gist: Random exploration reveals how an environment can be traversed before a goal is specified.
Terminology
Abstract
Random exploration reveals how an environment can be traversed before a goal is specified. Can this experience support long-range planning without policy-improvement training? Our random-walk analysis explains what temporal relations contain: short horizons reveal geodesic geometry in the diffusion limit, while longer horizons reveal connectivity between regions before mixing removes these distinctions. We learn these relations with a conditional energy-based model that estimates temporal log-density ratios through horizon-conditioned embeddings. The model is trained on observation pairs by noise-contrastive estimation, without action or reward labels. The planner queries these learned relations at different horizons as it moves toward the goal. At test time, a separate local dynamics model predicts candidate action outcomes, and the temporal model evaluates their progress toward the goal by selecting or aggregating estimated improvements across horizons. The agent executes one action and replans with both models fixed. Experiments demonstrate long-range maze planning from random exploration using states and images. Learned score fields, embedding probes, and planned routes exhibit properties of a multiscale cognitive map. We further demonstrate egocentric navigation from random exploration and manipulation planning from suboptimal data.
Sources
- Is Conditional Generative Modeling all you need for Decision-Making?
- On Evaluation of Embodied Navigation Agents
- V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning
- Mind the Gap: Promises and Pitfalls of Hierarchical Planning in LeWorldModel
- Fast LeWorldModel
- Flow-JEPA: Robust Latent Dynamics for JEPA World Models via Flow Matching
- Toward Physically Grounded JEPA World Models for Goal-Conditioned Robotic Planning
- LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels
- Latent Energy Action Planning with World Models
- Generative Actor Critic
- Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models
- Varadhan Asymptotics for the Heat Kernel on Finite Graphs
- Representation Learning with Contrastive Predictive Coding
- QQWorld: Quantile-Quantile Matching for World Model Regularization
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks