Calibration-risk routing for controlled world-model adaptation
cs.AI
Submitted: 2026-10-01
Updated: 2026-10-01
License: http://creativecommons.org/licenses/by/4.0/
The gist: Model-based reinforcement learning (MBRL) can exploit simulated experience, but a simulator-to-target shift creates a model-selection problem: correcting the simulator and fitting the target directly
Terminology
Abstract
Model-based reinforcement learning (MBRL) can exploit simulated experience, but a simulator-to-target shift creates a model-selection problem: correcting the simulator and fitting the target directly can each fail under limited target data. We introduce the Model-Corrected World Model (MC-WM), which separates initial target data into disjoint fit, selection, and calibration partitions and deploys the family with lower standardized calibration risk. A learned confidence signal and deterministic validity predicates weight one-step imagined policy updates without rewriting physical rewards. We evaluate 540 unique reported run cells across three controlled Multi-Joint dynamics with Contact (MuJoCo) shifts; one exact-routing cell was repeated after a pre-deployment artifact gate, giving 541 completed executions.
Sources
- Constrained Policy Optimization
- Inverse Constrained Reinforcement Learning
- Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
- Randomized Ensembled Double Q-Learning: Learning Fast Without a Model
- Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models
- Off-Dynamics Reinforcement Learning: Training for Transfer with Domain Classifiers
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- Dream to Control: Learning Behaviors by Latent Imagination
- Temporal Difference Learning for Model Predictive Control
- When to Trust Your Model: Model-Based Policy Optimization
- RMA: Rapid Motor Adaptation for Legged Robots
- Adapting World Models with Latent-State Dynamics Residuals
- H2O+: An Improved Framework for Hybrid Offline-and-Online RL with Dynamics Gaps
- Sim-to-Real Transfer of Robotic Control with Dynamics Randomization
- Agnostic System Identification for Model-Based Reinforcement Learning
- Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems