Beyond Ensemble Disagreement: Predictive STL Monitoring of RL Locomotion under Distribution Shift
cs.RO, cs.LG
Submitted: 2026-11-01
Updated: 2026-11-01
Comments: Accepted at the Modeling Uncertainty in Robotic World Models Workshop CoRL 2026
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: RL locomotion policies trained under nominal simulation conditions may encounter distribution shifts that compromise safety.
Terminology
Abstract
RL locomotion policies trained under nominal simulation conditions may encounter distribution shifts that compromise safety. Predictive monitors could anticipate violations, but their world-model forecasts may become unreliable under shift, and ensemble disagreement is not calibrated. We evaluate predictive Signal Temporal Logic (STL) monitoring for frozen Ant-v5 and HalfCheetah-v5 policies using closed-loop world-model rollouts and a three-model ensemble. Across external-force, observation-noise, and mass shifts, predicted robustness ranks HalfCheetah's future STL violations strongly, whereas disagreement does not reliably track observation prediction error or violations. Under severe noise, violations remain common (64.2 plus or minus24.4% at K = 20 across three PPO seeds), yet current-state robustness without prediction comes within 0.03 AUROC of predictive robustness. Penalizing predicted robustness by disagreement does not consistently improve ranking. Ant provides complementary per-specification evidence, but its composite target has no positive violations and its any-violation target is single-class in most evaluated seed-condition slices. Overall, predictive robustness provides useful STL-specific ranking but little demonstrated additional ranking value over the current state; these findings caution against treating ensemble disagreement as a reliable indicator of safety uncertainty.
Sources
- Proximal Policy Optimization Algorithms
- Mastering Diverse Domains through World Models
- Conformal Prediction for STL Runtime Verification
- Model Predictive Online Monitoring of Dynamical Systems for Nested Signal Temporal Logic Specifications
Related papers
- Rethinking Actions: Continuity-Aware Priors for World-Model Planning
- MOSAIC: Modular Obstacle and Spatial-Aware Framework for Manipulation In Cluttered Scenes
- SAGAS: Semantic-Aware Graph-Assisted Stitching for Offline Temporal Logic Planning
- DexSeed: Expanding Human Hand-Object Interactions into Robot Demonstrations for Dexterous Policy Learning
- LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
- CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation