Discovering What You Can Control: Interventional Boundary Discovery for Reinforcement Learning
cs.LG, cs.AI
Submitted: 2026-03-18
Updated: 2026-09-27
License: http://creativecommons.org/licenses/by/4.0/
The gist: When an RL agent's observations contain distractors driven by the same confounders as its true state, observational data alone cannot identify which dimensions the agent controls.
Terminology
Abstract
When an RL agent's observations contain distractors driven by the same confounders as its true state, observational data alone cannot identify which dimensions the agent controls. In our benchmarks, even state-conditioned observational selectors can collapse when distractors mimic controllable state variables. We propose Interventional Boundary Discovery (IBD), which treats the agent's own action channel as a source of randomized interventions: randomizing actions implements an interventional contrast, and per-dimension two-sample tests with FDR correction produce a binary mask over observation dimensions. Across 12 continuous-control settings with up to 100 distractors, IBD matches oracle return in 11 of 12 settings, while observational baselines including mutual information, state-conditioned forward models, and gradient-based sensitivity often underperform simply passing the full observation to SAC. Code is available at https://github.com/jiaxin26/IBD-RL
Sources
- A Forget-and-Grow Strategy for Deep Reinforcement Learning Scaling in Continuous Control
- WoCoCo: Learning Whole-Body Humanoid Control with Sequential Contacts
- DeepMind Control Suite
- Zero-Shot Reinforcement Learning Under Partial Observability
- PIGDreamer: Privileged Information Guided World Models for Safe Partially Observable Reinforcement Learning
- The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixels
- Provable RL with Exogenous Distractors via Multistep Inverse Dynamics
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks