Inferring Transferable Rewards via Active Inverse Reinforcement Learning
cs.LG, cs.HC
Submitted: 2026-11-02
Updated: 2026-11-02
License: http://creativecommons.org/licenses/by/4.0/
The gist: Inverse Reinforcement Learning (IRL) aims to recover expert rewards that transfer across environments with differing dynamics.
Terminology
Abstract
Inverse Reinforcement Learning (IRL) aims to recover expert rewards that transfer across environments with differing dynamics. Prior work has shown that such transfer is possible when gathering expert demonstrations across sufficiently different dynamics. In practice, however, expert demonstrations are costly, making it critical to select the right dynamics to reduce the quantity of expert data required for learning transferable rewards. We propose a two-phase approach: we first cycle through the available dynamics to obtain a reliable reward estimate, then actively select dynamics to reduce our uncertainty about the expert reward. Crucially, this yields novel sample complexity guarantees on transfer that hold for any dynamics, with a rate independent of the number of available dynamics. Empirically, our method significantly reduces the number of expert demonstrations needed to learn a transferable reward.
Sources
- Improved Online Confidence Bounds for Multinomial Logistic Bandits
- PAC Apprenticeship Learning with Bayesian Active Inverse Reinforcement Learning
- Reward Transfer from Inverse Reinforcement Learning: A Coupled Minimax Approach
- Generalizing Behavior via Inverse Reinforcement Learning with Closed-Form Reward Centroids
- Toward Information Theoretic Active Inverse Reinforcement Learning
- Convergence of a model-free entropy-regularized inverse reinforcement learning algorithm
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis