Fast Rates for Offline Contextual Bandits with Forward-KL Regularization under Single-Policy Concentrability
cs.LG, cs.AI, cs.IT, math.IT, math.ST, stat.ML, stat.TH
Submitted: 2026-05-09
Updated: 2026-11-09
Terminology
Sources
- Pessimistic Nonlinear Least-Squares Value Iteration for Offline Reinforcement Learning
- Is a Good Foundation Necessary for Efficient Reinforcement Learning? The Computational Role of the Base Model in Exploration
- Foundations of Reinforcement Learning and Interactive Decision Making
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- KL-Regularized Reinforcement Learning is Designed to Mode Collapse
- Correcting the Mythos of KL-Regularization: Direct Alignment without Overoptimization via Chi-Squared Preference Optimization
- Language Model Decoding as Direct Metrics Optimization
- Near-Optimal Regret for KL-Regularized Multi-Armed Bandits
- Coverage Improvement and Fast Convergence of On-policy Preference Learning
- Provably Efficient Regularized Online RLHF with Generalized Bilinear Preferences
- Achieving Logarithmic Regret in KL-Regularized Zero-Sum Markov Games
- Trust Region Policy Optimization
- A Comedy of Estimators: On KL Regularization in RL Training of LLMs
- On a few pitfalls in KL divergence gradient estimation for RL
- Beyond Reverse KL: Generalizing Direct Preference Optimization with Diverse Divergence Constraints
- Greedy Sampling Is Provably Efficient for RLHF
- Exploratory Preference Optimization: Harnessing Implicit Q*-Approximation for Sample-Efficient RLHF
- Nearly Minimax Optimal Offline Reinforcement Learning with Linear Function Approximation: Single-Agent MDP and Markov Game
- EMA Policy Gradient: Taming Reinforcement Learning for LLMs with EMA Anchor and Top-k KL
- Beyond Pessimism: Offline Learning in KL-regularized Games
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis