Sharp Convergence and Sample Complexity of Policy Mirror Descent for Average-Reward MDPs
cs.LG, math.OC
Submitted: 2026-10-02
Updated: 2026-10-02
Code: https://github.com/brett-daley/gym-classics
Terminology
Sources
- On the Theory of Policy Gradient Methods: Optimality, Approximation, and Distribution Shift
- Regret Analysis of Policy Gradient Algorithm for Infinite Horizon Average Reward Markov Decision Processes
- Linear Convergence of Entropy-Regularized Natural Policy Gradient with Linear Function Approximation
- Fast Global Convergence of Natural Policy Gradient Methods with Entropy Regularization
- A Sharper Global Convergence Analysis for Average Reward Reinforcement Learning via an Actor-Critic Approach
- Convergence of Natural Policy Gradient for a Family of Infinite-State Queueing MDPs
- Optimal Convergence Rate for Exact Policy Mirror Descent in Discounted Markov Decision Processes
- On the Global Convergence of Policy Gradient in Average Reward Markov Decision Processes
- Policy Mirror Descent for Reinforcement Learning: Linear Convergence, New Sampling Complexity, and Generalized Problem Classes
- Policy Gradient Algorithms in Average-Reward Multichain MDPs
- Stochastic first-order methods for average-reward Markov decision processes
- Performance of NPG in Countable State-Space Average-Cost RL
- Trust Region Policy Optimization
- Proximal Policy Optimization Algorithms
- Optimal Sample Complexity for Average Reward Markov Decision Processes
- Linear Convergence of Natural Policy Gradient Methods with Log-Linear Policies
- Deep Index Policy for Multi-Resource Restless Matching Bandit and Its Application in Multi-Channel Scheduling
- Policy Mirror Descent for Regularized Reinforcement Learning: A Generalized Framework with Linear Convergence
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks