Advanced Policies: A First-Principles Path from Policy Gradient to Q-Learning
cs.LG, stat.ML
Submitted: 2020-05-18
Updated: 2026-09-27
Comments: Substantially revised version. Corrects the earlier monotonic-improvement claim, develops a substantially revised formulation of Advanced Actor-Critic (AAC), and adds expanded theoretical analysis, continuous-action implementations, and substantially expanded experiments
Code: https://github.com/hill-a/stable-baselines
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- OpenAI Gym
- Addressing Function Approximation Error in Actor-Critic Methods
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- Bridging the Gap Between Value and Policy Based Reinforcement Learning
- Equivalence Between Policy Gradients and Soft Q-Learning
- Sample Efficient Actor-Critic with Experience Replay
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks