Hierarchical Multilevel Monte Carlo for Order-Optimal Neural Actor-Critic in Average-Reward CMDPs

Name: Hierarchical Multilevel Monte Carlo for Order-Optimal Neural Actor-Critic in Average-Reward CMDPs
Uploaded: 2026-10-20T16:23:51.299637+00:00
Duration: 979 s
Description: This work introduces a hierarchical Multilevel Monte Carlo neural critic to solve a bias-cost tradeoff in primal-dual actor-critic methods for average-reward constrained MDPs (CMDPs). It achieves first-order optimal convergence guarantees of O(T^-1/2) while eliminating the need to know the underlying mixing time, making it practical for large reinforcement learning problems.

This work introduces a hierarchical Multilevel Monte Carlo neural critic to solve a bias-cost tradeoff in primal-dual actor-critic methods for average-reward constrained MDPs (CMDPs). It achieves first-order optimal convergence guarantees of O(T^-1/2) while eliminating the need to know the underlying mixing time, making it practical for large reinforcement learning problems.

Open the full episode