Closing the Horizon Gap in Policy Optimization for Adversarial MDPs

Name: Closing the Horizon Gap in Policy Optimization for Adversarial MDPs
Uploaded: 2026-10-14T13:42:20.597304+00:00
Duration: 393 s
Description: Policy optimization updates actions locally at each state in online learning for adversarial MDPs, but its regret bounds are worse by a factor of the horizon H compared to global optimization methods. The authors use regularized Q-functions to control local update stability jointly across states, achieving better high-probability regret bounds for both known and unknown transitions.

Policy optimization updates actions locally at each state in online learning for adversarial MDPs, but its regret bounds are worse by a factor of the horizon H compared to global optimization methods. The authors use regularized Q-functions to control local update stability jointly across states, achieving better high-probability regret bounds for both known and unknown transitions.

Open the full episode