Taylor Expansion of Discount Factors
This work addresses discrepancies between discount factors used for value estimation and those used for objective definition in reinforcement learning. It introduces a family of objectives that interpolate value functions across two different discount factors, $\gamma$ and $\gamma_0$. This allows for new methods to estimate value functions and perform policy optimization updates by relating the discounted objective to an undiscounted one.