Fitted Q-Iteration with Continuous Actions and Adaptive Data: Smoothed Coverage and Minimax Rates
This work provides a unified theoretical framework for Fitted Q-Iteration (FQI) when actions are continuous and data is adaptive. It bridges measure theory with Bellman contraction to establish finite-sample performance bounds. The analysis uses sequential Rademacher complexity to show that greedy policies achieve specific rates, while distribution mismatch is quantified by a concentrability coefficient.