Infinitesimal Operator World Models: Finite-Sample Guarantees for Offline Reinforcement Learning
The paper proposes O-CTRL, an operator-based algorithm for offline continuous-time reinforcement learning. It models the environment using an infinitesimal generator learned in a Reproducing Kernel Hilbert Space (RKHS). This approach links RL to the Hamilton-Jacobi-Bellman equation, providing global convergence guarantees and explicit finite-sample bounds tied to system properties like smoothness and stability.