Provably Efficient Offline-to-Online Value Adaptation with General Function Approximation
The authors study how to adapt a value function learned from offline data to an online environment using limited new interactions. They propose O2O-LSVI, an algorithm that achieves better sample complexity than pure online RL under a specific condition called $\beta$-separable value gaps. This means offline pretraining can accelerate online learning if the initial value function has certain structural properties.