Exploiting Exogenous Structure for Sample-Efficient Reinforcement Learning
The paper introduces Exo-MDPs, a structured class of reinforcement learning problems with exogenous states that evolve independently of actions. The authors show that these problems are equivalent to simpler linear mixture MDPs, reducing the complexity from the total state space size to an effective dimension $r$. This structure allows for data-efficient learning, proving that observing exogenous states significantly improves performance bounds.