Marginalized Operators for Off-policy Reinforcement Learning
The paper introduces marginalized operators as a new class of off-policy evaluation tools that generalize multi-step operators. It defines a marginalized operator $M_w$ using specific stochastic estimates for importance sampling ratios ($w_x$). The core finding is that these marginalized operators are equivalent to multi-step operators under certain conditions, suggesting they offer variance reduction and improved performance in policy evaluation.