Routing Policy Optimization for Online Diffusion Reinforcement Learning
Routing Policy Optimization (RPO) improves online reinforcement learning for diffusion models by turning attention routing into a stochastic exploration action. It achieves this by sampling attention routes from the generator's native mass and scoring them with a Gaussian likelihood around the dense velocity. This method optimizes generation quality while preserving the original deterministic sampler, leading to nearly four times higher training efficiency than similar methods.