RFG: Self-Improving Diffusion Large Language Models with Reward-Free Guidance
Reward-free guidance (RFG) is a method to improve large language models during diffusion sampling without needing an explicit reward signal. It works by parameterizing a process reward using log-likelihood ratios between enhanced and reference models. This allows the model to be guided toward better reasoning and code generation, achieving significant performance gains across various benchmarks through test-time guidance.