Reinforcement Learning for Diffusion LLMs with Entropy-Guided Step Selection and Stepwise Advantages
cs.LG, cs.AI, cs.CL
Submitted: 2026-03-13
Updated: 2026-11-07
Code: https://github.com/vishnutez/egspo-dllm-rl
Terminology
Sources
- Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models
- Program Synthesis with Large Language Models
- Evaluating Large Language Models Trained on Code
- Training Verifiers to Solve Math Word Problems
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMs
- Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution
- Esoteric Language Models: A Family of Any-Order Diffusion LLMs
- Proximal Policy Optimization Algorithms
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference
- wd1: Weighted Policy Optimization for Reasoning in Diffusion Language Models
- Fine-Tuning of Continuous-Time Diffusion Models as Entropy-Regularized Control
- SPG: Sandwiched Policy Gradient for Masked Diffusion Language Models
- d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimation
- Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Models
- Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding
- MMaDA: Multimodal Large Diffusion Language Models
- Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning
- Dream 7B: Diffusion Large Language Models
Related papers
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis
- (Mis)generalization of Helpful-Only Fine-tuning