A Continuous-Time Reinforcement Learning Framework for Fine-Tuning Discrete Diffusion Models
cs.LG
Submitted: 2026-07-16
Updated: 2026-09-26
Comments: 37 pages, 5 figures
Code: https://github.com/dllm-reasoning/d1
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation
- Training Verifiers to Solve Math Word Problems
- Diffusion on the Probability Simplex
- Discrete Feynman-Kac Correctors
- WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inference
- Improved Large Language Diffusion Models
- Diffusion-State Policy Optimization for Masked Diffusion Language Models
- Categorical SDEs with Simplex Diffusion
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Regret of exploratory policy improvement and q-learning
- Process Advantage Signal Shaping: Advantage Construction for Process-Supervised Reinforcement Learning in LLM Reasoners
- LFPO: Likelihood-Free Policy Optimization for Masked Diffusion Models
- Dream 7B: Diffusion Large Language Models
- dTRPO: Trajectory Reduction in Policy Optimization of Diffusion Large Language Models
- Scores as Actions: a framework of fine-tuning diffusion models by continuous-time reinforcement learning
- DiFFPO: Training Diffusion LLMs to Reason Fast and Furious via Reinforcement Learning
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks