Towards Unbiased On-Policy Distillation for Block Diffusion Language Models
cs.CL
Submitted: 2026-10-04
Updated: 2026-10-15
Terminology
Sources
- GPT-4 Technical Report
- Program Synthesis with Large Language Models
- LLaDA2.0: Scaling Up Diffusion Language Models to 100B
- LLaDA2.1: Speeding Up Text Diffusion via Token Editing
- Evaluating Large Language Models Trained on Code
- Training Verifiers to Solve Math Word Problems
- dOPSD: On-Policy Self-Distillation for Diffusion Language Models
- Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes
- Soft Adaptive Policy Optimization
- Co-Evolving Policy Distillation
- Measuring Mathematical Problem Solving With the MATH Dataset
- Uni-OPD: Unifying On-Policy Distillation with a Dual-Perspective Recipe
- Qwen2.5-Coder Technical Report
- Entropy-Aware On-Policy Distillation of Language Models
- RePO: Replay-Enhanced Policy Optimization
- Self-Distilled Agentic Reinforcement Learning
- RLVR without Ineffective Samples: Group Prioritized Off-Policy Optimization for LLM Reasoning
- The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Models
- Large Language Diffusion Models
- RLCSD: Reinforcement Learning with Contrastive On-Policy Self-Distillation
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering