Distribution Matching Distillation for Continuous Diffusion Language Models
The study addresses high sampling costs in continuous diffusion language models by using distributional distillation. It proposes two methods, Simplex-DMD and Reinforce-DMD, to match a student model's token distributions to target data distributions. This allows for high-quality generation using significantly fewer network evaluations than standard diffusion baselines.