On the Design Space of Continuous Diffusion Language Models

Name: On the Design Space of Continuous Diffusion Language Models
Uploaded: 2026-10-18T14:33:29.476427+00:00
Duration: 216 s
Description: Self-conditioning improves training by making highly likely inference regions more dense, reducing errors between training and testing data. DoubleCat is a simple two-level method that recovers much of this benefit by mimicking self-conditioning with no extra model computation. This leads to significant accuracy gains, reaching discrete diffusion performance.

Self-conditioning improves training by making highly likely inference regions more dense, reducing errors between training and testing data. DoubleCat is a simple two-level method that recovers much of this benefit by mimicking self-conditioning with no extra model computation. This leads to significant accuracy gains, reaching discrete diffusion performance.

Open the full episode