Anchors Enhance Controllability and Reasoning of Diffusion Language Models
AnchorDLM is a framework that trains an anchor generator to create short output fragments and their positions for each query. These anchors act as control tokens placed in a frozen Diffusion Language Model to guide its denoising process. Training involves supervised fine-tuning and reinforcement learning using downstream task outcomes, showing significant improvements in controllability.