Steering Without Breaking: Mechanistically Informed Interventions for Discrete Diffusion Language Models
The research developed an adaptive scheduling method to control text generation in Discrete Diffusion Language Models (DLMs). By using mechanistic insights from Sparse Autoencoders, the method targets specific points in the denoising process where attributes emerge. This allows for precise steering of semantic features without degrading overall text quality, achieving high-fidelity control over complex multi-attribute tasks.