Commitment Before Realization: When Classifier-Free Guidance Becomes Unnecessary in Masked Diffusion Language Models
Fan Zhou, Weitian Wang, Tim Van de Cruys
cs.CL
Submitted: 2026-08-08
Updated: 2026-08-11
License: http://creativecommons.org/licenses/by/4.0/
The gist: Classifier-free guidance (CFG) is usually kept on throughout masked diffusion language model decoding, although its benefit varies across prompts and over time.
Terminology
Abstract
Classifier-free guidance (CFG) is usually kept on throughout masked diffusion language model decoding, although its benefit varies across prompts and over time. We study when CFG is actually needed by comparing, from any partial output, the probability of eventual constraint satisfaction under continued CFG and under base-only continuation. Their difference defines the remaining value of guidance. Guidance dependence is highly prompt-specific. Many prompts already succeed without CFG, while for others it provides no measurable benefit or can be harmful. For prompts that do benefit, the gain is often concentrated early. We define the commitment horizon as the earliest point from which switching all remaining decoding to the base model reduces final success by no more than a chosen tolerance. Under the base model, the corresponding success probability, or committor, is a martingale. To first order, CFG's per-step effect is governed by the covariance between the guidance logit direction and the successor committor. This gives a local account of when guidance can help, but it does not by itself locate the horizon. Among prompts with an observed preterminal horizon, is usually early and varies more within constraint families than between them. Freezing each prompt at its own cross-fitted horizon is noninferior to full CFG on all 13 subtasks at the prespecified margin, even while many tokens remain masked. This separates commitment from realization. The boundary also identifies a later region in which higher parallelism adds only a small cost in constraint success, although fluency still degrades with parallel width. For failed trajectories, reopening committed positions improves recovery in both failure modes.
Sources
- Theory of Speciation Transitions in Diffusion Models with General Class Structure
- The statistical thermodynamics of generative diffusion models: Phase transitions, symmetry breaking and critical instability
- dParallel: Learnable Parallel Decoding for dLLMs
- CFG++: Manifold-constrained Classifier Free Guidance for Diffusion Models
- DEFT: Efficient Fine-Tuning of Diffusion Models by Learning the Generalised $h$-transform
- A framework for conditional diffusion modelling with applications in motif scaffolding for protein design
- A Fourier Space Perspective on Diffusion Models
- Conditional Diffusion Guidance under Hard Constraint: A Stochastic Analysis Approach
- The Entropic Signature of Class Speciation in Diffusion Models
- What Exactly Does Guidance Do in Masked Discrete Diffusion Models
- Classifier-Free Diffusion Guidance
- Adaptive Classifier-Free Guidance via Dynamic Low-Confidence Masking
- Large Language Diffusion Models
- Unlocking Guidance for Discrete State-Space Diffusion and Flow Models
- Improving Classifier-Free Guidance in Masked Diffusion: Low-Dim Theoretical Insights with High-Dim Impact
- A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Data
- Deferred Commitment Decoding for Diffusion Language Models
- Diffusion Models Generate Images Like Painters: an Analytical Theory of Outline First, Details Later
- Analysis of Classifier-Free Guidance Weight Schedulers
- Estimating Committor Functions via Deep Adaptive Sampling on Rare Transition Paths
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering