Commit Locally, Exit Globally: Coordinating Adaptive Sampling and Early Exit in Diffusion Language Models
cs.CL, cs.AI
Submitted: 2026-07-30
Updated: 2026-10-14
Comments: Code is available at https://github.com/ming053l/C4-dLLM
Code: https://github.com/ming053l/C4-dLLM
License: http://creativecommons.org/licenses/by/4.0/
The gist: Diffusion language models expose a provisional prediction at every denoising step, and on many tasks the candidate answer inside it stabilizes before the step schedule is exhausted.
Terminology
Abstract
Diffusion language models expose a provisional prediction at every denoising step, and on many tasks the candidate answer inside it stabilizes before the step schedule is exhausted. This creates two acceleration opportunities, leaving a block early and stopping the sequence early, but the two require different criteria because block acceleration is local whereas sequence termination is global and freezes the graded answer. Existing methods usually optimize only one axis, and existing exit gates rely on fixed-region confidence or schedule-dependent rules rather than the candidate answer itself. We present C 4, which coordinates the two axes by giving each decision its own gate. C onfidence-Verified Early Exit (CVEE) decides when the sequence may stop, requiring confidence and sustained argmax stability over a candidate span re-extracted at every step. C ommit- C ore-Then- C onfirm (CCTC) decides which token positions a step may commit by borrowing an autoregressive freezing order inside each block: it commits a boundary-anchored core and confirms deferred positions one step later, so the answer block can be accelerated without allowing local commits inside the answer span to determine sequence-level termination. On 12 zero-shot tasks with LLaDA and Dream, one frozen configuration removes 64--95% of decoding steps and delivers measured end-to-end speedups of 2.6 to 8.6 over full decoding. Code is available at https://github.com/ming053l/C4-dLLM.
Sources
- Program Synthesis with Large Language Models
- Evaluating Large Language Models Trained on Code
- DPad: Efficient Diffusion Language Models with Suffix Dropout
- STDec: Spatio-Temporal Stability Guided Decoding for dLLMs
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Training Verifiers to Solve Math Word Problems
- Beyond Autoregression: Fast LLMs via Self-Distillation Through Time
- $R^2$-dLLM: Accelerating Diffusion Large Language Models via Spatio-Temporal Redundancy Reduction
- Early Decisions Matter: Proximity Bias and Initial Trajectory Shaping in Non-Autoregressive Diffusion Language Models
- KLASS: KL-Guided Fast Inference in Masked Diffusion Models
- DART: Draft-Agreement Routing for Training-Free Adaptive Thinking Budgets in Hybrid Reasoning Models
- LESS Is More: Mutual-Stability Sampling for Diffusion Language Models
- Fast-Decoding Diffusion Language Models via Progress-Aware Confidence Schedules
- Large Language Diffusion Models
- Reasoning on the Manifold: Bidirectional Consistency for Self-Verification in Diffusion Language Models
- TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decoding
- Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principles
- Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding
- Controlling the Risk of Corrupted Contexts for Language Models via Early-Exiting
- Statistical Early Stopping for Reasoning Models
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering