Two Clocks in Diffusion MLLMs: When Answers Stabilize Before Rationales Unfold
cs.CV, cs.AI
Submitted: 2026-10-01
Updated: 2026-10-01
Comments: NeurIPS 2026 Workshop on BeNTo (Beyond Next-Token Prediction - Diffusion & Flow Models for Next-Generation Decoding)
License: http://creativecommons.org/licenses/by/4.0/
The gist: An answer candidate in a masked diffusion MLLM can stabilize while its rationale is still unfolding.
Terminology
Abstract
An answer candidate in a masked diffusion MLLM can stabilize while its rationale is still unfolding. We distinguish retrospective stabilization of the logged candidate from token commitment, and examine these two clocks relative to rationale generation. Analyzing our results across three visual question-answering benchmarks, we find that 89.4-98.1% of the rationale-side canvas remains unwritten at stabilization in single-block, EOS-suppressed LaViDa runs. On V*Bench, reducing block length from 128 to 8 changes this fraction from 89.4% to 1.7%, together with answer coverage and the eligible observation window. Under EOS-enabled prompting, direct instructions improve Nemotron's overall accuracy by 15.0 and 19.5 percentage points on M3CoT and ScienceQA, but reduce LaViDa/V*Bench accuracy by 11.0 points. A symmetric decomposition associates the larger absolute component of each change with coverage rather than conditional accuracy. Matched-canvas image ablations measure visual sensitivity alongside answer stabilization, separating the two temporal readouts. Together, these measurements distinguish answer stabilization, rationale unfolding, and visual sensitivity, and identify coverage as the larger component of the prompting differences.
Sources
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting
- Measuring Faithfulness in Chain-of-Thought Reasoning
- Simple and Effective Masked Diffusion Language Models
- Diffusion Language Models Know the Answer Before Decoding
- Answer First, Reason Later: When Commitment Order Costs Accuracy in Diffusion Language Models
- Multi-Modal Hallucination Control by Visual Information Grounding
- Thinking Diffusion: Penalize and Guide Visual-Grounded Reasoning in Diffusion Multimodal Language Models
- ST-Veto: Spatio-Temporal Token Veto for Diffusion MLLMs via Taylor Prediction and Visual Grounding
- LaViDa: A Large Diffusion Language Model for Multimodal Understanding
- Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decoding
- DiffusionGemma Technical Report
- V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs
- Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models