DUD: Decoupled Update Dynamics for Reliable Uncertainty Quantification in Large Language Models
Yixin Bu, Runze Xia, Guanyun Zou, Yupeng Ji, Haodong Liu, Piji Li
cs.CL
Submitted: 2026-08-04
Comments: ACL 2026 Main Conference
License: http://creativecommons.org/licenses/by/4.0/
The gist: Accurate Uncertainty Quantification (UQ) is critical for reliable deployment of Large Language Models (LLMs), yet traditional probability-based metrics often fail to capture the model's true
Terminology
Abstract
Accurate Uncertainty Quantification (UQ) is critical for reliable deployment of Large Language Models (LLMs), yet traditional probability-based metrics often fail to capture the model's true epistemic state. While recent mechanistic approaches leverage hidden state dynamics, they typically aggregate residual stream updates, conflating the distinct roles of parametric memory (Feed-Forward Networks) and contextual processing (Attention). We argue that this aggregation obscures fine-grained mechanistic conflicts, such as memory-context misalignment, that are fundamental indicators of uncertainty. To address this, we introduce Decoupled Update Dynamics(DUD), a framework that explicitly decouples FFN and Attention contributions via noise-induced causal interventions. By quantifying the independent restoration capabilities of each module, we construct a dual-stream dynamic profile that captures the model's internal fragility. Extensive experiments demonstrate that DUD significantly outperforms state-of-the-art baselines in both uncertainty estimation and calibration, while exhibiting superior cross-dataset generalization, validating decoupled dynamics as a robust proxy for model faithfulness.
Sources
- The Llama 3 Herd of Models
- Qwen2.5-Coder Technical Report
- TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
- INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection
- Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps
- DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models
- Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs
- Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation
- HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models
- Uncertainty Estimation in Autoregressive Structured Prediction
- The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets
- SQuAD: 100,000+ Questions for Machine Comprehension of Text
- Out-of-Distribution Detection and Selective Generation for Conditional Language Models
- A Mechanistic Interpretation of Arithmetic Reasoning in Language Models using Causal Mediation Analysis
- Gemma 2: Improving Open Language Models at a Practical Size
- BERTScore: Evaluating Text Generation with BERT
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering