Improving Confidence Estimation in Language Models with Linear Representations of Ambiguity
cs.CL
Submitted: 2026-11-09
Updated: 2026-11-09
Comments: 31 pages, 21 figures, 5 tables
License: http://creativecommons.org/licenses/by/4.0/
The gist: As language models (LMs) are increasingly deployed in downstream applications, accurately estimating their likelihood of success on a given example is critical.
Terminology
Abstract
As language models (LMs) are increasingly deployed in downstream applications, accurately estimating their likelihood of success on a given example is critical. However, confidence estimates from LMs are poor predictors of their likelihood of success. Accurate confidence estimation is linked in part to the detection of aleatoric uncertainty; we hypothesize that LMs possess representations of at least some aspects of this uncertainty in the form of representations of the inherent ambiguity of the input. Using difference-in-means on minimal pairs of ambiguous and unambiguous sentences, we derive ambiguity representations in LM activations, and find that steering with them selectively (albeit weakly) modulates LMs' judgments of the input's ambiguity. We then propose DART, a fine-tuning method that encourages updates to lie within the span of the ambiguity directions, and penalizes changes to the LM activations outside of this subspace. DART greatly improves calibration, encodes these improvements into the model parameters, and maintains model accuracy on multiple-choice and generative QA tasks.
Sources
- Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs
- Differential syntactic and semantic encoding in LLMs
- Smooth ECE: Principled Reliability Diagrams via Kernel Smoothing
- Persona Vectors: Monitoring and Controlling Character Traits in Language Models
- Gemma 4 Technical Report
- The Llama 3 Herd of Models
- LoRA: Low-Rank Adaptation of Large Language Models
- Learning the Difference that Makes a Difference with Counterfactually-Augmented Data
- Same Answer, Different Confidence: Protocol Sensitivity in LLM Confidence Calibration
- How LLMs Detect and Correct Their Own Errors: The Role of Internal Confidence Signals
- A Taxonomy of Ambiguity Types for NLP
- TruthfulQA: Measuring How Models Mimic Human Falsehoods
- We're Afraid Language Models Aren't Modeling Ambiguity
- The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets
- Olmo 3
- Interpretability Can Be Actionable
- Features as Rewards: Scalable Supervision for Open-Ended Tasks via Interpretability
- Llamas Know What GPTs Don't Show: Surrogate Models for Confidence Estimation
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data