From Tokens to Semantics: Leveraging Complementary Signals for Hallucination Detection in Black-Box LLMs
cs.CL, cs.AI
Submitted: 2026-09-02
Updated: 2026-09-07
License: http://creativecommons.org/licenses/by/4.0/
The gist: When LLMs support public-facing or high-stakes workflows, missed fabrications can harm users and institutions, while false alarms consume limited human-review capacity.
Terminology
Abstract
When LLMs support public-facing or high-stakes workflows, missed fabrications can harm users and institutions, while false alarms consume limited human-review capacity. When no trusted context or reference document is available, we study two signals accessible through black-box model APIs: semantic entropy, which measures disagreement among sampled response meanings, and uncertainty derived from token log-probabilities. Their failure modes can be complementary: semantic entropy becomes uninformative when responses form one semantic cluster, while token uncertainty can miss consistently confident errors. We extend token-based uncertainty detection by aggregating token-level signals across sampled responses through our TopK method, evaluate the hybrid CoCoA method, which combines target-response uncertainty with semantic dissimilarity, and propose and study two supervised methods: Gated, which routes single-cluster cases to an aggregated-token-feature classifier, and Stacked, which learns jointly from semantic uncertainty and broader token features. We evaluate seven benchmarks, including five public benchmarks (four text datasets and multimodal handwritten-cheque extraction) and two constructed benchmarks (Financial Summaries and Long-Text QA), using four language models. In our evaluation across models and datasets, Stacked gave the best performance in nearly half of the cases, while TopK and CoCoA remain competitive without supervised training labels, although their thresholds require careful calibration. No method is universally strongest. We therefore evaluate performance at false-positive-rate budgets from 1% to 15%, assess their sensitivity to generation and calibration choices, and examine variation across dataset characteristics.
Sources
- Improving Uncertainty Estimation through Semantically Diverse Language Generation
- Complementing Self-Consistency with Cross-Model Disagreement for Uncertainty Quantification
- Language Models (Mostly) Know What They Know
- Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation
- Semantic Energy: Detecting LLM Hallucination Beyond Entropy
- Uncertainty Estimation in Autoregressive Structured Prediction
- Estimating Semantic Alphabet Size for LLM Uncertainty Quantification
- Beyond Semantic Entropy: Boosting LLM Uncertainty Quantification with Pairwise Semantic Similarity
- Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities
- HALT: Hallucination Assessment via Log-probs as Time series
- HaluNet: Learning Hallucination Risk from Internal Signals in LLM Question Answering
- Uncertainty Quantification for LLMs through Minimum Bayes Risk: Bridging Confidence and Consistency
- Fine-Grained Uncertainty Decomposition in Large Language Models: A Spectral Approach
- Uncertainty Under the Curve: A Sequence-Level Entropy Area Metric for Reasoning LLM
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering