Conformalized Large Language Models under Configuration Shift
cs.LG
Submitted: 2026-08-02
Updated: 2026-08-30
Comments: Accepted to EMNLP 2026 as a main conference paper
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: Conformal prediction (CP) is a distribution-free framework for uncertainty quantification that has recently been adapted to large language models (LLMs), providing prediction sets with finite-sample
Terminology
Abstract
Conformal prediction (CP) is a distribution-free framework for uncertainty quantification that has recently been adapted to large language models (LLMs), providing prediction sets with finite-sample coverage guarantees under exchangeability. Yet for LLMs, nonconformity scores are often induced by an inference pipeline, not just a fixed model, making them depend not only on the data distribution but also on configurable factors such as the prompt template, decoding parameters, and deployment setting. Since such configurations are routinely modified in practice but rarely treated as a source of shift, their impact on CP validity remains poorly understood. We call this configuration shift and study it systematically along three axes: prompt template, decoding temperature, and weight quantization. In a broad empirical study spanning 9 LLMs, 4 datasets, and 4 nonconformity scores, we find that configuration shift consistently erodes CP validity, often driving empirical coverage below the target. By contrast, efficiency is largely preserved: valid prediction sets remain close in size to the i.i.d. baseline. We derive coverage lower bounds that attribute this loss to a discrepancy between calibration and test score distributions, and use their finite-sample plug-in versions as empirical diagnostics of shift severity. We further show that these findings lead to practical mitigations: bound-inspired recalibration is effective with limited test examples, while fragility-aware calibration ensembling recovers much of the lost coverage without test data.
Sources
- A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification
- Gemma 3 Technical Report
- Statistically-Lossless Quantization of Large Language Models
- Language Models (Mostly) Know What They Know
- Game-Theoretic Defenses for Robust Conformal Prediction Against Adversarial Attacks in Medical Imaging
- A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directions
- The Llama 3 Herd of Models
- Multi-Distribution Robust Conformal Prediction
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks