PromptShift-CRC: Drift-Aware Conformal Risk Control for Foundation Models Under Prompt and Domain Shift
Jeffery Opoku, David Banahene
stat.ML, cs.LG
Submitted: 2026-06-14
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification
- Conformal Prediction with Large Language Models for Multi-Choice Question Answering
- When Meaning Stays the Same, but Models Drift: Evaluating Quality of Service under Token-Level Behavioral Instability in LLMs
- Uncertainty Awareness of Large Language Models Under Code Distribution Shifts: A Benchmark Study
- TruthfulQA: Measuring How Models Mimic Human Falsehoods
- G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment
- SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models
- Quantifying non deterministic drift in large language models
- Conformal Prediction Beyond the Seen: A Missing Mass Perspective for Uncertainty Quantification in Generative Models
- LongHalQA: Long-Context Hallucination Evaluation for MultiModal Large Language Models
- Measuring Distribution Shift in User Prompts and Its Effects on LLM Performance
- COPU: Conformal Prediction for Uncertainty Quantification in Natural Language Generation
- SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistency
Related papers
- Behavior of prediction performance metrics with rare events
- Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs
- A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
- Scalable Posterior Uncertainty for Flexible Density-Based Clustering
- A Transport Geometry of Benign Overfitting
- Efficient quadratic entropy with distance sketches