BiG-SURE - Bipartite Graph for Semantic Uncertainty and Reliability Estimation of LLMs
The discussion of 'BiG-SURE' explores a method for estimating semantic uncertainty and reliability in Large Language Models. The technique compares low-temperature anchor responses with high-temperature probe responses using a bipartite graph weighted by NLI entailment scores. This allows the models to quantify their own confidence and detect when they are drifting or hallucinating, leading to more trustworthy AI systems.