When Linguistic and Internal Confidence Diverge in Large Language Models
The episode discusses the paper "When Linguistic and Internal Confidence Diverge," which found that an AI's linguistic confidence often diverges from its internal probability. This divergence is structural, suggesting that the linguistic output acts as a "lossy channel." Hosts conclude that relying on a single confidence score is insufficient, necessitating a multi-axis evaluation framework.