Confidently Wrong: Detecting Hallucinations in Financial Question Answering from LLM Internal States
Researchers tested whether internal model activations can detect errors in confidently wrong answers from large language models used for financial questions. They trained linear probes on these internal states and found they significantly outperformed standard output confidence signals in spotting hallucinations across different LLMs. This suggests probing is a cost-effective way to triage risky LLM outputs for human review.