On the Tip of the Tongue: Why LLMs Hallucinate Answers They Can Decode
The study found LLMs hallucinate because they fail to properly integrate internal uncertainty signals into their final output. While models can detect unanswerable inputs, this detection signal is weakly linked to the output layer, causing uncertainty representations to fragment instead of forming a unified 'I don't know' state. This decoupling allows training objectives to force confident, incorrect predictions.