Understanding Why Language Models Hallucinate: Testing Reasoning Against Priors
The study investigates whether language model hallucinations stem from missing knowledge or incorrect reasoning paths. It finds that hallucinations arise from 'inference misalignment,' where statistical shortcuts learned during training override prompt constraints, favoring frequently observed associations over required factual paths.