Dual-Pathway Circuits of Object Hallucination in Vision-Language Models
The study investigated why vision-language models hallucinate objects by finding a consistent dual-pathway organization across diverse models. It discovered a visual grounding pathway for correct predictions and a separate hallucination pathway that drives errors. This mechanism is shared across architectures and can be suppressed to reduce hallucinations without significantly hurting accuracy.