From Suppression to Repair: Mitigating Object Hallucination in Large Vision-Language Models via Localized Distribution Alignment
Large vision-language models often hallucinate objects when generating content. Instead of simply removing potentially useful information, this method proposes ResOT, a training-free technique that repairs these errors during inference. It works by identifying and aligning the 'hallucinated' parts of the model's hidden representations with the 'faithful' ones using distribution alignment, significantly reducing hallucinations while maintaining or improving overall performance.