Attention at Rest Stays at Rest: Breaking Visual Inertia to Mitigate Relation Hallucinations
The research introduces Inertia-aware Visual Excitation (IVE) to combat cognitive hallucinations in multimodal models. It breaks 'visual inertia'—where attention gets stuck on key visual areas—by dynamically shifting focus toward newly relevant visual cues. IVE uses trend deviation and a penalty system to ensure the model captures necessary relational understanding without needing extra training.