Rethinking the Effectiveness of Contrastive Decoding in Mitigating Hallucinations in MLLMs

arXiv:2607.25196 · cs.LG · Submitted 2026-07-28 · Read on arXiv

cs.LG

Submitted: 2026-07-28

Updated: 2026-10-03

Terminology

• Hallucination — 6× in this paper · explained in 4 episodes
• Benchmark — 2× in this paper · explained in 2 episodes
• Grounding — 2× in this paper · explained in 3 episodes
• Visual Grounding — 2× in this paper · explained in 2 episodes
• Distribution Shift — 1× in this paper · explained in 7 episodes
• Generalizability — 1× in this paper · explained in 6 episodes

Sources

• Improved Baselines with Visual Instruction Tuning
• Qwen2.5-VL Technical Report
• InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models
• Learning Transferable Visual Models From Natural Language Supervision
• A Survey on Hallucination in Large Vision-Language Models
• Object Hallucination in Image Captioning
• Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning
• Contrastive Decoding: Open-ended Text Generation as Optimization
• Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding
• Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding
• Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models
• Evaluating Object Hallucination in Large Vision-Language Models
• BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
• MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
• Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
• How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites
• InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
• DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models
• Multi-Modal Hallucination Control by Visual Information Grounding
• IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decoding

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks