ClueTracer: Question-to-Vision Clue Tracing for Training-Free Hallucination Suppression in Multimodal Reasoning
cs.CV, cs.AI
Submitted: 2026-02-02
Updated: 2026-10-28
Comments: 27 pages, 12 figures, accept by EMNLP 2026 (Oral)
Code: https://github.com/VLM-RL/Ocean-R1
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large multimodal reasoning models solve challenging visual problems via explicit long-chain inference: they gather visual clues from images and decode clues into textual tokens.
Terminology
Abstract
Large multimodal reasoning models solve challenging visual problems via explicit long-chain inference: they gather visual clues from images and decode clues into textual tokens. Yet this capability also increases hallucinations, where the model generates content that is not supported by the input image or the question. To understand this failure mode, we identify reasoning drift: during clue gathering, the model over-focuses on question-irrelevant entities, diluting focus on task-relevant cues and gradually decoupling the reasoning trace from visual grounding. As a consequence, many inference-time localization or intervention methods developed for non-reasoning models fail to pinpoint the true clues in reasoning settings. Motivated by these insights, we introduce ClueRecall, a metric for assessing visual clue retrieval, and present ClueTracer, a training-free, parameter-free, and architecture-agnostic plugin for hallucination suppression. ClueTracer starts from the question and traces how key clues propagate along the model's reasoning pathway (question to outputs to visual tokens), thereby localizing task-relevant patches while suppressing spurious attention to irrelevant regions. Remarkably, without any additional training, ClueTracer improves all reasoning architectures (including R1-OneVision, Ocean-R1, MM-Eureka, etc.) by 1.21 times on reasoning benchmarks
Sources
- Qwen2.5-VL Technical Report
- Hallucination of Multimodal Large Language Models: A Survey
- Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas
- MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM
- GoT-R1: Unleashing Reasoning Capability of MLLM for Visual Generation with Reinforcement Learning
- GRIT: Teaching MLLMs to Think with Images
- Visual Description Grounding Reduces Hallucinations and Boosts Reasoning in LVLMs
- Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models
- Self-Aug: Query and Entropy Adaptive Decoding for Large Vision-Language Models
- VLM-R$^3$: Region Recognition, Reasoning, and Refinement for Enhanced Multimodal Chain-of-Thought
- See What You Are Told: Visual Attention Sink in Large Multimodal Models
- The Hidden Life of Tokens: Reducing Hallucination of Large Vision-Language Models via Visual Information Steering
- More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models
- Visual Instruction Tuning
- Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement
- Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Models
- Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering
- One RL to See Them All: Visual Triple Unified Reinforcement Learning
- MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learning
- Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Models
Related papers
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs
- From Formulas to Protocols: A Compositional View of Binary Target Segmentation Metrics
- Traffic Sign Recognition in Autonomous Driving: Dataset, Benchmark, and Field Experiment
- Efficient All-in-One Weather Restoration using Spectral Harmonization
- Peak-End-Net: A Peak-End Rule Inspired Framework for Generalizable Video Aesthetic Assessment