BIRD: Distilling Decision Boundaries into Rationales for MLLM Adaptation
cs.AI
Submitted: 2026-09-27
Updated: 2026-09-27
Comments: 25 pages, 13 figures
License: http://creativecommons.org/licenses/by/4.0/
The gist: Adapting general-purpose multimodal large language models (MLLMs) to specialized domains requires learning domain-specific decision criteria, which often hinge on subtle visual distinctions between
Terminology
Abstract
Adapting general-purpose multimodal large language models (MLLMs) to specialized domains requires learning domain-specific decision criteria, which often hinge on subtle visual distinctions between otherwise plausible answers. Rationale augmentation aims to expose such evidence through additional observations or inter-sample comparisons, yet a visually valid cue is not necessarily decision-relevant: it may describe how samples differ without changing the model's relative preference between competing answers. We therefore introduce BIRD, a self-improving Boundary-Informed Rationale Distillation framework that uses model-specific confusions to locate unresolved local decision boundaries and distills the evidence that resolves these confusions into rationales. For each sample, BIRD retrieves candidate neighbors from the target MLLM's own representation space and selects the most confusable one according to its answer preferences. It then generates answer-blind candidate evidence from their visual differences and functionally verifies which evidence most effectively strengthens the model's preference for the correct answer while avoiding inappropriate transfer across the pair. The verified evidence is then distilled into a single-sample rationale for standard supervised fine-tuning. Experiments on medical and chart VQA show that BIRD outperforms competing rationale-augmentation methods across two target MLLMs, while further analyses demonstrate clearer separation of confusable answers and stronger gains from model-matched supervision.
Sources
- OpenMedReason: Scientific Reasoning Supervision for Medical Vision-Language Models
- ChartQA-X: Generating Explanations for Visual Chart Reasoning
- Med-Scout: Curing MLLMs' Geometric Blindness in Medical Perception via Geometry-Aware RL Post-Training
- Diving into Self-Evolving Training for Multimodal Reasoning
- Through the Lens of Contrast: Self-Improving Visual Reasoning in VLMs
- Improving Reasoning in Vision-Language Models via Perception Verified Self-Training
- Kimi K3: Open Frontier Intelligence
- Towards Rationale-Answer Alignment of LVLMs via Self-Rationale Calibration
- Retrieving Counterfactuals Improves Visual In-Context Learning
- ChartBench: A Benchmark for Complex Visual Reasoning in Charts
- Do Medical Vision Language Models Actually See? A Counterfactual Grounding Framework and Hard-Negative Contrastive Training for Visually-Reliant Medical VLMs
- InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models
- MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understanding
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems