Answer or Defer? Testing VLMs' Stated Confidence on Brain MRI
cs.CV, cs.AI, cs.LG
Submitted: 2026-11-09
Updated: 2026-11-09
Comments: Accepted to the WACV 2027 FABRIC Workshop
Code: https://github.com/amir-sbg/VLM-Answer-or-Defer
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Sources
- Qwen2.5-VL Technical Report
- Overconfidence and Calibration in Medical VQA: Empirical Findings and Hallucination-Aware Mitigation
- Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling
- The 2024 Brain Tumor Segmentation (BraTS) Challenge: Glioma Segmentation on Post-treatment MRI
- Gemma 3 Technical Report
- Gemma 4 Technical Report
- GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning
- Language Models (Mostly) Know What They Know
- Adapt or Calibrate? How Local Labels Buy Certified Coverage in Medical VQA
- Reported Confidence in LLMs Tracks Commitment More Than Correctness
- Label Agreement Does Not Measure Authorization
- MedGemma Technical Report
- Lost in the Hype: Revealing and Dissecting the Performance Degradation of Medical Multimodal Large Language Models in Image Classification
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs