Seeing Isn't Knowing: Do VLMs Know When Not to Answer Spatial Questions (and Why)?
This work tests if vision-language models know when visual evidence is unreliable for spatial questions. Using a controlled framework with occlusion and perspective ambiguity, researchers found models often give overconfident wrong answers. The study shows that models struggle to recognize when to abstain and need better methods, like diverse fine-tuning, to handle uncertainty reliably.