Vision-language models for chest radiography do not always need the image
Researchers tested whether medical vision-language models truly read chest X-rays or just use linguistic knowledge. They used a 'causal audit' by masking image regions to see if correct answers changed. The finding is that benchmark accuracy doesn't prove image use; an intervention test is needed to distinguish between reading an image and relying on prior text knowledge.