Query-Dependent Use of Generated Descriptions for Reliable Visual Question Answering
The study investigates how self-generated captions affect a Vision-Language Model's answers, finding that caption usefulness depends entirely on the specific question asked. A caption helps global questions but harms detail questions because it redirects the model's attention away from precise visual details. This leads to GEASS, a training-free module that adaptively decides how much trust to place in a caption for each query by fusing three signals.