MG-VQA: Manipulation Grounded Visual Question Answering with VLMs
This research addresses complex visual question answering by moving vision-language models beyond static images to physical interaction tasks called MG-VQA. The PROBE framework benchmarks these agents using a simulator and evaluation suite, showing that agentic tool use significantly boosts performance over perception-only methods. A fine-tuning recipe successfully distills manipulation strategies into smaller open models with good generalization.