VRIQ: Benchmarking and Diagnosing the Visual-Reasoning IQ of Vision-Language Models

arXiv:2602.05382 · cs.CV, cs.LG · Submitted 2026-02-05 · Read on arXiv

cs.CV, cs.LG

Submitted: 2026-02-05

Updated: 2026-10-10

Terminology

• Accuracy — 2× in this paper · explained in 1 episode
• Benchmark — 2× in this paper · explained in 2 episodes
• Benchmarking — 1× in this paper · explained in 4 episodes
• Vision-language models — 1× in this paper · explained in 2 episodes

Sources

• Qwen3-VL Technical Report
• MM-IQ: Benchmarking Human-Like Abstraction and Reasoning in Multimodal Models
• TheoremQA: A Theorem-driven Question Answering dataset
• Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
• DesignQA: A Multimodal Benchmark for Evaluating Large Language Models' Understanding of Engineering Documentation
• MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models
• Open-ended VQA benchmarking of Vision-Language models by exploiting Classification datasets and their semantic hierarchy
• GPT-4o System Card
• Intrinsic Electrochemical Limits Preceding Dendrite Penetration in Ceramic Electrolytes
• What's "up" with vision-language models? Investigating their struggle with spatial reasoning
• Vision language models are unreliable at trivial spatial cognition
• SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension
• EEE-Bench: A Comprehensive Multimodal Electrical And Electronics Engineering Benchmark
• Diverse and Tailored Image Generation for Zero-shot Multi-label Classification
• MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts
• E$^2$M: Double Bounded $\alpha$-Divergence Optimization for Tensor-based Discrete Density Estimation
• LLaMA: Open and Efficient Foundation Language Models
• The radial variation of the LMC-induced reflex motion of the Milky Way disc observed in the stellar halo
• Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset
• SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models