Visual Parallel Search: Learning to Search High-Resolution Images with Parallel Tile Inspection and Adaptive Zoom
cs.CV, cs.AI
Submitted: 2026-09-29
Updated: 2026-09-30
Project page: https://xijia-tao.github.io/vps
License: http://creativecommons.org/licenses/by/4.0/
The gist: High-resolution visual question answering often fails because a multimodal model does not acquire the small, spatially localized evidence needed to answer a question.
Terminology
Abstract
High-resolution visual question answering often fails because a multimodal model does not acquire the small, spatially localized evidence needed to answer a question. Sequential zooming can recover detail, but it asks the main model to choose a region before obtaining a reliable overview. We introduce VPS, a visual parallel-search framework in which a main agent first invokes grid search to inspect image tiles in parallel with question-conditioned sub-agents, and then adaptively invokes zoom in on a precise or merged region. The same interface supports both training-free inference and post-training of the main and sub-agents. Across five benchmark splits and three model sizes, VPS improves mean accuracy over dedicated zoom-only search in 14 of 15 same-model comparisons, with gains up to 8.0 points and especially strong improvements for smaller main models. ZoomBench retains an approximately 3.2-point gain at every tested size. We further develop a supervision pipeline with hint-free verification and a paired role-specific GRPO surrogate for learning the controller and tile-reader roles. SFT improves observed accuracy on all five benchmark splits, including a 4.17-point gain on HR-Bench 4K. Role-specific RL further reshapes search behavior: main-only RL reduces mean tool use from 2.65 to 2.11 with similar pass@1 in an internal four-response evaluation, while external accuracy changes are mixed. Joint training reveals an asymmetry between local evidence reading and global search control. Together, these results support VPS as an effective inference-time scaffold and a trainable decomposition for visual evidence acquisition.
Sources
- Multi-Agent Collaboration via Evolving Orchestration
- LoRA: Low-Rank Adaptation of Large Language Models
- MAS-Orchestra: Understanding and Improving Multi-Agent Reasoning Through Holistic Orchestration and Controlled Benchmarks
- TIR-Bench: A Comprehensive Benchmark for Agentic Thinking-with-Images Reasoning
- Proximal Policy Optimization Algorithms
- Thinking With Tools, Not With Pixels: Tool Calls as Text Scaffolds for Visual Reasoning
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration
- Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception
- WideSearch: Benchmarking Agentic Broad Info-Seeking
- Small Model as Master Orchestrator: Learning Unified Agent-Tool Orchestration with Parallel Subtask Decomposition
- Orchestra-o1: Omnimodal Agent Orchestration
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models