DR Pixel: Page-Native Deep Research with Vision-Language Models
cs.CV, cs.CL, cs.IR
Submitted: 2026-11-02
Updated: 2026-11-02
License: http://creativecommons.org/licenses/by/4.0/
The gist: Complex documents such as scientific papers contain rich information in tables, figures, and equations; however, existing deep research (DR) agents typically reduce papers to OCR-extracted text.
Terminology
Abstract
Complex documents such as scientific papers contain rich information in tables, figures, and equations; however, existing deep research (DR) agents typically reduce papers to OCR-extracted text. We instead ask whether vision-language models (VLMs) can enable page-native deep research: directly retrieving and reasoning over rendered document page images. We introduce DR Pixel, the first open-source end-to-end multimodal DR agent that uses rendered page images as its sole retrieval and reasoning unit, operating over a corpus of 1.06M pages. To evaluate this setting, we construct DRPixelBench, a benchmark on scientific papers requiring reasoning over text, figures, tables, and equations. Surprisingly, page-native DR already works well without training, outperforming OCR-based baselines across state-of-the-art models like GPT-5.4, Gemini-3.5-Flash, and Qwen3.6-27B, while requiring fewer tool calls. We next turn to DR training. A key challenge is that teacher trajectories for agentic supervised finetuning are noisy, mixing critical reasoning and action steps with redundant or failed attempts. To address this, we introduce counterfactual credit reweighting, a fine-grained credit assignment method that weights individual reasoning and action steps by their counterfactual contribution to the final answer. Our training improves retrieval recall by about 15 points and final accuracy by over 10 points on DRPixelBench, and likewise outperforms the untrained baseline on four existing benchmarks. On out-of-distribution ViDoRe-V3, DR Pixel comes within 1.8 points of an oracle that is given the gold pages directly. Our results suggest that the conventional OCR-then-retrieve pipeline is not only unnecessary but suboptimal: directly retrieving and reasoning over visual document pages makes complex DR both more effective and more efficient.
Sources
- OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs
- FireAct: Toward Language Agent Fine-tuning
- Nemotron ColEmbed V2: Top-Performing Late Interaction Embedding Models for Visual Document Retrieval
- What's the Best Way to Retrieve Slides? A Comparative Study of Multimodal, Caption-Based, and Hybrid Retrieval Techniques
- MMSearch: Benchmarking the Potential of Large Models as Multi-modal Search Engines
- OCR-free Document Understanding Transformer
- Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding
- Tongyi DeepResearch Technical Report
- Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Ranking
- MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agents
- Let's Verify Step by Step
- The Semantic Reader Project: Augmenting Scholarly Documents through AI-Powered Interactive Reading Interfaces
- CRAFT: Counterfactual Credit Assignment from Free Sibling Rollouts for Self-Distilled Agentic Reinforcement Learning
- VISOR: Agentic Visual Retrieval-Augmented Generation via Iterative Search and Over-horizon Reasoning
- IRPAPERS: A Visual Document Benchmark for Scientific Retrieval and Question Answering
- SlideVQA: A Dataset for Document Visual Question Answering on Multiple Images
- PIXELRAG: Web Screenshots Beat Text for Retrieval-Augmented Generation
- DeepSeek-OCR 2: Visual Causal Flow
- Beyond Outcome Rewards: Step-Level Self-Distilled Policy Optimization for Deep Search Agents
- AutoResearchBench: Benchmarking AI Agents on Complex Scientific Literature Discovery
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs