EviViT: Evidence-Adaptive Vision Transformers for Fine-Grained Perception
cs.CV
Submitted: 2026-09-29
Updated: 2026-09-29
Comments: 23 pages, 9 figures. Code: https://github.com/YXNiu/EviViT Data: https://huggingface.co/datasets/YXNiu/Human-Search-Traces
Code: https://github.com/YXNiu/EviViT
License: http://creativecommons.org/licenses/by/4.0/
The gist: Fine-grained visual perception enables vision-language models to distinguish subtle attributes and ground their answers in visual evidence.
Terminology
Abstract
Fine-grained visual perception enables vision-language models to distinguish subtle attributes and ground their answers in visual evidence. In high-resolution scenes, processing the whole image at greater resolution spends visual tokens on irrelevant content, while isolated crops can lose the context needed to interpret the selected evidence. We introduce EviViT, a lightweight attachment that learns where a pretrained vision transformer should acquire detail. Human visual-search traces supervise a question-conditioned evidence density, which guides regional re-reading from the original pixels and the allocation of visual tokens. A sparse, coordinate-aware bridge then connects the regional features to the global scene, allowing the host to interpret precise evidence in context. Learned with the host backbone frozen, the attachment serves both the base model and compatible post-trained descendants without refitting. Experiments across nine hosts show consistent gains in average fine-grained accuracy. Matched-budget comparisons further show that EviViT outperforms global-only processing at every tested token ceiling while using fewer visual tokens.
Sources
- Qwen3-VL Technical Report
- OPD-V: Visual On-Policy Self-Distillation with Modality Balance
- Token Merging: Your ViT But Faster
- VisRL: Intention-Driven Visual Perception via Reinforced Reasoning
- SIFThinker: Spatially-Aware Image Focus for Visual Reasoning
- 4DThinker: Thinking with 4D Imagery for Dynamic Spatial Understanding
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- PixelEyes: Decoupling Perception and Reasoning for Pinpoint Visual Evidence Seeking
- Self-Prophetic Decoding to Unlock Visual Search in LVLMs
- LoRA: Low-Rank Adaptation of Large Language Models
- OccamToken: Efficient VLM Inference with Training-Free and Budget-Adaptive Token Pruning
- PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models
- Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression
- Vero: An Open RL Recipe for General Visual Reasoning
- Focus-Scan-Refine: From Human Visual Perception to Efficient Visual Token Pruning
- Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution
- Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception
- Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models