Readout is not Recovery: Dissociating Coordinate Emission from Visual-Corruption Repair in Vision-Language Models

arXiv:2609.32757 · cs.AI, cs.CV, cs.LG · Submitted 2026-09-26 · Read on arXiv

cs.AI, cs.CV, cs.LG

Submitted: 2026-09-26

Updated: 2026-09-26

Comments: 29 pages (14 main + appendix), 2 figures, 13 tables

License: http://creativecommons.org/licenses/by/4.0/

The gist: VLM bounding-box localization is both language generation and spatial commitment.

Terminology

Abstract

VLM bounding-box localization is both language generation and spatial commitment. Parseable fields such as bbox 2d make localization easy to score, but dimensions that emit coordinate tokens need not repair localization after visual evidence is damaged. We study this readout/recovery separation in Qwen3-VL-4B-Instruct on single-object COCO grounding. We compare clean coordinate-token readout rankings with corruption-derived repair rankings, using object-mask endpoint replacement for recovery and clean-input flooring for depth localization. In Qwen3-VL, coordinate-token rankings are inert through layer 24, load-bearing from layers 32-35, and peak at layer 34; corruption-derived rankings harm layers 16-24 but become beneficial near layer 35/final. A Kimi-VL-A3B diagnostic shows a matching output-proximal transition despite a different box format. Object-mask recovery separates rank budgets: k=250 shows necessity, k=500 shows Top- k restoration above random, and k=d/2 is largely capacity-driven. Partial-occlusion sweeps reveal that high-overlap coordinate-token sets can hurt at k=1000 and help mainly at half-width, while population corruption-derived sets provide no reliable fixed repair set. Edge-attribution patching shows coordinate-token paths are high precision but low recall for detection recovery, and RMSNorm quasi-layer controls do not close the endpoint-repair gap. Endpoint coordinate triage is therefore a useful circuit prior, but occlusion recovery requires a separate benchmark.

Sources

Related papers