Semantic Capability Acquisition and Specialization During Vision-Language Model Fine-Tuning

arXiv:2610.07385 · cs.CV · Submitted 2026-10-05 · Read on arXiv

cs.CV

Submitted: 2026-10-05

Updated: 2026-10-05

Comments: 67 pages, including supplementary material

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Fine-Tuning — 1× in this paper · explained in 5 episodes

Sources

• The Evolution of Out-of-Distribution Robustness Throughout Fine-Tuning
• Zero-Shot Fine-Grained Image Classification Using Large Vision-Language Models
• Real Classification by Description: Extending CLIP's Limits of Part Attributes Recognition
• Hidden in plain sight: VLMs overlook their visual representations
• Visual Classification via Description from Large Language Models
• TULIP: Token-length Upgraded CLIP
• VLMs Need Words: Vision Language Models Ignore Visual Detail In Favor of Semantic Anchors
• FG-CLIP 2: A Bilingual Fine-grained Vision-Language Alignment Model
• FG-CLIP: Fine-Grained Visual and Textual Alignment
• When and why vision-language models behave like bags-of-words, and what to do about it?
• VL-CheckList: Evaluating Pre-trained Vision-Language Models with Objects, Attributes and Relations

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models