CV-OPD: Contrastive Vision On-policy Distillation

arXiv:2610.13264 · cs.CV, cs.LG, cs.MM · Submitted 2026-10-09 · Read on arXiv

cs.CV, cs.LG, cs.MM

Submitted: 2026-10-09

Updated: 2026-10-09

Comments: 16 pages, 7 figures, contrastive learning for on-policy distillation of VLMs. Computer Vision. Post-training. VLM. LLM

Code: https://github.com/HaoLin3419/Contrastive-Vision-OPD

Project page: https://haolin3419.github.io/Contrastive-Vision-OPD

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Distillation — 1× in this paper · explained in 2 episodes
• On-Policy Distillation — 1× in this paper · explained in 3 episodes

Sources

• Qwen3-VL Technical Report
• SenseNova-MARS: Empowering Multimodal Agentic Reasoning and Search via Reinforcement Learning
• Kimi K2.5: Visual Agentic Intelligence
• Answer-Conditioned Chains of Thought Degrade Verifiable-Reasoning Distillation in Large Language Models
• Visual Contrastive Self-Distillation
• Visual-Advantage On-Policy Distillation for Vision-Language Models
• gpt-oss-120b & gpt-oss-20b Model Card
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
• Self-Distillation Enables Continual Learning
• Region-Level Policy Optimization for Fine-grained MLLM Perception
• V-Zero: Answer-Label-Free On-Policy Distillation with Contrastive Evidence Gating for Fine-Grained Visual Reasoning
• Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception
• AMR-SD: Asymmetric Meta-Reflective Self-Distillation for Token-Level Credit Assignment
• Decomposed On-Policy Distillation for Vision-Language Reasoning: Steering Gradients for Visual Grounding
• Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation
• VAD: Attributing Visual Evidence for Target Reconstruction in Multimodal On-Policy Distillation
• MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models