V-Zero: Gating On-Policy Distillation with Contrastive Visual Evidence for Fine-Grained Visual Reasoning
summary
The gist
Fine-grained visual reasoning requires multimodal large language models (MLLMs) to identify task-relevant visual evidence and ground their reasoning in local image regions The single most informative…
In short
V-Zero is an answer-label-free framework for visual reasoning that uses contrastive evidence gating to prune redundant reasoning tokens during On-Policy Distillation. It trains a model by contrasting positive and negative visual views of the same image, using this signal to focus distillation on the most informative parts of the visual evidence, leading to significant performance gains.
Key concepts
- On-Policy Distillation (OPD)
- This is a training method where a student model learns from an older 'teacher' model by observing its behavior on actual data trajectories. V-Zero reinterprets OPD as an objective that can be improved by considering visual evidence, rather than relying solely on textual answers.
- Contrastive Evidence Gating
- This mechanism uses paired positive and negative views of a visual input to create a 'gate' for token-level distillation. It compares the log-probabilities generated under these different views to determine which parts of the visual evidence are most reliable, allowing the model to focus its learning on high-quality information.
- Trajectory-Level Evidence Score
- This score aggregates the contrastive evidence gap across an entire reasoning path (trajectory). By measuring how much better a positive visual view is compared to a negative one for different tokens in a sequence, V-Zero creates an overall advantage score that guides the distillation process.
- Answer-Label-Free Framework
- V-Zero operates without needing any pre-annotated textual answers or labels during training. It derives its learning signals purely from visual evidence comparisons, making it more efficient and flexible than methods dependent on ground-truth text.
Terminology used across episodes
This episode discusses
- V-Zero: Gating On-Policy Distillation with Contrastive Visual Evidence for Fine-Grained Visual Reasoning · Paper Radio
- On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes
- Qwen3-VL Technical Report
- Are We on the Right Way for Evaluating Large Vision-Language Models?
- SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- GRIT: Teaching MLLMs to Think with Images
- Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes
- DeepEyesV2: Toward Agentic Multimodal Model
- Reinforcement Learning via Self-Distillation
- Video-OPD: Efficient Post-Training of Multimodal Large Language Models for Temporal Video Grounding via On-Policy Distillation
- Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
- Self-Distillation Enables Continual Learning
- Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers
- Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning
- Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception
- Self-Distilled RLVR
- MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities
- Thyme: Think Beyond Images
- MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?
- Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models
The paper
V-Zero: Gating On-Policy Distillation with Contrastive Visual Evidence for Fine-Grained Visual Reasoning · Read on arXiv
Haoxiang Sun, Zhihang Yi, Langxuan Deng, Yuhao Zhou, Peiqi Jia, Jian Zhao, Li Yuan, Jiancheng Lv
Sichuan University · –Xi’an Jiaotong University · –TeleAI of China Telecom · –Peking University
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "V-Zero: Gating On-Policy Distillation with Contrastive Visual Evidence for Fine-Grained Visual Reasoning".
Paper summary: Tom: So to wrap up this V-Zero paper, Haoxiang Sun and his team have proposed an answer label free framework for visual reasoning using contrastive evidence gating within on policy distillation. They show that by comparing positive and negative visual evidence views from a teacher side, we can gate the dense token level distillation process.
Jane: It’s about moving away from needing human annotated answer labels by using this contrastive mechanism to evaluate trajectory reliability during training. The core finding is that this method improves fine grained visual reasoning by an average of three point one points compared with a Qwen three point five base model, all while being significantly more efficient than prior methods.
Lu: The paper demonstrates that this contrasts with the limitations of standard on policy distillation, which only aligns tokens locally and can’t score the correctness of the full trajectory once a mistake is made. V-Zero addresses that by using those paired views to gate supervision.
Meng: For deployment, it means we can potentially train these models faster and cheaper because we aren't relying on massive amounts of human labeled reasoning traces for supervised fine tuning. The efficiency gains are significant, over five times faster than some supervised fine tuning methods and over ten times faster than reinforcement learning baselines.
Jane: In simpler terms, this paper shows a clever way to train an AI to reason visually by having the model learn from its own experience while using a teacher's comparison of different visual views to guide it away from mistakes without needing human answers. That’s the essence of V-Zero: Gating On-Policy Distillation with Contrastive Visual Evidence for Fine-Grained Visual Reasoning.
Conclusion: Tom: So, V-Zero is this new framework for visual reasoning that uses contrastive evidence gating to prune reasoning tokens.
Jane: It seems like they’re trying to get around needing those expensive human answer labels by using visual comparisons instead.
Tom: Exactly, it reinterprets on policy distillation as a negative-free stopgradient alignment objective. It’s about cutting out the exploration part that takes so much time with reinforcement learning.
Jane: So the core idea is building a training signal by contrasting positive and negative views of visual evidence to gate that token-level distillation process.
Tom: And they found this contrastive gating strengthens the model for rollouts where the tokens are better supported by those positive views. It discards the negative view, which is smart.
Jane: The result they showed is an average improvement of three point one points on fine-grained visual reasoning compared to a Qwen three point five base model.
Tom: That number's concrete, that’s what I like to see. And they also said this method is over five times faster than previous supervised fine tuning methods and ten times faster than reinforcement learning baselines.
Jane: It shows that you can get better reasoning performance without needing massive amounts of labeled data, which is a big deal for deployment.
Tom: True, because we’re seeing these gains from training-time visual evidence crops rather than relying on ground-truth answer labels during the final fine tuning step.
Jane: But what this really means for us is that we can build more efficient AI that still has strong visual perception across different benchmarks like HR four K and ZoomBench.
Tom: Yeah, it seems to be a solid method for improving how AI learns to see and reason about complex visuals without the heavy overhead of traditional supervision.
Jane: We’ll keep digging into those ablation studies next to see exactly where that gating mechanism provides the most benefit for different visual tasks.
More episodes
- 2603.28595-Optimistic Actor-Critic with Parametric Policies for Linear Markov Decision Processes
- 2603.29765-Training-Free Dynamic Upcycling of Expert Language Models
- 2605.00778-Observable Performance Does Not Fully Reflect Adaptive System Organization: A Multi-Level Analysis of Gait Dynamics Under Occlusal Constraint
- 2605.06906-TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond
- 2605.09165-Sparse Layers are Critical to Scaling Looped Language Models
- 2605.09439-Inverse Design for Conditional Distribution Matching
- 2605.10313-Signature Approach for Contextual Bandits with Nonlinear and Path-dependent Rewards: Regret Bound and Statistical Inference
- 2605.10878-Minimum Neural Weight Norm = Kolmogorov Complexity
- 2605.11617-MIST: Reliable Streaming Decision Trees for Online Class-Incremental Learning via McDiarmid Bound
- 2605.15085-From Data to Action: Accelerating Refinery Optimization with AI