Harness Compilation: Which Decisions Should a Small Vision-Language Model Keep?

arXiv:2610.11231 · cs.AI, cs.CV · Submitted 2026-10-08 · Read on arXiv

cs.AI, cs.CV

Submitted: 2026-10-08

Updated: 2026-10-08

Comments: 58 pages, 10 figures, including appendices

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Accuracy — 32× in this paper · explained in 1 episode
• Trace — 26× in this paper · explained in 2 episodes
• Adaptation — 23× in this paper · explained in 1 episode
• Abstention — 16× in this paper · explained in 1 episode
• Decode — 15× in this paper · explained in 2 episodes
• Ablation — 14× in this paper · explained in 1 episode
• Benchmark — 8× in this paper · explained in 2 episodes
• Harness Compilation — 7× in this paper
• In International Conference on Learning Representations — 7× in this paper
• RAG — 7× in this paper · explained in 3 episodes
• Test-time Scaling — 5× in this paper · explained in 2 episodes
• Consistency — 4× in this paper · explained in 3 episodes
• Self-consistency — 4× in this paper · explained in 2 episodes
• Frozen model — 3× in this paper · explained in 2 episodes
• Calibration — 2× in this paper · explained in 7 episodes
• Distillation — 2× in this paper · explained in 2 episodes
• Pruning — 2× in this paper · explained in 4 episodes
• Robustness — 2× in this paper · explained in 5 episodes
• Additive Decomposition — 1× in this paper · explained in 1 episode
• Admissibility — 1× in this paper · explained in 2 episodes
• Cosine Similarity — 1× in this paper · explained in 3 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Teacher-Student Gap — 1× in this paper · explained in 2 episodes
• Transformer — 1× in this paper · explained in 2 episodes
• Vision-Language Model (VLM) — 1× in this paper · explained in 3 episodes

Sources

• GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning
• Large Language Models as Tool Makers
• Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?
• LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models
• Seeking and Updating with Live Visual Knowledge
• BLINK: Multimodal Large Language Models Can See but Not Perceive
• Visual Programming: Compositional visual reasoning without training
• AVIS: Adaptive Test-Time Scaling for Vision-Language Models
• Efficient Test-Time Scaling for Small Vision-Language Models
• DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines
• Meta-Harness: End-to-End Optimization of Model Harnesses
• Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents
• SPair-71k: A Large-scale Benchmark for Semantic Correspondence
• Scaling Open-Vocabulary Object Detection
• Where Does Harness-Optimization Value Live? Localized Gains and the Budget-Splitting Trap in Self-Evolving LLM Agents
• Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs
• DINOv2: Learning Robust Visual Features without Supervision
• AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnesses
• AgentDistill: Training-Free Agent Distillation with Generalizable MCP Boxes

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• Agents' Last Exam
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems