Who Verifies the Verifier? Co-Evolving Inspectable Graders with Self-Improving Agents
cs.AI, cs.CL, cs.LG
Submitted: 2026-10-08
Updated: 2026-10-08
Comments: Accepted at the NeurIPS 2026 Workshop: Who Verifies the Agents? Toward Reliable Agent Development
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: We changed the agent: did it actually get better? Every self-improving agent loop answers this hundreds of times, and every answer comes from a verifier.
Terminology
Abstract
We changed the agent: did it actually get better? Every self-improving agent loop answers this hundreds of times, and every answer comes from a verifier. On open-ended tasks none exists, so the loop is handed a hand-written rubric or a bare LLM judge grading output from a model like itself, inviting reward hacking and shared blind spots. We make the verifier the evolving object: an inspectable expression over small, mostly deterministic drawback detectors, synthesized from clustered failures, gated at birth, and selected for agreement with a ten-item anchored reference set plus consensus over unlabeled outputs, never for the agent's score. On MBPP+ it gains +0.21 held-out agreement over the hand-authored seed composition, on every seed, and ends ahead of the bare LLM judge it contains. One finding should change how co-evolved verifiers are validated: removing the anchor guards collapses the verifier into a vacuous always-pass grader, yet that collapsed verifier trains skills just as well. Downstream task score cannot certify a self-evolved verifier. Score does answer sufficiency, and there an evolved verifier can substitute: Double Ratchet, pairing the verifier with a lifecycle-managed skill loop, retains 88-110% of the lift that ground truth or a rubric buys the same loop, across code generation, enterprise text-to-SQL, and reference-free report generation. When evolved skills gamed the report rubric, an outer judge caught it and one added detector repaired it; the judge itself was wrong until given the task contract.
Sources
- Ratchet: How Reliable Must an LLM Judge Be to Retire a Skill?
- Agent Workflow Memory
- EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecycle
- AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution
- SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning
- Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills
- From Procedural Skills to Strategy Genes: Towards Experience-Driven Test-Time Evolution
- Experience Compression Spectrum: Unifying Memory, Skills, and Rules in LLM Agents
- SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks
- A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence
- TextGrad: Automatic "Differentiation" via Text
- AlphaEvolve: A coding agent for scientific and algorithmic discovery
- Concrete Problems in AI Safety
- Constitutional AI: Harmlessness from AI Feedback
- A Survey on LLM-as-a-Judge
- Agent-as-a-Judge: Evaluate Agents with Agents
- Hallucination Detection for LLM-based Text-to-SQL Generation via Two-Stage Metamorphic Testing
- ReFoRCE: A Text-to-SQL Agent with Self-Refinement, Consensus Enforcement, and Column Exploration
- Program Synthesis with Large Language Models
- Evaluating Large Language Models Trained on Code
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems