VERA: Scaling Verifiable Environments for Agentic co-Evolution
cs.AI
Submitted: 2026-10-05
Updated: 2026-10-05
Code: https://github.com/zapier/AutomationBench
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: Competent agents need precise and verifiable environments, such as sandboxes that are resumable at any stage and evolve from observable evidence.
Abstract
Competent agents need precise and verifiable environments, such as sandboxes that are resumable at any stage and evolve from observable evidence. However, most long-horizon work exposes how rare these are: for example, an agent in medical research must ground a finding, classify it, and write a report over dozens of dependent steps, yet recent environments score only the outcome. To address the challenges in stable training, we present VERA, which builds such environments at scale and lets agents evolve on them. VERA builds these environments from initial trajectories: an agent writes rubrics, executable checks, a judge verifies each sandbox, and only those that pass enter the training bank. On these environments, VERA alternates between two updates: train the model with rubric rewards, or edit the harness skills. We also create a verifier which gates model checkpoints and harness edits using explicit development-set acceptance criteria. This attribution distinguishes VERA's co-evolution from single-axis baselines: its updates target not only the cause but the outcome. With an open-source corpus of 9,000+ long-horizon verifiable environments, a 9B model paired with its co-evolved agent beats the strongest baseline by 10.3 and 13.0 points in the two domains. At 27B, it surpasses the baseline on AutoCoWorkBench (71.6) and AutoMedBench (80.7), transfers to unseen workflows, and retains general capabilities.
Sources
- On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes
- GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning
- ReflectRL: Learning from Golden Negative Trajectories via Reflective-to-Direct Reasoning
- MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains
- R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents
- Adam: A Method for Stochastic Optimization
- Tulu 3: Pushing Frontiers in Open Language Model Post-Training
- Meta-Harness: End-to-End Optimization of Model Harnesses
- Let's Verify Step by Step
- AutoMedBench: Towards Medical AutoResearch with Agentic AI Models
- Agent Lightning: Train ANY AI Agents with Reinforcement Learning
- Training Software Engineering Agents and Verifiers with SWE-Gym
- GPQA: A Graduate-Level Google-Proof Q&A Benchmark
- GTM: Simulating the World of Tools for AI Agents
- AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environments
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems