Veritas: A Verifier Generation Harness for Agent Tasks
cs.AI, cs.CR
Submitted: 2026-11-09
Updated: 2026-11-09
Code: https://github.com/anomalyco/opencode
Terminology
Sources
- Heterogeneous Prompting and Execution Feedback for SWE Issue Test Generation and Selection
- MCP-Atlas: A Large-Scale Benchmark for Tool-Use Competency with Real MCP Servers
- CodeT: Code Generation with Generated Tests
- TICKing All the Boxes: Generated Checklists Improve LLM Evaluation and Generation
- SABER: Small Actions, Big Errors -- Safeguarding Mutating Steps in LLM Agents
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- ExecRubrics: Executable Tool-Augmented Rubrics for Verifiable and Efficient Long-Form Evaluation
- GLM-5: from Vibe Coding to Agentic Engineering
- Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains
- CLI-Universe: Towards Verifiable Task Synthesis Engine for Terminal Agents
- DeepSWE: Measuring Frontier Coding Agents on Original, Long-Horizon Engineering Tasks
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
- AssertFlip: Reproducing Bugs via Inversion of LLM-Generated Passing Tests
- SAGE: State-Grounded, Abstention-Aware Evaluation of Task-Oriented Dialogue Agents
- Tulu 3: Pushing Frontiers in Open Language Model Post-Training
- Meta-Harness: End-to-End Optimization of Model Harnesses
- Let's Verify Step by Step
- Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses
- AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories
- Rethinking Verification for LLM Code Generation: From Generation to Testing
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents