FORALL-LEAN-AGENT for Auditable Reasoning in Formal Mathematics and Software Verification

arXiv:2610.00885 · cs.SE, cs.AI, cs.LO · Submitted 2026-10-01 · Read on arXiv

cs.SE, cs.AI, cs.LO

Submitted: 2026-10-01

Updated: 2026-10-01

Comments: Accepted to NeurIPS 2026 VeriCodeGen

Code: https://github.com/SkidanovAlex/putnambench-deepseek

Project page: https://trishullab.github.io/PutnamBench/leaderboard.html

License: http://creativecommons.org/licenses/by/4.0/

The gist: Coding agents increasingly automate Lean proof development, but successful compilation alone does not establish that a candidate proves the intended statement under acceptable assumptions.

Terminology

Abstract

Coding agents increasingly automate Lean proof development, but successful compilation alone does not establish that a candidate proves the intended statement under acceptable assumptions. We present FORALL-LEAN-AGENT, a frontend-agnostic framework for auditable reasoning in formal mathematics and software verification. The framework combines isolated workspaces, Lean tools, and fresh review with statement comparison, axiom audits, and independent proof checking where supported. Verification evidence and reviewer decisions are bound to the same candidate artifact, making acceptance traceable. We evaluate the framework on VeriSoftBench, PutnamBench, and both problems in the Lean Eval softwareverification track. On the 100-task VeriSoftBench subset, integration with FORALLLEAN-AGENT raises benchmark-rule success from 93 to 100 for GPT-5.6 Sol at low effort while reducing cost from 69 to 62. The PutnamBench evaluation accepts all 672 problems at an average of 4.72 each. These results show that agent harness design can improve correctness and efficiency while providing evidence beyond aggregate solve counts.

Sources

Related papers