TxBench: Verifying Public Incident Data to Benchmark Smart-Contract Attack Detectors
cs.CR
Submitted: 2026-11-03
Updated: 2026-11-11
Comments: 5 pages
Code: https://github.com/sanbir/evm-hack-registry
License: http://creativecommons.org/licenses/by/4.0/
The gist: Malicious transactions on smart contracts cause billions of dollars in losses.
Terminology
Abstract
Malicious transactions on smart contracts cause billions of dollars in losses. To evaluate their detectors, most datasets copy the attack transactions that public incident databases cite and pair them with random transactions. However, these databases never check their citations against execution, and random transactions rarely resemble attacks. We present TXBENCH, a benchmark that verifies both attacks and legitimate transactions against their execution. TXBENCH labels a cited transaction as an attack only when its execution takes value from others and independent evidence ties it to its incident. Its legitimate transactions perform the same operations as attacks and carry positive evidence of legitimacy. Applying these checks to two public databases, we find that they share only 10.6% of their 1,850 cited transactions and that only 1,012 are verifiable attacks, worth 868.2 million. A blind manual review confirms 98/100 sampled attacks and 100/100 legitimate transactions.
Sources
- Blockchain Large Language Models
- EVMbench: Evaluating AI Agents on Smart Contract Security
- CyberChainBench: Can AI Agents Secure Smart Contracts Against Real-World On-Chain Vulnerabilities?
Related papers
- From Strategy Search to Policy Learning: MCTS-Guided Adversarial Co-Evolution for LLM Safety Alignment
- Privacy Leakage from Recurrent State in LLM Inference
- Incrementally Verifiable Oracle Aided Computation
- Privilege and confidentiality in generative AI workflows
- BadMoE: Backdooring Mixture-of-Experts LLMs via Routing-Aware Triggers and Task-Underutilized Experts
- CoVault: Secure, Scalable Analytics of Personal Data