TxBench: Verifying Public Incident Data to Benchmark Smart-Contract Attack Detectors

arXiv:2611.04411 · cs.CR · Submitted 2026-11-03 · Read on arXiv

cs.CR

Submitted: 2026-11-03

Updated: 2026-11-11

Comments: 5 pages

DOI: 10.5281/zenodo.23758032

Code: https://github.com/sanbir/evm-hack-registry

License: http://creativecommons.org/licenses/by/4.0/

The gist: Malicious transactions on smart contracts cause billions of dollars in losses.

Terminology

Abstract

Malicious transactions on smart contracts cause billions of dollars in losses. To evaluate their detectors, most datasets copy the attack transactions that public incident databases cite and pair them with random transactions. However, these databases never check their citations against execution, and random transactions rarely resemble attacks. We present TXBENCH, a benchmark that verifies both attacks and legitimate transactions against their execution. TXBENCH labels a cited transaction as an attack only when its execution takes value from others and independent evidence ties it to its incident. Its legitimate transactions perform the same operations as attacks and carry positive evidence of legitimacy. Applying these checks to two public databases, we find that they share only 10.6% of their 1,850 cited transactions and that only 1,012 are verifiable attacks, worth 868.2 million. A blind manual review confirms 98/100 sampled attacks and 100/100 legitimate transactions.

Sources

Related papers