CyberBinGym: Can AI Agents Turn Binary Patches into Real Exploits?
cs.CR, cs.AI, cs.LG
Submitted: 2026-11-07
Updated: 2026-11-07
Code: https://github.com/UKGovernmentBEIS/inspect_ai
Terminology
Sources
- CrackMeBench: Binary Reverse Engineering for Agents
- ExploitBench: A Capability Ladder Benchmark for LLM Cybersecurity Agents
- ARVO: Atlas of Reproducible Vulnerabilities for Open-Source Software
- The Next Challenge for Agentic Cybersecurity: A Realistic, Contamination-Free Reverse Engineering Benchmark
- Decompile-Bench: Million-Scale Binary-Source Function Pairs for Real-World Binary Decompilation
- REBENCH: A Procedural, Fair-by-Construction Benchmark for LLMs on Stripped-Binary Types and Names (Extended Version)
- Large Language Models for Cyber Security: A Systematic Literature Review
- BountyBench: Dollar Impact of AI Agent Attackers and Defenders on Real-World Cybersecurity Systems
Related papers
- From Strategy Search to Policy Learning: MCTS-Guided Adversarial Co-Evolution for LLM Safety Alignment
- Privacy Leakage from Recurrent State in LLM Inference
- Incrementally Verifiable Oracle Aided Computation
- Privilege and confidentiality in generative AI workflows
- BadMoE: Backdooring Mixture-of-Experts LLMs via Routing-Aware Triggers and Task-Underutilized Experts
- CoVault: Secure, Scalable Analytics of Personal Data