AI Security Leaderboard: Methodology, Results and Minimal Standard
Jasper Timm, Lukas Struppek, Ziwei Xu, Grace Cheong, Oscar Mata, Dan Zhao, Mick Yang, Isadora De Andrade, Xiaojun Jia, Yiming Li, Samuel Bauer, Heather McIntyre, Adam Gleave, Edward Yee, Kellin Pelrine
cs.CR, cs.AI, cs.CL
Submitted: 2026-08-04
License: http://creativecommons.org/licenses/by/4.0/
The gist: The AI Security Leaderboard is an independent benchmark that ranks the safeguards of frontier AI models from least to most secure.
Terminology
Abstract
The AI Security Leaderboard is an independent benchmark that ranks the safeguards of frontier AI models from least to most secure. It tests models against the FAR. AI Minimal Standard for Safeguards, which represents a minimum bar for security: meeting it does not guarantee a secure model, but failing to meet it guarantees a lack of state-of-the-art security. Version 1.0 covers severe misuse requests across chemical, biological, radiological, nuclear, and explosive (CBRNE) threats and offensive cybersecurity. In this report, we tested four leading models for universal jailbreaks in the context of this minimal standard, and found more than a hundredfold difference in security. Claude Fable 5 and GPT-5.6 Sol held against every attack we ran, with no universal jailbreak found; we estimate they would likely cost more than 14,200 to jailbreak, if it is possible with this methodology at all. Meanwhile, we found hundreds of universal jailbreaks for Grok 4.5 and Gemini 3.1 Pro; each broke for under 300, with universal jailbreaks in Grok's weakest domain, cybersecurity, accessible for as little as 24. The gap is fixable: every weakness we found belongs to a known class of attack that already has a defense deployed in production models. The leaderboard will be updated on a rolling basis as new models are released, and the evaluation methodology and Minimal Standard will be periodically revised to take into account the latest capabilities and the state-of-the-art in safeguards. The leaderboard is available at leaderboard.far.ai.
Sources
- gpt-oss-120b & gpt-oss-20b Model Card
- Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation
- International AI Safety Report 2026
- Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreaks
- Boundary Point Jailbreaking of Black-Box LLMs
- Deliberative Alignment: Reasoning Enables Safer Language Models
- Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmark
- STACK: Adversarial Attacks on LLM Safeguard Pipelines
- The Jailbreak Tax: How Useful are Your Jailbreak Outputs?
- Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming
- Exposing the Systematic Vulnerability of Open-Weight Models to Prefill Attacks
- The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
- CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale
- Jailbroken Frontier Models Retain Their Capabilities
- Universal and Transferable Adversarial Attacks on Aligned Language Models
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Estimating the Social Cost of Corporate Data Breaches
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills