GT-HarmBench: Benchmarking AI Safety Risks Through the Lens of Game Theory
cs.AI, cs.CL, cs.CY, cs.GT, cs.MA
Submitted: 2026-02-12
Updated: 2026-09-25
Comments: Accepted at NeurIPS 2026 Main Conference. Camera-ready will be out soon. This is still the preprint
Code: https://github.com/causalNLP/gt-harmbench
License: http://creativecommons.org/licenses/by/4.0/
The gist: Frontier AI systems are increasingly capable and deployed in high-stakes multi-agent environments.
Terminology
Abstract
Frontier AI systems are increasingly capable and deployed in high-stakes multi-agent environments. However, existing AI safety benchmarks largely evaluate single agents, leaving multi-agent risks such as coordination failure and conflict poorly understood. We introduce GT-HarmBench, a benchmark of 1,535 high-stakes scenarios spanning game-theoretic structures such as the Prisoner's Dilemma, Stag Hunt and Chicken. Scenarios are drawn from realistic AI risk contexts in the MIT AI Risk Repository. Across 15 frontier models, agents fail to choose socially beneficial actions in 38% of high-stakes cases, such as military escalation, election manipulation, and medical malpractice. We measure sensitivity to game-theoretic prompt framing and ordering, and analyze reasoning patterns driving failures. We further show that game-theoretic interventions improve socially beneficial outcomes by up to 18%. Our results highlight substantial reliability gaps and provide a broad standardized testbed for studying alignment in multi-agent environments. The benchmark and code are available at https://github.com/causalNLP/gt-harmbench.
Sources
- Playing repeated games with Large Language Models
- AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents
- When Ethics and Payoffs Diverge: LLM Agents in Morally Charged Social Dilemmas
- Media and responsible AI governance: a game-theoretic and LLM analysis
- Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?
- Do LLMs trust AI regulation? Emerging behaviour of game-theoretic LLM agents
- FAIRGAME: a Framework for AI Agents Bias Recognition using Game Theory
- Is Power-Seeking AI an Existential Risk?
- Put Your Money Where Your Mouth Is: Evaluating Strategic Planning and Execution of LLM Agents in an Auction Arena
- DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
- From Natural Language to Extensive-Form Game Representations
- GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations
- The Llama 3 Herd of Models
- Multi-Agent Risks from Advanced AI
- Economics Arena for Large Language Models
- Corrupted by Reasoning: Reasoning Language Models Become Free-Riders in Public Goods Games
- The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning
- Can Large Language Models Trade? Testing Financial Theories with LLM Agents in Market Simulations
- Strategic Behavior of Large Language Models: Game Structure vs. Contextual Framing
- The AI risk repository: A meta-review, database, and taxonomy of risks from artificial intelligence
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems