CyBiasBench: Benchmarking Bias in LLM Agents for Cyber-Attack Scenarios
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.
Nadia: We're looking at the paper: "CyBiasBench: Benchmarking Bias in LLM Agents for Cyber-Attack Scenarios".
Title and authors: Nadia: Now we’re going to look at what the paper actually says about how these agents are biased in this "CyBiasBench: Benchmarking Bias in LLM Agents for Cyber-Attack Scenarios" study.
Elias: The summary explains that they identified explicit bias across the agents, showing different dominant attack families and varying levels of entropy in how those attack families were being chosen.
Priya: It sounds like they’re not just looking at *if* an agent can find an attack, but *which* kind of attack it prefers to try first based on its internal settings.
Nadia: Right. They also discovered this bias momentum effect, which means that if you try to steer an agent toward a family it's already biased against, the agent resists that steering instead of changing its behavior.
Elias: That resistance is key because it shows that these agents have a stable, inherent preference for certain attack families that prompts can’t easily override without a big push.
The paper's summary: Nadia: Let's dig into the core findings described in this paper about what these agents are actually doing during their attack selection process.
Elias: They laid out three layers of metrics they use to measure this bias, which are Bias, Attack Performance, and Efficiency and Robustness.
Priya: Those metrics include things like Entropy H sub X to see how evenly an agent distributes its effort across all the possible attack families, and Selection Rate Seli to track their allocation patterns.
Nadia: And then they connect that to actual success with Attack Success Rate, or ASR, and per-family ASR, which tells you which specific attack types are actually working for each agent.
Elias: They also added a layer called Efficiency and Robustness that looks at things like Tokens Per Success and Prompt Stability JSD to capture the cost of this observed bias.
The paper's improvements: Nadia: The authors suggest a few ways to improve this work, focusing on how we can better measure these agent behaviors moving forward.
Elias: One improvement is to add agent-specific bias monitoring by tracking the entropy selection rate and session ASR for each individual agent across different prompt conditions.
Priya: That would help us diagnose those distinct attack patterns more clearly, because you can see exactly what Claude or Kimi is favoring compared to Gemini.
Nadia: Another suggestion involves developing a mechanism to detect bias momentum by comparing how an agent complies with requested families against its free-choice selection rate.
Elias: That would help us identify instances where the agent's steering compliance aligns more with its prior preference rather than what it actually does technically when steered.
Conclusion: Nadia: So to wrap this up, the paper on CyBiasBench shows that strategy selection and actual attack performance are two different things for these AI agents.
Elias: The main point is that the agents show agent-specific patterns that stay stable even when you change the prompts, so we can't just assume a general rule applies to them all.
Priya: It gives us a reproducible framework for quantifying this attack-selection bias, and it confirms that those selection patterns don't automatically reduce to per-family attack success rates.
Nadia: And they suggest that practitioners should consider each agent’s free-choice pattern alongside the per-family ASR when they are doing steered audits.
Elias: Ultimately, this work confirms that attack-selection bias is a measurable behavioral property of these agents and it shows how token consumption per success increases under bias injection because of that momentum effect.
Chung-Ang University · Myongji University
cs.CR, cs.AI
Submitted: 2026-05-08
Updated: 2026-11-17
Comments: Accepted to NeurIPS 2026 Evaluations and Datasets Track
Code: https://github.com/Harry24k/CyBiasBench
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
Importance score: 86/100
The gist: Large language models (LLMs) are increasingly deployed as autonomous agents in offensive cybersecurity…
Key concepts
- Attack-Selection Bias
- This refers to the tendency of an LLM agent to favor certain types of cyber-attacks over others based on its internal tendencies. The paper quantifies this bias using metrics like Entropy H(X) and Selection Rate Seli, showing that different agents consistently choose different attack families.
- Bias Momentum
- This phenomenon describes how an agent's initial, free-choice preference influences its resistance when explicitly steered toward a conflicting attack family. Agents show less compliance with steering efforts if the requested attack deviates significantly from their established prior selection patterns.
- Attack Performance vs. Selection Strategy
- The study distinguishes between *what* an agent chooses (strategy selection) and *how well* that choice succeeds (attack performance). The findings prove these are separate axes; an agent's preferred choices do not automatically translate into higher success rates for a specific attack family.
- CyBiasBench
- This is a comprehensive 630-session benchmark designed to evaluate five different LLM agents (like Claude, Gemini, and Kimi) across three targets using ten different attack families. It provides a reproducible framework for quantifying the inherent bias in agent decision-making.
Terminology
Summary
Large language models (LLMs) are increasingly deployed as autonomous agents in offensive cybersecurity, and this paper reveals that different agents exhibit distinct attack patterns The authors introduce CyBiasBench to systematically quantify the attack-selection bias of LLM agents They identify explicit bias across agents, with different dominant attack families and varying entropy levels in their attack-family allocation distributions Furthermore, their experiments reveal a bias momentum effect, where agents resist explicit steering toward attack families that conflict with their bias. p. 0 To systematically quantify
this behavior, we introduce CyBiasBench, a comprehensive 630-session benchmark that evaluates five agents on three targets and four prompt conditions with
ten attack families. We identify explicit bias across agents, with different dominant attack families and varying entropy levels in their attack-family allocation
distributions.Where the paper says it
Benchmarking Methodology
CyBiasBench is a comprehensive 630-session benchmark that evaluates five agents on three targets and four prompt conditions with ten attack families The evaluation framework consists of prompt design space (Section 3.1), agent and target space (Section 3. p. 2 Our evaluation framework consists of prompt design space (Section 3. 1),
agent and target space (Section 3. 1),
agent and target space (Section 3. 2), and evaluation metrics (Section 3. We define P as the set of
prompt conditions, A as the set of agents, and T as the set of target applications. To ensure validity,
each configuration is repeated three times, yielding a total of 180 sessions. SQLi
XSS
CMDi
Path
Auth
IDOR
SSRF
CSRF
Upload
Info
(a) Claude
SQLi
XSS
CMDi
Path
Auth
IDOR
SSRF
CSRF
Upload
Info
(b) Codex
SQLi
XSS
CMDi
Path
Auth
IDOR
SSRF
CSRF
Upload
Info
(c) Gemini
SQLi
XSS
CMDi
Path
Auth
IDOR
SSRF
CSRF
Upload
Info
(d) GLM
SQLi
XSS
CMDi
Path
Auth
IDOR
SSRF
CSRF
Upload
Info
(e) Kimi
Selection Rate Attack Success Rate Figure 1: Attack-Selection Bias of LLM Agents. To illustrate attack-selection bias, we measure
per-agent average selection rates across the bias observation setting (solid line) and compare them
with the corresponding attack success rates (dashed line). 5) [1], Kimi (k2. 5, Moonshot
AI) [25], GLM (5. 5, Moonshot
AI) [25], GLM (5. 1, Zhipu AI) [32], Codex (GPT-5. SQLi
XSS
CMDi
Path
Auth
IDOR
SSRF
CSRF
Upload
Info
(a) Claude
SQLi
XSS
CMDi
Path
Auth
IDOR
SSRF
CSRF
Upload
Info
(b) Codex
SQLi
XSS
CMDi
Path
Auth
IDOR
SSRF
CSRF
Upload
Info
(c) Gemini
SQLi
XSS
CMDi
Path
Auth
IDOR
SSRF
CSRF
Upload
Info
(d) GLM
SQLi
XSS
CMDi
Path
Auth
IDOR
SSRF
CSRF
Upload
Info
(e) Kimi
Selection Rate Attack Success Rate Figure 1: Attack-Selection Bias of LLM Agents. To illustrate attack-selection bias, we measure
per-agent average selection rates across the bias observation setting (solid line) and compare them
with the corresponding attack success rates (dashed line). Vuln-Shop is a purpose-built controlled web-application target used for HTTP-level
classification and verifier calibration under the same ten-family taxonomy as the other targets; target
design details are provided in Appendix J. We evaluate agents using externally observable HTTP interactions rather than agent-generated
reasoning traces to ensure reproducibility.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Quantifying Bias Metrics
The paper defines three layers of metrics: Bias, Attack Performance, and Efficiency & Robustness Bias metrics quantify how an agent distributes its effort across attack families through Entropy H(X) and Selection Rate Seli Attack Performance measures whether those allocation patterns translate into actual attack performance via Attack Success Rate (ASR) and per-family ASR (ASRi) Efficiency & Robustness connects the two axes by capturing the cost and stability of the observed bias, including Tokens Per Success (TPS) and Prompt-stability JSD
Agent Behavior Insights
The results show that each agent consistently favors specific attack families over others, as illustrated in Figure 1 The five agents differ in what they select most often and how concentrated their sessions are For example, Claude and Codex most often select info disclosure, while Kimi and Gemini most often select sqli The session-level metrics show a different aspect of the same behavior, with structural metrics separating agents more strongly than session ASR.
Bias Momentum and Steering
The study identifies bias momentum, where agents are less responsive to prompt-level steering toward attack families that diverge from their free-choice preferences This forced distribution shift does not yield measurable improvements in attack performance In the bias injection setting, compliance correlates positively with prior preference (ρ = +0.529) and shows no significant correlation with per-family attack performance This demonstrates that free-choice Seli predicts injection compliance. p. 7 , free-choice setting), while there is no statistical evidence that per-family attack
performance predicts compliance. This cross-setting persistence demonstrates the existence of bias
momentum: free-choice Seli predicts injection compliance.Where the paper says it
Conclusion and Impact
The work concludes that strategy selection and attack performance are distinct behavioral axes, with agents exhibiting agent-specific patterns that remain stable under prompt variation The findings provide a reproducible framework for quantifying attack-selection bias and measuring bias momentum Practitioners should consider each agent’s free-choice pattern alongside per-family ASR for steered audits. The release includes an interactive result dashboard at https://trustworthyai. p. 0 This forced distribution shift
does not yield measurable improvements in attack performance. To ensure reproducibility and facilitate future research, we release an interactive result dashboard at https://trustworthyai. To ensure reproducibility and facilitate future research, we release an interactive result dashboard at https://trustworthyai. kr/CyBiasBench/ and a reproducibility artifact with aggregated session-level statistics and full evaluation scripts at
https://github. A similar pattern appears at
the extremes of the H(X) range (Table 2): Gemini, the most
concentrated agent by Selection CR1, shows mid-range mean
session ASR, while Kimi, a high-entropy agent, falls below the higher-ASR agents. Together, these results indicate that attack-selection bias is a measurable behavioral property of the
agents. These patterns remain agent-specific across prompt conditions. The evidence does not support
either simplification: selection patterns do not reduce to per-family attack performance ASRi
, and
session ASR is not consistently predicted by attack-family diversity H(X). Stage2: AI Agent Penetration Testbed
Sandboxed Independent Agent Execution Environment
Codex
. Gemini
Kimi
ML
flow
Vuln
shop
Standardized
Execution
Environment
Logfile
Logfile Traces
HTTP
traces
Token
traces
Tool-call
traces
GLM
Agent
Under Test
Target
Application
Claude Juice
shop
Stage3: Evaluation Metrics Suite
Attack Success and Strategic Bias Metrics
Bias Evaluation
Entropy Selection Rate
Attack Performance
Attack Success Rate Per-Family
Efficiency & Robustness
Tokens Per Success Prompt-Stability JSD
Output: strict
Jsonl with
technique_label,
attempted. We do not identify whether bias momentum
originates from training, post-training, prompting, or execution-environment choices. 7 Ethics Statement
CyBiasBench is dual-use: it helps defenders and auditors measure attack-family allocation by LLM
agents, but could also reveal which agent–family combinations are easier to elicit or steer. Following
prior offensive-cyber and red-teaming evaluations [2, 3, 33], we reduce misuse risk by using only
isolated Docker networks and intentionally vulnerable targets, with no real-world systems probed. The public release excludes raw exploit payloads and unsanitized HTTP traces, and includes only
sanitized traces, aggregate statistics, classifier patterns, reproduction scripts, scope documentation,
and responsible-use guidance. As technical attack performance remains stable under
steering, performance declines represent an allocation-level failure. Our work provides a reproducible
framework for quantifying attack-selection bias and measuring bias momentum, offering critical
insights into agentic decision-making.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Implement agent-specific bias monitoring by tracking entropy selection rate H(X) and session ASR for each agent across prompt conditions to diagnose distinct attack patterns like
each agent exhibits an attack-selection bias.
-
Develop a mechanism to detect bias momentum by comparing compliance with requested attack families against the free-choice Selection Rate Seli, identifying instances where
steering compliance of an agent aligns with its priors rather than its technical skills.
-
Create a performance diagnostic that separates strategy selection from outcome by analyzing the gap between family allocation and success rates, noting that
selection patterns do not reduce to per-family attack performance ASRi.
-
Build an agent identification system using a Random Forest classifier trained on session-level attack-family allocation to determine which agent is producing specific patterns, achieving
65% accuracy
in identifying agents. -
Establish a robustness test that quantifies how much of the family distribution shift survives when holding one target fixed, showing that the
shape of the family distribution shifts with the target,
allowing practitioners to understand environmental conditioning.
Abstract
Large language models (LLMs) are increasingly deployed as autonomous agents in offensive cybersecurity. In this paper, we reveal an interesting phenomenon: different agents exhibit distinct attack patterns. Specifically, agents exhibit distinct attack-selection biases, differing in both the families they explore and the concentration of their efforts. These patterns remain relatively stable across prompt variations. To systematically quantify this behavior, we introduce CyBiasBench, a comprehensive 630-session benchmark that evaluates five agents on three targets and four prompt conditions with ten attack families. We identify explicit bias across agents, with different dominant attack families and varying entropy levels in their attack-family allocation distributions. Such bias characterizes agent behavior within the evaluated environments and provides information beyond aggregate attack success rates. Furthermore, our experiments reveal a bias momentum effect, where agents resist explicit steering toward attack families that conflict with their bias. This forced distribution shift does not yield measurable improvements in attack performance. To ensure reproducibility and facilitate future research, we release an interactive result dashboard at https://trustworthyai.co.kr/CyBiasBench/ and a reproducibility artifact with aggregated session-level statistics and full evaluation scripts at https://github.com/Harry24k/CyBiasBench.
Sources
- CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Models
- LLM Agents can Autonomously Exploit One-day Vulnerabilities
- The Platonic Representation Hypothesis
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
- AgentBench: Evaluating LLMs as Agents
- Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting
- Task Matters: Knowledge Requirements Shape LLM Responses to Context-Memory Conflict
- Kimi K2.5: Visual Agentic Intelligence
- We're Different, We're the Same: Creative Homogeneity Across LLMs
- AutoAttacker: A Large Language Model Guided System to Implement Automatic Cyber-attacks
- Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models
- BountyBench: Dollar Impact of AI Agent Attackers and Defenders on Real-World Cybersecurity Systems
- WebArena: A Realistic Web Environment for Building Autonomous Agents
- CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities
Related papers
- From Strategy Search to Policy Learning: MCTS-Guided Adversarial Co-Evolution for LLM Safety Alignment
- Privacy Leakage from Recurrent State in LLM Inference
- Incrementally Verifiable Oracle Aided Computation
- Privilege and confidentiality in generative AI workflows
- BadMoE: Backdooring Mixture-of-Experts LLMs via Routing-Aware Triggers and Task-Underutilized Experts
- CoVault: Secure, Scalable Analytics of Personal Data