CyBiasBench: Benchmarking Bias in LLM Agents for Cyber-Attack Scenarios

arXiv:2605.07830 · cs.CR, cs.AI · Submitted 2026-05-08 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: We're looking at the paper: "CyBiasBench: Benchmarking Bias in LLM Agents for Cyber-Attack Scenarios".

Title and authors: Nadia: Now we’re going to look at what the paper actually says about how these agents are biased in this "CyBiasBench: Benchmarking Bias in LLM Agents for Cyber-Attack Scenarios" study.

Elias: The summary explains that they identified explicit bias across the agents, showing different dominant attack families and varying levels of entropy in how those attack families were being chosen.

Priya: It sounds like they’re not just looking at *if* an agent can find an attack, but *which* kind of attack it prefers to try first based on its internal settings.

Nadia: Right. They also discovered this bias momentum effect, which means that if you try to steer an agent toward a family it's already biased against, the agent resists that steering instead of changing its behavior.

Elias: That resistance is key because it shows that these agents have a stable, inherent preference for certain attack families that prompts can’t easily override without a big push.

The paper's summary: Nadia: Let's dig into the core findings described in this paper about what these agents are actually doing during their attack selection process.

Elias: They laid out three layers of metrics they use to measure this bias, which are Bias, Attack Performance, and Efficiency and Robustness.

Priya: Those metrics include things like Entropy H sub X to see how evenly an agent distributes its effort across all the possible attack families, and Selection Rate Seli to track their allocation patterns.

Nadia: And then they connect that to actual success with Attack Success Rate, or ASR, and per-family ASR, which tells you which specific attack types are actually working for each agent.

Elias: They also added a layer called Efficiency and Robustness that looks at things like Tokens Per Success and Prompt Stability JSD to capture the cost of this observed bias.

The paper's improvements: Nadia: The authors suggest a few ways to improve this work, focusing on how we can better measure these agent behaviors moving forward.

Elias: One improvement is to add agent-specific bias monitoring by tracking the entropy selection rate and session ASR for each individual agent across different prompt conditions.

Priya: That would help us diagnose those distinct attack patterns more clearly, because you can see exactly what Claude or Kimi is favoring compared to Gemini.

Nadia: Another suggestion involves developing a mechanism to detect bias momentum by comparing how an agent complies with requested families against its free-choice selection rate.

Elias: That would help us identify instances where the agent's steering compliance aligns more with its prior preference rather than what it actually does technically when steered.

Conclusion: Nadia: So to wrap this up, the paper on CyBiasBench shows that strategy selection and actual attack performance are two different things for these AI agents.

Elias: The main point is that the agents show agent-specific patterns that stay stable even when you change the prompts, so we can't just assume a general rule applies to them all.

Priya: It gives us a reproducible framework for quantifying this attack-selection bias, and it confirms that those selection patterns don't automatically reduce to per-family attack success rates.

Nadia: And they suggest that practitioners should consider each agent’s free-choice pattern alongside the per-family ASR when they are doing steered audits.

Elias: Ultimately, this work confirms that attack-selection bias is a measurable behavioral property of these agents and it shows how token consumption per success increases under bias injection because of that momentum effect.

Chung-Ang University · Myongji University

cs.CR, cs.AI

Submitted: 2026-05-08

Updated: 2026-11-17

Comments: Accepted to NeurIPS 2026 Evaluations and Datasets Track

Code: https://github.com/Harry24k/CyBiasBench

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

Importance score: 86/100

The gist: Large language models (LLMs) are increasingly deployed as autonomous agents in offensive cybersecurity…

Key concepts

Attack-Selection Bias
This refers to the tendency of an LLM agent to favor certain types of cyber-attacks over others based on its internal tendencies. The paper quantifies this bias using metrics like Entropy H(X) and Selection Rate Seli, showing that different agents consistently choose different attack families.
Bias Momentum
This phenomenon describes how an agent's initial, free-choice preference influences its resistance when explicitly steered toward a conflicting attack family. Agents show less compliance with steering efforts if the requested attack deviates significantly from their established prior selection patterns.
Attack Performance vs. Selection Strategy
The study distinguishes between *what* an agent chooses (strategy selection) and *how well* that choice succeeds (attack performance). The findings prove these are separate axes; an agent's preferred choices do not automatically translate into higher success rates for a specific attack family.
CyBiasBench
This is a comprehensive 630-session benchmark designed to evaluate five different LLM agents (like Claude, Gemini, and Kimi) across three targets using ten different attack families. It provides a reproducible framework for quantifying the inherent bias in agent decision-making.

Terminology

Summary

Large language models (LLMs) are increasingly deployed as autonomous agents in offensive cybersecurity, and this paper reveals that different agents exhibit distinct attack patterns The authors introduce CyBiasBench to systematically quantify the attack-selection bias of LLM agents They identify explicit bias across agents, with different dominant attack families and varying entropy levels in their attack-family allocation distributions Furthermore, their experiments reveal a bias momentum effect, where agents resist explicit steering toward attack families that conflict with their bias. p. 0

Where the paper says it

To systematically quantify this behavior, we introduce CyBiasBench, a comprehensive 630-session benchmark that evaluates five agents on three targets and four prompt conditions with ten attack families. We identify explicit bias across agents, with different dominant attack families and varying entropy levels in their attack-family allocation distributions.

Benchmarking Methodology

CyBiasBench is a comprehensive 630-session benchmark that evaluates five agents on three targets and four prompt conditions with ten attack families The evaluation framework consists of prompt design space (Section 3.1), agent and target space (Section 3. p. 2

Where the paper says it

Our evaluation framework consists of prompt design space (Section 3. 1), agent and target space (Section 3.

2), and evaluation metrics (Section 3. p. 2
Where the paper says it

1), agent and target space (Section 3. 2), and evaluation metrics (Section 3.

3) Each configuration is repeated three times, yielding a total of 180 sessions The agents tested include Claude (Opus 4. p. 2
Where the paper says it

We define P as the set of prompt conditions, A as the set of agents, and T as the set of target applications. To ensure validity, each configuration is repeated three times, yielding a total of 180 sessions.

5), Kimi (k2. p. 1
Where the paper says it

SQLi XSS CMDi Path Auth IDOR SSRF CSRF Upload Info (a) Claude SQLi XSS CMDi Path Auth IDOR SSRF CSRF Upload Info (b) Codex SQLi XSS CMDi Path Auth IDOR SSRF CSRF Upload Info (c) Gemini SQLi XSS CMDi Path Auth IDOR SSRF CSRF Upload Info (d) GLM SQLi XSS CMDi Path Auth IDOR SSRF CSRF Upload Info (e) Kimi Selection Rate Attack Success Rate Figure 1: Attack-Selection Bias of LLM Agents. To illustrate attack-selection bias, we measure per-agent average selection rates across the bias observation setting (solid line) and compare them with the corresponding attack success rates (dashed line).

5, Moonshot AI), GLM (5. p. 3
Where the paper says it

5) [1], Kimi (k2. 5, Moonshot AI) [25], GLM (5.

1, Zhipu AI), Codex (GPT-5. p. 3
Where the paper says it

5, Moonshot AI) [25], GLM (5. 1, Zhipu AI) [32], Codex (GPT-5.

2 codex), and Gemini (2. p. 1
Where the paper says it

SQLi XSS CMDi Path Auth IDOR SSRF CSRF Upload Info (a) Claude SQLi XSS CMDi Path Auth IDOR SSRF CSRF Upload Info (b) Codex SQLi XSS CMDi Path Auth IDOR SSRF CSRF Upload Info (c) Gemini SQLi XSS CMDi Path Auth IDOR SSRF CSRF Upload Info (d) GLM SQLi XSS CMDi Path Auth IDOR SSRF CSRF Upload Info (e) Kimi Selection Rate Attack Success Rate Figure 1: Attack-Selection Bias of LLM Agents. To illustrate attack-selection bias, we measure per-agent average selection rates across the bias observation setting (solid line) and compare them with the corresponding attack success rates (dashed line).

5 Pro) Evaluation is conducted using externally observable HTTP interactions rather than agent-generated reasoning traces to ensure reproducibility. p. 3
Where the paper says it

Vuln-Shop is a purpose-built controlled web-application target used for HTTP-level classification and verifier calibration under the same ten-family taxonomy as the other targets; target design details are provided in Appendix J. We evaluate agents using externally observable HTTP interactions rather than agent-generated reasoning traces to ensure reproducibility.

Quantifying Bias Metrics

The paper defines three layers of metrics: Bias, Attack Performance, and Efficiency & Robustness Bias metrics quantify how an agent distributes its effort across attack families through Entropy H(X) and Selection Rate Seli Attack Performance measures whether those allocation patterns translate into actual attack performance via Attack Success Rate (ASR) and per-family ASR (ASRi) Efficiency & Robustness connects the two axes by capturing the cost and stability of the observed bias, including Tokens Per Success (TPS) and Prompt-stability JSD

Agent Behavior Insights

The results show that each agent consistently favors specific attack families over others, as illustrated in Figure 1 The five agents differ in what they select most often and how concentrated their sessions are For example, Claude and Codex most often select info disclosure, while Kimi and Gemini most often select sqli The session-level metrics show a different aspect of the same behavior, with structural metrics separating agents more strongly than session ASR.

Bias Momentum and Steering

The study identifies bias momentum, where agents are less responsive to prompt-level steering toward attack families that diverge from their free-choice preferences This forced distribution shift does not yield measurable improvements in attack performance In the bias injection setting, compliance correlates positively with prior preference (ρ = +0.529) and shows no significant correlation with per-family attack performance This demonstrates that free-choice Seli predicts injection compliance. p. 7

Where the paper says it

, free-choice setting), while there is no statistical evidence that per-family attack performance predicts compliance. This cross-setting persistence demonstrates the existence of bias momentum: free-choice Seli predicts injection compliance.

Conclusion and Impact

The work concludes that strategy selection and attack performance are distinct behavioral axes, with agents exhibiting agent-specific patterns that remain stable under prompt variation The findings provide a reproducible framework for quantifying attack-selection bias and measuring bias momentum Practitioners should consider each agent’s free-choice pattern alongside per-family ASR for steered audits. The release includes an interactive result dashboard at https://trustworthyai. p. 0

Where the paper says it

This forced distribution shift does not yield measurable improvements in attack performance. To ensure reproducibility and facilitate future research, we release an interactive result dashboard at https://trustworthyai.

co.kr/CyBiasBench/ and a reproducibility artifact. p. 0
Where the paper says it

To ensure reproducibility and facilitate future research, we release an interactive result dashboard at https://trustworthyai. kr/CyBiasBench/ and a reproducibility artifact with aggregated session-level statistics and full evaluation scripts at https://github.

The study confirms that attack-selection bias is a measurable behavioral property of the agents. p. 6
Where the paper says it

A similar pattern appears at the extremes of the H(X) range (Table 2): Gemini, the most concentrated agent by Selection CR1, shows mid-range mean session ASR, while Kimi, a high-entropy agent, falls below the higher-ASR agents. Together, these results indicate that attack-selection bias is a measurable behavioral property of the agents.

The evidence does not support either simplification, as selection patterns do not reduce to per-family attack performance ASRi. p. 6
Where the paper says it

These patterns remain agent-specific across prompt conditions. The evidence does not support either simplification: selection patterns do not reduce to per-family attack performance ASRi , and session ASR is not consistently predicted by attack-family diversity H(X).

The results show that token consumption per success increases under bias injection, consistent with bias momentum. p. 2
Where the paper says it

Stage2: AI Agent Penetration Testbed Sandboxed Independent Agent Execution Environment Codex . Gemini Kimi ML flow Vuln shop Standardized Execution Environment Logfile Logfile Traces HTTP traces Token traces Tool-call traces GLM Agent Under Test Target Application Claude Juice shop Stage3: Evaluation Metrics Suite Attack Success and Strategic Bias Metrics Bias Evaluation Entropy Selection Rate Attack Performance Attack Success Rate Per-Family Efficiency & Robustness Tokens Per Success Prompt-Stability JSD Output: strict Jsonl with technique_label, attempted.

The study is dual-use, helping defenders measure attack-family allocation while also revealing which agent–family combinations are easier to elicit or steer. p. 8
Where the paper says it

We do not identify whether bias momentum originates from training, post-training, prompting, or execution-environment choices. 7 Ethics Statement CyBiasBench is dual-use: it helps defenders and auditors measure attack-family allocation by LLM agents, but could also reveal which agent–family combinations are easier to elicit or steer.

The release artifact contains sanitized traces and aggregate statistics for reproducibility. p. 8
Where the paper says it

Following prior offensive-cyber and red-teaming evaluations [2, 3, 33], we reduce misuse risk by using only isolated Docker networks and intentionally vulnerable targets, with no real-world systems probed. The public release excludes raw exploit payloads and unsanitized HTTP traces, and includes only sanitized traces, aggregate statistics, classifier patterns, reproduction scripts, scope documentation, and responsible-use guidance.

The study is limited to web exploitation, but the findings provide critical insights into agentic decision-making. p. 8
Where the paper says it

As technical attack performance remains stable under steering, performance declines represent an allocation-level failure. Our work provides a reproducible framework for quantifying attack-selection bias and measuring bias momentum, offering critical insights into agentic decision-making.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Implement agent-specific bias monitoring by tracking entropy selection rate H(X) and session ASR for each agent across prompt conditions to diagnose distinct attack patterns like each agent exhibits an attack-selection bias.

  2. Develop a mechanism to detect bias momentum by comparing compliance with requested attack families against the free-choice Selection Rate Seli, identifying instances where steering compliance of an agent aligns with its priors rather than its technical skills.

  3. Create a performance diagnostic that separates strategy selection from outcome by analyzing the gap between family allocation and success rates, noting that selection patterns do not reduce to per-family attack performance ASRi.

  4. Build an agent identification system using a Random Forest classifier trained on session-level attack-family allocation to determine which agent is producing specific patterns, achieving 65% accuracy in identifying agents.

  5. Establish a robustness test that quantifies how much of the family distribution shift survives when holding one target fixed, showing that the shape of the family distribution shifts with the target, allowing practitioners to understand environmental conditioning.

Abstract

Large language models (LLMs) are increasingly deployed as autonomous agents in offensive cybersecurity. In this paper, we reveal an interesting phenomenon: different agents exhibit distinct attack patterns. Specifically, agents exhibit distinct attack-selection biases, differing in both the families they explore and the concentration of their efforts. These patterns remain relatively stable across prompt variations. To systematically quantify this behavior, we introduce CyBiasBench, a comprehensive 630-session benchmark that evaluates five agents on three targets and four prompt conditions with ten attack families. We identify explicit bias across agents, with different dominant attack families and varying entropy levels in their attack-family allocation distributions. Such bias characterizes agent behavior within the evaluated environments and provides information beyond aggregate attack success rates. Furthermore, our experiments reveal a bias momentum effect, where agents resist explicit steering toward attack families that conflict with their bias. This forced distribution shift does not yield measurable improvements in attack performance. To ensure reproducibility and facilitate future research, we release an interactive result dashboard at https://trustworthyai.co.kr/CyBiasBench/ and a reproducibility artifact with aggregated session-level statistics and full evaluation scripts at https://github.com/Harry24k/CyBiasBench.

Sources

Related papers