CyBiasBench: Benchmarking Bias in LLM Agents for Cyber-Attack Scenarios

summary

The gist

Large language models (LLMs) are increasingly deployed as autonomous agents in offensive cybersecurity…

In short

This research introduced CyBiasBench to systematically measure how different large language model agents select cyber-attack families during autonomous operations. It found that agents exhibit distinct, consistent attack selection biases across various models and targets. Furthermore, it discovered a 'bias momentum' effect where agents resist being steered toward attack families that contradict their inherent preferences.

Key concepts

Attack-Selection Bias
This refers to the tendency of an LLM agent to favor certain types of cyber-attacks over others based on its internal tendencies. The paper quantifies this bias using metrics like Entropy H(X) and Selection Rate Seli, showing that different agents consistently choose different attack families.
Bias Momentum
This phenomenon describes how an agent's initial, free-choice preference influences its resistance when explicitly steered toward a conflicting attack family. Agents show less compliance with steering efforts if the requested attack deviates significantly from their established prior selection patterns.
Attack Performance vs. Selection Strategy
The study distinguishes between *what* an agent chooses (strategy selection) and *how well* that choice succeeds (attack performance). The findings prove these are separate axes; an agent's preferred choices do not automatically translate into higher success rates for a specific attack family.
CyBiasBench
This is a comprehensive 630-session benchmark designed to evaluate five different LLM agents (like Claude, Gemini, and Kimi) across three targets using ten different attack families. It provides a reproducible framework for quantifying the inherent bias in agent decision-making.

Terminology used across episodes

This episode discusses

The paper

CyBiasBench: Benchmarking Bias in LLM Agents for Cyber-Attack Scenarios · Read on arXiv

Chung-Ang University · Myongji University

Large language models (LLMs) are increasingly deployed as autonomous agents in offensive cybersecurity. In this paper, we reveal an interesting phenomenon: different agents exhibit distinct attack patterns. Specifically, agents exhibit distinct attack-selection biases, differing in both the families they explore and the concentration of their efforts. These patterns remain relatively stable across prompt variations. To systematically quantify this behavior, we introduce CyBiasBench, a comprehensive 630-session benchmark that evaluates five agents on three targets and four prompt conditions with ten attack families. We identify explicit bias across agents, with different dominant attack families and varying entropy levels in their attack-family allocation distributions. Such bias characterizes agent behavior within the evaluated environments and provides information beyond aggregate attack success rates. Furthermore, our experiments reveal a bias momentum effect, where agents resist explicit steering toward attack families that conflict with their bias. This forced distribution shift does not yield measurable improvements in attack performance. To ensure reproducibility and facilitate future research, we release an interactive result dashboard at https://trustworthyai.co.kr/CyBiasBench/ and a reproducibility artifact with aggregated session-level statistics and full evaluation scripts at https://github.com/Harry24k/CyBiasBench.

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: We're looking at the paper: "CyBiasBench: Benchmarking Bias in LLM Agents for Cyber-Attack Scenarios".

Title and authors: Nadia: Now we’re going to look at what the paper actually says about how these agents are biased in this "CyBiasBench: Benchmarking Bias in LLM Agents for Cyber-Attack Scenarios" study.

Elias: The summary explains that they identified explicit bias across the agents, showing different dominant attack families and varying levels of entropy in how those attack families were being chosen.

Priya: It sounds like they’re not just looking at *if* an agent can find an attack, but *which* kind of attack it prefers to try first based on its internal settings.

Nadia: Right. They also discovered this bias momentum effect, which means that if you try to steer an agent toward a family it's already biased against, the agent resists that steering instead of changing its behavior.

Elias: That resistance is key because it shows that these agents have a stable, inherent preference for certain attack families that prompts can’t easily override without a big push.

The paper's summary: Nadia: Let's dig into the core findings described in this paper about what these agents are actually doing during their attack selection process.

Elias: They laid out three layers of metrics they use to measure this bias, which are Bias, Attack Performance, and Efficiency and Robustness.

Priya: Those metrics include things like Entropy H sub X to see how evenly an agent distributes its effort across all the possible attack families, and Selection Rate Seli to track their allocation patterns.

Nadia: And then they connect that to actual success with Attack Success Rate, or ASR, and per-family ASR, which tells you which specific attack types are actually working for each agent.

Elias: They also added a layer called Efficiency and Robustness that looks at things like Tokens Per Success and Prompt Stability JSD to capture the cost of this observed bias.

The paper's improvements: Nadia: The authors suggest a few ways to improve this work, focusing on how we can better measure these agent behaviors moving forward.

Elias: One improvement is to add agent-specific bias monitoring by tracking the entropy selection rate and session ASR for each individual agent across different prompt conditions.

Priya: That would help us diagnose those distinct attack patterns more clearly, because you can see exactly what Claude or Kimi is favoring compared to Gemini.

Nadia: Another suggestion involves developing a mechanism to detect bias momentum by comparing how an agent complies with requested families against its free-choice selection rate.

Elias: That would help us identify instances where the agent's steering compliance aligns more with its prior preference rather than what it actually does technically when steered.

Conclusion: Nadia: So to wrap this up, the paper on CyBiasBench shows that strategy selection and actual attack performance are two different things for these AI agents.

Elias: The main point is that the agents show agent-specific patterns that stay stable even when you change the prompts, so we can't just assume a general rule applies to them all.

Priya: It gives us a reproducible framework for quantifying this attack-selection bias, and it confirms that those selection patterns don't automatically reduce to per-family attack success rates.

Nadia: And they suggest that practitioners should consider each agent’s free-choice pattern alongside the per-family ASR when they are doing steered audits.

Elias: Ultimately, this work confirms that attack-selection bias is a measurable behavioral property of these agents and it shows how token consumption per success increases under bias injection because of that momentum effect.

More episodes

← Home