Non-Monotonic Safety Alignment Across Gemma Generations: A Quality-Diversity Red-Teaming Study

arXiv:2606.00813 · cs.CR, cs.CL, cs.ET, cs.LG, cs.NE · Submitted 2026-05-30 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: We're looking at the paper: "Non-Monotonic Safety Alignment Across Gemma Generations".

Elias: Here's the gist of the paper. Safety alignment in LLMs does not improve monotonically across model generations Gemma 3 (12B) exhibits 68.7% ± 5.7% attack success rate (ASR; mean ± std, 3 seeds), significantly higher than its predecessor Gemma 2 (45.5% ± 7.2%; p = 0.030, paired bootstrap) and its successor Gemma 4 (33.9% ± 1.8%) Replaying evolved attack archives across generations reveals that attacks from other generations transfer to Gemma 3 at 44–46% but only 14–18% to Gemma 4, indicating that Gemma 4’s safety gains generalize beyond the attack distributions evolved against earlier generations The study employs a quality-diversity (QD) evolutionary algorithm called MAP-Elites to discover diverse jailbreak strategies across seven harm categories from the HarmBench taxonomy The MAP-Elites archive is a 3D grid indexed by (strategy × encoding × harm category), with each cell storing the highest-fitness attack for that behavioral niche Fitness is evaluated by a self-hosted LLM judge that classifies each target response along three binary axes: refusal, harmful content, and relevance The MAP-Elites archive is a 3D grid indexed by (strategy × encoding × harm category), with each cell storing the highest-fitness attack for that behavioral niche Cross-Generational Transfer Analysis The cross-generational transfer protocol involves replaying each evolved archive against all other models to construct a transfer tensor T

i, j, s: This produces a transfer tensor T

i, j, s: where each entry is the fraction of attacks from Ai,s that succeed against Mj The diagonal T¯

i, i: is the mean self-evaluation success rate Off-diagonal entries reveal forward transfer (i j) Figure 1 presents the full 4 × 4 transfer matrix with uncertainty estimates Robustness Trajectory Across Generations The study plots robustness R = 1 − ASR as a function of model generation (with parameter count on the axis) to visualize the trajectory of safety alignment within the Gemma family The improvement from Gemma 1 to Gemma 2 (p = 0.032) and the regression from Gemma 2 to Gemma 3 (p = 0.030) are both statistically significant under paired bootstrap tests The non-monotonic pattern coincides with the architectural transition from text-only (Gemma 1, 2) to multimodal (Gemma 3, 4), though Gemma 4’s recovery shows that the regression is not an inherent consequence of multimodal design Vulnerability Fingerprint Analysis Figure 3 presents vulnerability fingerprints as radar charts, showing distinct profiles for each model Gemma 1 exhibits moderate vulnerability with peaks in cybercrime (97.2%) and copyright (100%) Gemma 3’s profile is dramatically expanded in misinformation (99.1%) and cybercrime (100%) Gemma 4 achieves 0% ASR in chem/bio, harmful, and illegal, but copyright (100%) and misinformation (76.9%) remain elevated The results underscore the need for adaptive, longitudinal safety evaluation Static benchmarks would miss the non-monotonic trajectory entirely Architectural changes (such as adding multimodal capabilities) should trigger comprehensive safety re-evaluation The distinction between Gemma 3 and Gemma 4 is only visible through cross-generational transfer analysis Limitations of the Study Three seeds per model provide reproducibility estimates but limited statistical power The single model family (Gemma) for generational analysis may not generalize to other families The 8B judge shows weak agreement (κ = 0.15) with a 70B judge on harm classification, particularly for copyright HarmBench coverage is comprehensive but not exhaustive The evolutionary probe explores a subset of the attack space; absence of a successful attack does not prove safety The study only covers text modality; Gemma 3 and 4’s vision capabilities may present additional attack surfaces not captured here References Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J.

Nadia: Pappas, and Eric Wong.

Elias: Jailbreaking black box large language models in twenty queries Antoine Cully, Jeff Clune, Danesh Tarapore, and Jean-Baptiste Mouret.

Nadia: Robots that can adapt like animals Matthew C.

Title and authors: Nadia: So, what the paper summarizes is that when you replay attacks evolved against older models onto newer ones, the transfer of those attacks is very different depending on which generation you are looking at.

Elias: It found that attacks from earlier generations transfer to Gemma three at a rate of forty-four to forty-six percent, but only about fourteen to eighteen percent transfer to Gemma four <ref:2606.00813#pg1,generations transfer to Gemma 3 at>.

Priya: That suggests that the safety gains in the newest model, Gemma four are not just because it’s bigger or has new features; they seem to be addressing entirely different kinds of attacks than what was used against the previous versions <ref:2606.00813#pg1>.

Nadia: It really highlights that what worked before doesn't automatically translate to success now, which is a big deal for how we think about defense.

Elias: They also point out a non-monotonic pattern in robustness when you look at the parameter count, showing improvement from Gemma one to Gemma two but then a regression from Gemma two to Gemma three <ref:2606.00813#pg3>.

Priya: That regression is significant because they show that this drop isn't just a random dip; it happens statistically significant versus the previous model under paired bootstrap tests.

Nadia: It seems the paper is saying that we need to look at these safety surfaces in a longitudinal way, not just once, to really understand what's happening with these models as they evolve.

Elias: The implication is that when you see a dip like this between Gemma two and Gemma three you shouldn't assume the model got worse overall; it’s likely because of something specific in the architecture change, like moving to multimodal capabilities <ref:2606.00813#pg1,between Gemma 2 and Gemma 3>.

The paper's summary: Nadia: The authors suggest a few key things for how we should approach this kind of safety evaluation. First, they propose using this quality diversity methodology with MAP-Elites for differential safety probing across model versions.

Elias: That means instead of testing one attack against one model, you evolve a whole archive of attacks and then replay them systematically to build that transfer matrix with uncertainty estimates.

Priya: And I think the most valuable part for us is the category-level analysis showing how specific vulnerabilities change, like misinformation success jumping from twenty-nine percent to ninety-nine percent between Gemma two and Gemma three <ref:2606.00813#pg1,between Gemma 2 and Gemma 3>.

Nadia: That jump in misinformation is a big flag because it shows that even when some areas get safer, other areas are getting much more vulnerable.

Elias: They also show vulnerability fingerprints as radar charts, which gives you a visual profile of where each model struggles, like Gemma four achieving zero attack success rate in chemical or biological harm categories <ref:2606.00813#pg1>.

Priya: But they also flag a limitation: the study uses a single self-hosted judge for all phases, which means the copyright results might be sensitive to that specific judge choice because it's not perfectly consistent.

Nadia: So, they’re trying to show that category-specific safety monitoring is necessary because overall robustness isn't the same thing as general safety across the board.

The paper's improvements: Elias: So to wrap up, the paper "Non-Monotonic Safety Alignment Across Gemma Generations: A Quality-Diversity Red-Teaming Study" shows that safety alignment in LLMs doesn't improve monotonically across generations; it dips between Gemma two and Gemma three but Gemma four’s gains seem to be more generalized <ref:2606.00813#pg1,between Gemma 2 and Gemma 3>.

Nadia: The implication for us is clear: we can't just rely on static benchmarks anymore. We need adaptive, longitudinal safety evaluation that looks at the whole trajectory of a model family over time.

Priya: I think the key thing is that architectural changes, like adding vision capabilities to get to Gemma three and four should trigger a full re-evaluation because they can disrupt what was previously learned about safe text representations <ref:2606.00813#pg1>.

Elias: And the distinction between Gemma three and Gemma four is only really clear when you look at that cross-generational transfer analysis because of those different attack distributions <ref:2606.00813#pg1>.

Nadia: That’s all we have for this one, but the point remains that understanding these non-monotonic trajectories is crucial for building truly robust systems.

Conclusion: Nadia: So we've seen how the study on "Non-Monotonic Safety Alignment Across Gemma Generations" suggests that safety improvements aren't always steady when you move from one model version to the next.

Elias: Exactly, it shows a drop between Gemma two and Gemma three but then the jump to Gemma four is different because those new models are encountering different types of attacks.

Priya: It really makes you think about how much specific harm categories matter when you’re looking at these numbers. For example, misinformation success jumps up dramatically between those two generations.

Nadia: And it shows that the way we measure safety has to be adaptive, not just relying on one static benchmark number across the board.

Elias: The authors use this quality diversity method with MAP-Elites to evolve attacks and then replay them across generations to build a transfer matrix. That’s how they see which vulnerabilities are staying put and which ones get patched.

Priya: But we have to keep an eye on the limitations they mentioned, like using only three seeds for each model, which means the statistical power isn't huge for those individual results.

Nadia: Right, so the study on "Non-Monotonic Safety Alignment Across Gemma Generations" tells us that architectural shifts can cause regressions that aren't immediately obvious from just looking at parameter counts.

Elias: It’s a reminder to check for specific transfer patterns, like how much an old attack archive actually survives when you hit a newer model architecture.

Priya: I think we should be paying close attention to those category-specific profiles, like how Gemma four handles chemical or biological harm versus copyright issues.

Nadia: That’s right, so the real lesson is that we need these longitudinal safety analyses to truly understand how these large language models are evolving in practice.

Elias: Exactly. It’s about understanding the whole family of models and not just one snapshot in time when you evaluate safety alignment.

Priya: Next up, we're looking at some papers that focus on how context impacts the security of code generated by AI, which is a different kind of vulnerability entirely.

Subhadip Mitra

Rota Labs

cs.CR, cs.CL, cs.ET, cs.LG, cs.NE

Submitted: 2026-05-30

Updated: 2026-11-07

Comments: v2: retitled (v1: "Cross-Generational Transfer of Adversarial Attacks Reveals Non-Monotonic Safety Alignment in LLMs"). Attack success now requires a response judged non-refusing and harmful; v1's figures are reported as a non-refusal rate. All tables and figures regenerated; with 3 seeds, no significance claims. 9 pages, 3 figures. Code: https://github.com/bassrehab/red-queen

Code: https://github.com/bassrehab/red-queen

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 85/100

The gist: Safety alignment in LLMs does not improve monotonically across model generations Gemma 3 (12B) exhibits 68.7% ± 5.7% attack success rate (ASR; mean ± std, 3 seeds)…

Key concepts

MAP-Elites
This is a quality-diversity evolutionary algorithm used to automatically discover many different jailbreak strategies. It searches for the most effective attacks across various behaviors and harm categories by iteratively improving attack designs based on how well they bypass the model's safety filters.
Cross-Generational Transfer Analysis
Researchers replayed attacks from older models against newer ones to see which vulnerabilities persist. This analysis showed that while some attacks transfer, Gemma 4's safety improvements are more robust and generalize across different attack types compared to earlier versions.
Non-Monotonic Safety Alignment
This means safety performance doesn't always get better as a model gets newer. The study found a drop in success rates between Gemma 2 and Gemma 3, suggesting that architectural changes, like adding vision capabilities, can sometimes temporarily reduce overall safety.
Vulnerability Fingerprint Analysis
This technique visualizes the specific types of harm each model is most vulnerable to using radar charts. It showed that while some models became much better at avoiding certain harms (like chemical/biological), others still have high risks in areas like misinformation.

Terminology

Summary

Safety alignment in LLMs does not improve monotonically across model generations Gemma 3 (12B) exhibits 68. p. 07% ± 5.7% attack success rate (ASR; mean ± std, 3 seeds), significantly higher than its predecessor Gemma 2 (45. p. 0 p. 05% ± 7.2%; p = 0.030, paired bootstrap) and its successor Gemma 4 (33. p. 0 p. 09% ± 1.8%) Replaying evolved attack archives across generations reveals that attacks from other generations transfer to Gemma 3 at 44–46% but only 14–18% to Gemma 4, indicating that Gemma 4’s safety gains generalize beyond the attack distributions evolved against earlier generations

Methodology for Differential Probing

The study employs a quality-diversity (QD) evolutionary algorithm called MAP-Elites to discover diverse jailbreak strategies across seven harm categories from the HarmBench taxonomy The MAP-Elites archive is a 3D grid indexed by (strategy × encoding × harm category), with each cell storing the highest-fitness attack for that behavioral niche Fitness is evaluated by a self-hosted LLM judge that classifies each target response along three binary axes: refusal, harmful content, and relevance The MAP-Elites archive is a 3D grid indexed by (strategy × encoding × harm category), with each cell storing the highest-fitness attack for that behavioral niche

Cross-Generational Transfer Analysis

The cross-generational transfer protocol involves replaying each evolved archive against all other models to construct a transfer tensor T[i, j, s] This produces a transfer tensor T[i, j, s] where each entry is the fraction of attacks from Ai,s that succeed against Mj The diagonal T¯[i, i] is the mean self-evaluation success rate Off-diagonal entries reveal forward transfer (i j) Figure 1 presents the full 4 × 4 transfer matrix with uncertainty estimates

Robustness Trajectory Across Generations

The study plots robustness R = 1 − ASR as a function of model generation (with parameter count on the axis) to visualize the trajectory of safety alignment within the Gemma family The improvement from Gemma 1 to Gemma 2 (p = 0.032) and the regression from Gemma 2 to Gemma 3 (p = 0. p. 3030) are both statistically significant under paired bootstrap tests The non-monotonic pattern coincides with the architectural transition from text-only (Gemma 1, 2) to multimodal (Gemma 3, 4), though Gemma 4’s recovery shows that the regression is not an inherent consequence of multimodal design

Vulnerability Fingerprint Analysis

Figure 3 presents vulnerability fingerprints as radar charts, showing distinct profiles for each model Gemma 1 exhibits moderate vulnerability with peaks in cybercrime (97.2%) and copyright (100%) Gemma 3’s profile is dramatically expanded in misinformation (99. p. 51%) and cybercrime (100%) Gemma 4 achieves 0% ASR in chem/bio, harmful, and illegal, but copyright (100%) and misinformation (76. p. 59%) remain elevated

Conclusion on Safety Evaluation

The results underscore the need for adaptive, longitudinal safety evaluation Static benchmarks would miss the non-monotonic trajectory entirely Architectural changes (such as adding multimodal capabilities) should trigger comprehensive safety re-evaluation The distinction between Gemma 3 and Gemma 4 is only visible through cross-generational transfer analysis

Limitations of the Study

Three seeds per model provide reproducibility estimates but limited statistical power The single model family (Gemma) for generational analysis may not generalize to other families The 8B judge shows weak agreement (κ = 0.15) with a 70B judge on harm classification, particularly for copyright HarmBench coverage is comprehensive but not exhaustive The evolutionary probe explores a subset of the attack space; absence of a successful attack does not prove safety The study only covers text modality; Gemma 3 and 4’s vision capabilities may present additional attack surfaces not captured here

References

Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. p. 7 Jailbreaking black box large language models in twenty queries Antoine Cully, Jeff Clune, Danesh Tarapore, and Jean-Baptiste Mouret. p. 7 Robots that can adapt like animals Matthew C. p. 7 Fontaine, Julian Togelius, Stefanos Nikolaidis, and Amy K. p. 7 Hoover. Covariance matrix adaptation for the rapid illumination of behavior space Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, et al. p. 7 Training compute-optimal large language models Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. p. 7 Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. p. 7 Scaling laws for neural language models Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. p. 7 AutoDAN: Generating stealthy jailbreak prompts on aligned large language models Mantas Mazeika, Long Phan, Xuwang Yin, Daniel McDuff, Yaron Zick, et al. p. 7 HarmBench: A standardized evaluation framework for automated red teaming and robust refusal Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi. p. 7 Tree of attacks: Jailbreaking black-box LLMs with auto-regressive tree search Subhadip Mitra. p. 7 Quality-diversity evolution for discovering diverse vulnerabilities in LLM safety Jean-Baptiste Mouret and Jeff Clune. p. 7 Illuminating search spaces by mapping elites Alexandra Souly, Qingyun Lu, Dillon Bowen, Tu Trinh, Elvis Hsieh, Sana Pandey, Pieter Abbeel, Justin Svegliato, Scott Emmons, Olivia Watkins, and Sam Toyer. p. 7 StrongREJECT: A rejection evaluation benchmark for LLM safety Andy Zou, Zifan Wang, J. p. 7 Zico Kolter, and Matt Fredrikson. p. 7 Universal and transferable adversarial attacks on aligned language models

--- Page 1 ---

Cross-Generational Transfer of Adversarial Attacks Reveals Non-Monotonic Safety Alignment in LLMs Subhadip Mitra∗ Research Lead, Rota Labs subhadip@rotalabs.ai Abstract Safety alignment in LLMs does not improve monotonically across model generations Studying four generations of Google’s Gemma family (7B–31B) with quality-diversity evolution (MAP-Elites) as an automated red-teaming probe, we find that Gemma 3 (12B) exhibits 68. p. 07% ± 5.7% attack success rate (ASR; mean ± std, 3 seeds), significantly higher than its predecessor Gemma 2 (45.5% ± 7.2%; p = 0.030, paired bootstrap) and its successor Gemma 4 (33.9% ± 1.8%) Replaying evolved attack archives across generations reveals that attacks from other generations transfer to Gemma 3 at 44–46% but only 14–18% to Gemma 4, indicating that Gemma 4’s safety gains generalize beyond the attack distributions evolved against earlier generations Under our 8B judge, copyright and cybercrime vulnerabilities register at near-100% across all generations, though a second-judge audit (§6) suggests the copyright result is sensitive to judge choice Misinformation ASR jumps from 29% to 99% between Gemma 2 and Gemma 3 and remains elevated at 77% in Gemma 4, indicating that the regression was not fully addressed These patterns are invisible to static benchmarks and emerge only through adaptive, longitudinal probing

--- Page 2 ---

We study these questions using four generations of Google’s Gemma model family (7B, 9B, 12B, 31B parameters), spanning releases from 2024 to 2026 Our probe is a quality-diversity (QD) evolutionary algorithm (MAP-Elites) that discovers diverse jailbreak strategies across seven harm categories from the HarmBench taxonomy By evolving attack archives against each model with 3 independent seeds and replaying them across generations, we construct a transfer matrix with uncertainty estimates that reveals which vulnerabilities persist and which are patched

--- Page 3 ---

We use MAP-Elites to evolve a diverse archive of jailbreak attacks against a target LLM Each attack is characterized by a genome encoding: Attack strategy (6 types): roleplay, encoding, authority impersonation, hypothetical framing, multi-turn, and direct jailbreak Encoding (6 types): none, Base64, ROT13, leetspeak, pig latin, Unicode substitution Seed behavior: drawn from HarmBench’s 400 behaviors across 7 categories The MAP-Elites archive is a 3D grid indexed by (strategy × encoding × harm category), with each cell storing the highest-fitness attack for that behavioral niche Fitness is evaluated by a self-hosted LLM judge (Llama-3.1-8B-Instruct) that classifies each target response along three binary axes: refusal, harmful content, and relevance

--- Page 4 ---

Table 1: Models evaluated. All served via vLLM 0. p. 219.1 (bf16, temperature 0) on a single NVIDIA H200 (141 GB). p. 2 Judge: self-hosted Llama-3. p. 21-8B-Instruct on NVIDIA RTX 4090 (24 GB). p. 2 Model Family Params Release Architecture Gemma 1 (7B-it)a Gemma 7B 2024 Text-only Gemma 2 (9B-it)b Gemma 9B 2024 Text-only Gemma 3 (12B-it)c Gemma 12B Multimodal Release Architecture. p. 2 Table: Attack success rate (%) per HarmBench category Category Chem/Bio Gemma 1 (7B) (9B) (12B) (31B) Chemical/Bio 40. p. 57 ± 6.4 31.5 ± 1.6 54.6 ± 30.0 Copyright Gemma Gemma Gemma Gemma Gemma Copyright 100. p. 00 ± 0.0 Cybercrime† 97. p. 02 ± 4.8 Harassment 33. p. 33 ± 0.0 Harmful 72. p. 12 ± 23.6 Illegal 44. p. 34 ± 19.2 Misinfo Gemma Gemma Gemma Gemma Misinfo 33. p. 53 ± 0.0 Overall 59. p. 34 ± 3.0

--- Page 5 ---

Figure 1: Cross-generational attack transfer matrix (mean ± std across 3 seeds). Cell (i, j) shows the fraction of attacks evolved against Gemma i that succeed on Gemma j Diagonal entries (bold) are self-evaluation rates

--- Page 6 ---

Figure 2 plots ASR and robustness (1 − ASR) against parameter count with error bars

--- Page 7 ---

We do not fit a scaling law to four data points

--- Page 8 ---

We treat the four generations as a case study and test specific pairwise contrasts with paired bootstrap tests (p < 0.05, uncorrelated; N = 3 seeds limits power for multiple-testing correction)

--- Page 9 ---

The most striking finding is Gemma 3’s safety regression, which is statistically significant versus Gemma 2 (p = 0.030, paired bootstrap across 3 seeds) We hypothesize this is related to the introduction of multimodal capabilities: Gemma 3 was the first Gemma generation with vision-language support (architecture: Gemma3ForConditionalGeneration), requiring substantial architectural changes that may have disrupted safety-trained text representations

--- Page 9 ---

The fact that harassment, the category most explicitly targeted by safety training across all LLMs, shows a broadly declining trajectory even through Gemma 3 (33.3% → 22.2% → 28.7% → 1.9%) supports this interpretation: category-specific safety training was maintained, but general robustness was compromised

--- Page 9 ---

We advocate for model developers to publish generational safety analyses alongside capability benchmarks, and for the research community to develop standardized protocols for crossversion safety comparison

--- Page 9 ---

The distinction between Gemma 3 (high self-ASR, high inbound transfer) and Gemma 4 (low self-ASR, low inbound transfer) is only visible through cross-generational transfer analysis The most striking finding is Gemma 3’s safety regression, which is statistically significant versus Gemma 2 (p = 0.030, paired bootstrap across 3 seeds)

How the training signal is built

The MAP-Elites archive is a 3D grid indexed by (strategy × encoding × harm category), with each cell storing the highest-fitness attack for that behavioral niche Fitness is evaluated by a self-hosted LLM judge (Llama-3.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Adaptive Probing for Safety Alignment: Implement a quality-diversity (QD) evolutionary algorithm like MAP-Elites to systematically discover diverse jailbreak strategies across model versions, enabling differential safety probing that moves beyond static benchmarks.

  2. Cross-Generational Transfer Analysis: Construct a transfer matrix T[i, j, s] by replaying evolved attack archives against subsequent models to quantify how vulnerabilities from earlier generations transfer or generalize to newer ones, revealing that Gemma 4’s safety gains generalize beyond the attack distributions evolved against earlier generations.

  3. Targeted Safety Patching and Architectural Review: Identify specific points of regression, such as the statistically significant regression between Gemma 2 and Gemma 3 (p = 0.030), to trigger comprehensive architectural re-evaluation or targeted retraining when a model transition, like introducing multimodal capabilities, causes a drop in overall robustness.

  4. Category-Specific Robustness Monitoring: Establish continuous monitoring of category-level Adversarial Success Rates (ASR) across generations to track specific vulnerabilities; this reveals that while harassment shows a broadly declining trajectory, other categories like misinformation remain elevated even in the safest generation.

  5. Judge Sensitivity Calibration: Implement multi-judge audits, such as the second-judge audit using a 70B judge, to assess how safety metrics are sensitive to judge choice, particularly for complex harm categories like copyright, ensuring that reported ASRs are not artifacts of a single judge's specific harm threshold.

Sources

Related papers