Non-Monotonic Safety Alignment Across Gemma Generations: A Quality-Diversity Red-Teaming Study

summary

The gist

Safety alignment in LLMs does not improve monotonically across model generations Gemma 3 (12B) exhibits 68.7% ± 5.7% attack success rate (ASR; mean ± std, 3 seeds)…

In short

The study used an evolutionary algorithm to find diverse jailbreak attacks against four Gemma model generations. It found that safety improvement is not steady; Gemma 3 showed a significant regression compared to Gemma 2, but Gemma 4's safety gains generalize better than previous models. This requires adaptive, longitudinal safety testing.

Key concepts

MAP-Elites
This is a quality-diversity evolutionary algorithm used to automatically discover many different jailbreak strategies. It searches for the most effective attacks across various behaviors and harm categories by iteratively improving attack designs based on how well they bypass the model's safety filters.
Cross-Generational Transfer Analysis
Researchers replayed attacks from older models against newer ones to see which vulnerabilities persist. This analysis showed that while some attacks transfer, Gemma 4's safety improvements are more robust and generalize across different attack types compared to earlier versions.
Non-Monotonic Safety Alignment
This means safety performance doesn't always get better as a model gets newer. The study found a drop in success rates between Gemma 2 and Gemma 3, suggesting that architectural changes, like adding vision capabilities, can sometimes temporarily reduce overall safety.
Vulnerability Fingerprint Analysis
This technique visualizes the specific types of harm each model is most vulnerable to using radar charts. It showed that while some models became much better at avoiding certain harms (like chemical/biological), others still have high risks in areas like misinformation.

Terminology used across episodes

This episode discusses

The paper

Non-Monotonic Safety Alignment Across Gemma Generations: A Quality-Diversity Red-Teaming Study · Read on arXiv

Subhadip Mitra

Rota Labs

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: We're looking at the paper: "Non-Monotonic Safety Alignment Across Gemma Generations".

Elias: Here's the gist of the paper. Safety alignment in LLMs does not improve monotonically across model generations Gemma 3 (12B) exhibits 68.7% ± 5.7% attack success rate (ASR; mean ± std, 3 seeds), significantly higher than its predecessor Gemma 2 (45.5% ± 7.2%; p = 0.030, paired bootstrap) and its successor Gemma 4 (33.9% ± 1.8%) Replaying evolved attack archives across generations reveals that attacks from other generations transfer to Gemma 3 at 44–46% but only 14–18% to Gemma 4, indicating that Gemma 4’s safety gains generalize beyond the attack distributions evolved against earlier generations The study employs a quality-diversity (QD) evolutionary algorithm called MAP-Elites to discover diverse jailbreak strategies across seven harm categories from the HarmBench taxonomy The MAP-Elites archive is a 3D grid indexed by (strategy × encoding × harm category), with each cell storing the highest-fitness attack for that behavioral niche Fitness is evaluated by a self-hosted LLM judge that classifies each target response along three binary axes: refusal, harmful content, and relevance The MAP-Elites archive is a 3D grid indexed by (strategy × encoding × harm category), with each cell storing the highest-fitness attack for that behavioral niche Cross-Generational Transfer Analysis The cross-generational transfer protocol involves replaying each evolved archive against all other models to construct a transfer tensor T

i, j, s: This produces a transfer tensor T

i, j, s: where each entry is the fraction of attacks from Ai,s that succeed against Mj The diagonal T¯

i, i: is the mean self-evaluation success rate Off-diagonal entries reveal forward transfer (i j) Figure 1 presents the full 4 × 4 transfer matrix with uncertainty estimates Robustness Trajectory Across Generations The study plots robustness R = 1 − ASR as a function of model generation (with parameter count on the axis) to visualize the trajectory of safety alignment within the Gemma family The improvement from Gemma 1 to Gemma 2 (p = 0.032) and the regression from Gemma 2 to Gemma 3 (p = 0.030) are both statistically significant under paired bootstrap tests The non-monotonic pattern coincides with the architectural transition from text-only (Gemma 1, 2) to multimodal (Gemma 3, 4), though Gemma 4’s recovery shows that the regression is not an inherent consequence of multimodal design Vulnerability Fingerprint Analysis Figure 3 presents vulnerability fingerprints as radar charts, showing distinct profiles for each model Gemma 1 exhibits moderate vulnerability with peaks in cybercrime (97.2%) and copyright (100%) Gemma 3’s profile is dramatically expanded in misinformation (99.1%) and cybercrime (100%) Gemma 4 achieves 0% ASR in chem/bio, harmful, and illegal, but copyright (100%) and misinformation (76.9%) remain elevated The results underscore the need for adaptive, longitudinal safety evaluation Static benchmarks would miss the non-monotonic trajectory entirely Architectural changes (such as adding multimodal capabilities) should trigger comprehensive safety re-evaluation The distinction between Gemma 3 and Gemma 4 is only visible through cross-generational transfer analysis Limitations of the Study Three seeds per model provide reproducibility estimates but limited statistical power The single model family (Gemma) for generational analysis may not generalize to other families The 8B judge shows weak agreement (κ = 0.15) with a 70B judge on harm classification, particularly for copyright HarmBench coverage is comprehensive but not exhaustive The evolutionary probe explores a subset of the attack space; absence of a successful attack does not prove safety The study only covers text modality; Gemma 3 and 4’s vision capabilities may present additional attack surfaces not captured here References Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J.

Nadia: Pappas, and Eric Wong.

Elias: Jailbreaking black box large language models in twenty queries Antoine Cully, Jeff Clune, Danesh Tarapore, and Jean-Baptiste Mouret.

Nadia: Robots that can adapt like animals Matthew C.

Title and authors: Nadia: So, what the paper summarizes is that when you replay attacks evolved against older models onto newer ones, the transfer of those attacks is very different depending on which generation you are looking at.

Elias: It found that attacks from earlier generations transfer to Gemma three at a rate of forty-four to forty-six percent, but only about fourteen to eighteen percent transfer to Gemma four <ref:2606.00813#pg1,generations transfer to Gemma 3 at>.

Priya: That suggests that the safety gains in the newest model, Gemma four are not just because it’s bigger or has new features; they seem to be addressing entirely different kinds of attacks than what was used against the previous versions <ref:2606.00813#pg1>.

Nadia: It really highlights that what worked before doesn't automatically translate to success now, which is a big deal for how we think about defense.

Elias: They also point out a non-monotonic pattern in robustness when you look at the parameter count, showing improvement from Gemma one to Gemma two but then a regression from Gemma two to Gemma three <ref:2606.00813#pg3>.

Priya: That regression is significant because they show that this drop isn't just a random dip; it happens statistically significant versus the previous model under paired bootstrap tests.

Nadia: It seems the paper is saying that we need to look at these safety surfaces in a longitudinal way, not just once, to really understand what's happening with these models as they evolve.

Elias: The implication is that when you see a dip like this between Gemma two and Gemma three you shouldn't assume the model got worse overall; it’s likely because of something specific in the architecture change, like moving to multimodal capabilities <ref:2606.00813#pg1,between Gemma 2 and Gemma 3>.

The paper's summary: Nadia: The authors suggest a few key things for how we should approach this kind of safety evaluation. First, they propose using this quality diversity methodology with MAP-Elites for differential safety probing across model versions.

Elias: That means instead of testing one attack against one model, you evolve a whole archive of attacks and then replay them systematically to build that transfer matrix with uncertainty estimates.

Priya: And I think the most valuable part for us is the category-level analysis showing how specific vulnerabilities change, like misinformation success jumping from twenty-nine percent to ninety-nine percent between Gemma two and Gemma three <ref:2606.00813#pg1,between Gemma 2 and Gemma 3>.

Nadia: That jump in misinformation is a big flag because it shows that even when some areas get safer, other areas are getting much more vulnerable.

Elias: They also show vulnerability fingerprints as radar charts, which gives you a visual profile of where each model struggles, like Gemma four achieving zero attack success rate in chemical or biological harm categories <ref:2606.00813#pg1>.

Priya: But they also flag a limitation: the study uses a single self-hosted judge for all phases, which means the copyright results might be sensitive to that specific judge choice because it's not perfectly consistent.

Nadia: So, they’re trying to show that category-specific safety monitoring is necessary because overall robustness isn't the same thing as general safety across the board.

The paper's improvements: Elias: So to wrap up, the paper "Non-Monotonic Safety Alignment Across Gemma Generations: A Quality-Diversity Red-Teaming Study" shows that safety alignment in LLMs doesn't improve monotonically across generations; it dips between Gemma two and Gemma three but Gemma four’s gains seem to be more generalized <ref:2606.00813#pg1,between Gemma 2 and Gemma 3>.

Nadia: The implication for us is clear: we can't just rely on static benchmarks anymore. We need adaptive, longitudinal safety evaluation that looks at the whole trajectory of a model family over time.

Priya: I think the key thing is that architectural changes, like adding vision capabilities to get to Gemma three and four should trigger a full re-evaluation because they can disrupt what was previously learned about safe text representations <ref:2606.00813#pg1>.

Elias: And the distinction between Gemma three and Gemma four is only really clear when you look at that cross-generational transfer analysis because of those different attack distributions <ref:2606.00813#pg1>.

Nadia: That’s all we have for this one, but the point remains that understanding these non-monotonic trajectories is crucial for building truly robust systems.

Conclusion: Nadia: So we've seen how the study on "Non-Monotonic Safety Alignment Across Gemma Generations" suggests that safety improvements aren't always steady when you move from one model version to the next.

Elias: Exactly, it shows a drop between Gemma two and Gemma three but then the jump to Gemma four is different because those new models are encountering different types of attacks.

Priya: It really makes you think about how much specific harm categories matter when you’re looking at these numbers. For example, misinformation success jumps up dramatically between those two generations.

Nadia: And it shows that the way we measure safety has to be adaptive, not just relying on one static benchmark number across the board.

Elias: The authors use this quality diversity method with MAP-Elites to evolve attacks and then replay them across generations to build a transfer matrix. That’s how they see which vulnerabilities are staying put and which ones get patched.

Priya: But we have to keep an eye on the limitations they mentioned, like using only three seeds for each model, which means the statistical power isn't huge for those individual results.

Nadia: Right, so the study on "Non-Monotonic Safety Alignment Across Gemma Generations" tells us that architectural shifts can cause regressions that aren't immediately obvious from just looking at parameter counts.

Elias: It’s a reminder to check for specific transfer patterns, like how much an old attack archive actually survives when you hit a newer model architecture.

Priya: I think we should be paying close attention to those category-specific profiles, like how Gemma four handles chemical or biological harm versus copyright issues.

Nadia: That’s right, so the real lesson is that we need these longitudinal safety analyses to truly understand how these large language models are evolving in practice.

Elias: Exactly. It’s about understanding the whole family of models and not just one snapshot in time when you evaluate safety alignment.

Priya: Next up, we're looking at some papers that focus on how context impacts the security of code generated by AI, which is a different kind of vulnerability entirely.

More episodes

← Home