JailBound: FOL-Guided Jailbreak Evaluation for Exploring LLM Behavioral Boundaries

arXiv:2611.07530 · cs.CR · Submitted 2026-11-05 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: We're looking at the paper: "JailBound: FOL-Guided Jailbreak Evaluation for Exploring LLM Behavioral Boundaries".

Elias: Here's the gist of the paper. As large language models (LLMs) are increasingly deployed as general-purpose conversational assistants, they must handle diverse user requests while reliably refusing harmful ones This paper presents JailBound, a unified jailbreak evaluation framework that combines automated benchmark construction with first-order loss (FOL)-guided dual-branch search It addresses the limitations of existing benchmarks by systematically extending coverage to new threat conditions and exploring local behavioral changes between safe refusal and unsafe response Framework Overview JailBound organizes threats by risk category, application domain, and attack type Conditioned on this taxonomy, a fine-tuned meta-attack generator synthesizes evaluation prompts across diverse threat conditions FOL serves as a local sensitivity signal for exploring high-value attack states and behavior-sensitive states The framework uses a hierarchical threat taxonomy over risk categories, application domains, and attack types How the training signal is built The framework begins with a hierarchical threat taxonomy that organizes jailbreak evaluation across risk categories, attack types, and application domains Based on this classification structure, we construct a large annotated jailbreak corpus and fine-tune Qwen2.5-14B as an automated meta-attack generator The generator produces structured meta-attack prompts that encode seed attack intent s and threat condition c To train the generator, we collect diverse open-source jailbreak datasets, re-annotate them under our taxonomy using Qwen3-72B-Embiggened, construct approximately 100k structured training instances, and fine-tune Qwen2.5-14B as the meta-attack generator using quantized low-rank adaptation (QLoRA) Where the gains come from The core of JailBound involves a FOL-guided dual-branch search in continuous embedding space to explore both highly vulnerable regions and locally transition-sensitive states This strategy jointly explores two complementary regions of the attack landscape: high-value regions with strong jailbreak potential and transition-sensitive regions near the safe–unsafe transition The minus branch, O−(e), corresponds to the High-Value Branch, which favors locally stable attack embeddings with high attack utility The plus branch, O+(e), corresponds to the Safety-Sensitivity Branch, which favors transition-prone states with high local sensitivity Evaluation Results and Insights Under matched settings, the High-Value Branch improves average ASR over a ZOL-only baseline by 5.1 and 4.9 percentage points across the two evaluation settings Under local perturbations, states from the Safety-Sensitivity Branch exhibit higher behavioral flip rates than those from the High-Value Branch across all tested radii for JailBound and S-Eval, indicating greater local sensitivity of judge-assigned safety labels The Safety-Sensitivity states have a higher BFR at each representative radius for both JailBound and S-Eval Furthermore, the Safety-Sensitivity Branch achieves higher TSR than the High-Value Branch for every target reported in Table 17 Limitations and Scope First, FOL is used as a local behavioral-sensitivity signal on the optimization model; it does not estimate a global distance to a model’s safety boundary Second, continuous optimization must be materialized into discrete text before evaluation, and this conversion reduces ASR by an average of 6.9 percentage points in our diagnostic experiment Third, the reported scores depend on the fixed external-judge threshold Finally, transfer patterns are observational: family similarity, deployment-time filtering, model scale, and alignment differences are not isolated as causal factors The authors treat JailBound as a controlled evaluation and diagnosis framework rather than a complete model of real-world safety risk In this paper, we present JailBound, a jailbreak evaluation framework integrating hierarchical threat modeling, automated meta-attack generation, and first-order loss (FOL)-guided dual-branch search in embedding space By organizing threats across risk categories, application domains, and attack types, JailBound enables systematic evaluation under diverse threat conditions Its two optimization branches reveal complementary vulnerabilities by identifying high-value attack states and locally behavior-sensitive states Overall, these findings highlight the importance of combining structured threat coverage with sensitivity-aware optimization for a more comprehensive assessment of LLM jailbreak robustness References

1: Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, Sang T.

Nadia: Truong; DecodingTrust: A comprehensive assessment of trustworthiness in GPT models.

Elias: In Advances in Neural Information Processing Systems; 10

2: Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu; HarmBench: A standardized evaluation framework for automated red teaming and robust refusal.

Nadia: In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller; Proceedings of the 41st International Conference on Machine Learning; 21

3: Tinghao Xie, Xiangyu Qi, Yi Zeng, Yangsibo Huang, Udari Sehwag, Kaixuan Huang; SORRY-Bench: Systematically evaluating large language model safety refusal.

Title and authors: Nadia: So we're looking at this paper called JailBound: FOL-Guided Jailbreak Evaluation for Exploring LLM Behavioral Boundaries. It sounds like they're trying to build a better way to test how resilient these models are when people try to trick them into breaking their safety rules.

Elias: Yeah, the title says they're using first order loss, which is this local sensitivity signal, to explore those behavioral boundaries between what the model safely refuses and what it actually does unsafe stuff. It’s about going beyond just seeing if a prompt works or not and looking at how the model behaves right around that line.

Priya: From a measurement standpoint, I'm curious how they're handling the sheer variety of threats out there with this approach. The paper mentions they organize threats by risk category, application domain, and attack type to build their framework. That sounds like a lot of structure for testing.

Nadia: Exactly. Instead of just throwing random prompts at the model, they’re using this taxonomy to systematically create evaluation prompts across different threat conditions. It gives them a way to cover more ground than just the standard sets we usually see in benchmarks <ref:2611.07530#pg2>.

Elias: And that systematic construction leads into how they train their meta-attack generator, which is a fine tuned Qwen two point five fourteen billion model using some low rank adaptation. They’re essentially training an AI to write these structured meta attack prompts based on the taxonomy they built.

Priya: But what does this structure actually tell us about the vulnerabilities? The paper says they use this search strategy to explore two complementary regions of the attack landscape, one for high value and one for local sensitivity. That’s interesting because it suggests there are different kinds of jailbreaks out there.

Nadia: Right, so they have this minus branch, which is what they call the High Value Branch, that favors locally stable attack embeddings with high attack utility. And then they have the plus branch, which is the Safety-Sensitivity Branch that favors transition prone states with high local sensitivity <ref:2611.07530#pg2>.

Title and authors: Elias: That distinction is key because it separates attacks that just succeed from attacks that are right on the edge of failure. The paper shows that under matched settings, this High Value Branch improves the average attack success rate over a zero order loss only baseline by five point one and four point nine percentage points across both evaluation settings.

Priya: But then when they introduce local perturbations, things get interesting because the Safety-Sensitivity Branch actually shows higher behavioral flip rates than the High Value Branch across all tested radii for both JailBound and S eval <ref:2611.07530#pg2>. That suggests that locally, those transition states are more sensitive to change in the model's safety labels.

Nadia: So what does this mean practically? It means that even when you’re testing a specific attack, you might find that small changes near the safety line cause a bigger jump in how the model responds compared to just pushing it into a stable, high utility state.

Elias: And Elias is worried about the mechanism here because he points out that first order loss is only used as a local behavioral sensitivity signal on the optimization model; it doesn't estimate a global distance to the safety boundary <ref:2611.07530#pg2>. It’s not telling you how far away from danger you are overall.

Priya: That’s a fair point. And I also see a limitation mentioned where they have to materialize these continuous embedding states into discrete text before they can evaluate them, and that conversion reduces the average attack success rate by six point nine percentage points in their diagnostic experiment <ref:2611.07530#pg2>. So the actual process of turning the math into test text hurts their results a bit.

Nadia: That’s a real hurdle for anyone trying to use this framework in practice, because you have to go through that materialization step just to get an evaluation score from the judge. It makes it less direct than I'd like it to be for real red teaming work.

Title and authors: Elias: And they also mention that the reported scores depend on a fixed external judge threshold, which means their results are tied to how that specific judge is set up <ref:2611.07530#pg2>. It’s a dependency we have to keep in mind when looking at these numbers.

Priya: The paper also flags that the transfer patterns they observe are observational, meaning things like family similarity or model scale aren't isolated as direct causal factors for the results <ref:2611.07530#pg2>. They are treating JailBound as a controlled evaluation and diagnosis framework rather than a complete model of real world safety risk.

Nadia: So to wrap up on the improvements, they’re adding that Risk Coverage Rate or RCR, which measures whether at least one successful jailbreak is found in each risk category, giving them a way to check if their structured generation actually covers diverse harm dimensions <ref:2611.07530#pg2>.

Elias: And the core improvement is combining this structured threat modeling with the FOL guided dual branch search to identify both high value and locally transition sensitive states, which gives a more comprehensive view of where vulnerabilities lie <ref:2611.07530#pg2>.

Priya: It seems they've moved beyond just checking if a model fails or not and are trying to map out the local sensitivity around the safety boundary, which is something I think is really valuable for understanding how models actually operate in practice <ref:2611.07530#pg2>.

Nadia: So overall, JailBound gives us a way to systematically probe model robustness by combining structured threat coverage with this sensitivity aware optimization. It shows that we need to look both at the big picture threats and the fine grained local behavior changes <ref:2611.07530#pg2>.

Elias: Yeah, it's a framework for evaluation, not a fix for safety itself. It’s about diagnosing where the model is locally sensitive versus where it has high utility in attacking.

Priya: I just think getting this level of detail on local sensitivity is important because it helps us understand the fine tuning details that make these models vulnerable or resilient <ref:2611.07530#pg2>.

Nadia: Yeah, so we've seen how they built this framework and what the initial results showed regarding high value versus safety sensitivity states. That's where it was going.

The paper's summary: Nadia: So, to recap, JailBound is this new framework that combines building its own test cases automatically with this search method using first order loss to find where LLMs break their safety rules.

Elias: Right, it’s about organizing those threats by risk type and application domain so you can systematically test the AI across a wide range of conditions.

Nadia: And the main thing they’re doing is splitting the search into two branches, one that looks for high value jailbreaks and another that looks for states where the model is just on the edge of switching from safe to unsafe.

Elias: Exactly. They call those branches High Value Branch and Safety Sensitivity Branch, and it shows you can find both stable attacks and those really tricky transition points near the safety boundary.

Nadia: The results they showed are pretty telling; under normal settings, that high value branch actually improved the attack success rate by about five point one percent over a simple baseline.

Elias: But then when things get locally perturbed, that safety sensitivity branch starts showing higher flip rates than the high value branch across all tested radii.

Nadia: That’s significant because it suggests that those states right near the line are more sensitive to small changes in how you ask something compared to just pushing for a high utility attack.

Elias: It means you’re not just looking for a successful jailbreak, but you’re also mapping out the fine-grained local behavior around that safety boundary.

Nadia: But they also pointed out some real practical hurdles, like having to turn those continuous math states into actual text before you can even run an evaluation with an external judge.

Elias: And they noted that this conversion step actually drops the attack success rate by about six point nine percent on average in their testing.

Nadia: Plus, they have to remember that the scores depend on a fixed threshold set by the external judge, so it’s not just a pure measure of how bad the model is.

Elias: And they also admitted that things like family similarity or how big the model is aren't isolated as direct causes for these results; they're treating this more like a controlled diagnosis tool than a full map of real world risk.

Nadia: So, what this means for us in practice is that we need to combine that structured threat coverage with this sensitivity-aware search to get a fuller picture of how robust these models actually are.

Elias: It’s about moving beyond just seeing if the model fails or not and starting to map out exactly where those local sensitivities live.

The paper's improvements: Tom: So, moving on from how they set up the test cases, we're looking at what this paper suggests as improvements to JailBound itself and where it goes next.

Nadia: Basically, they suggest adding a Risk Coverage Rate or RCR, which is a way to check if their structured generation actually hits every single risk category in their testing.

Elias: That’s a good addition because it gives you a metric for coverage rather than just success rates on specific tasks.

Nadia: Exactly, and they also talk about how the search strategy can be used to find both those high utility states and those locally sensitive transition points in one go.

Elias: It seems they’re pushing the idea that we should be looking at both the big picture attack potential and the fine grain local behavior changes simultaneously.

Nadia: They also discuss how you can use this framework to do white-box evaluations on specific targets while still testing transfer patterns across different models.

Elias: That’s important because it allows researchers to see if a jailbreak pattern found on one model actually applies when you move that same attack idea onto a different architecture.

Nadia: It means the framework is designed to be more flexible for real world red teaming where we have to test things in practice rather than just in theory.

Elias: And they admit that first order loss is only a local signal, so they’re focused on using it for sensitivity rather than trying to estimate a total distance from safety.

Nadia: So the limitation there is that you can't use this method to tell you globally how far away from the safety boundary any given prompt is.

Elias: Right, and another point they make is that turning those continuous math embeddings into actual text for evaluation actually reduces the attack success rate by about six point nine percent.

Nadia: It’s a trade-off between having a mathematically precise search and being able to actually run a concrete evaluation with an external judge.

Elias: That conversion step is something that needs to be factored into any practical application of this framework, especially if you're trying to build automated testing pipelines.

Nadia: So, the paper’s takeaway is that for safety research, we need to combine structured threat modeling with this sensitivity-aware search and always keep an eye on those practical evaluation hurdles.

Conclusion: Tom: So we’re wrapping up on JailBound, which is this framework that uses first order loss to find those high value and locally sensitive jailbreak states across different threat categories.

Nadia: It really shows how important it is to move past just checking if a model fails or not, and start mapping out the fine grain local behavior changes around the safety boundary.

Elias: I think that’s the core idea, using that dual branch search to find both stable attacks and those transition points where things are about to flip.

Priya: From a measurement standpoint, it’s interesting because they show you can get this structured coverage while still being able to measure how sensitive those states actually are using behavioral flip rates.

Nadia: Right, so we’ve seen how they built this framework and what the initial results showed regarding high value versus safety sensitivity states.

Elias: It really highlights the need to look at both the big picture threats and those fine grained local behavior changes for a more complete assessment of LLM jailbreak robustness.

Priya: I just think getting this level of detail on local sensitivity is important because it helps us understand the fine tuning details that make these models vulnerable or resilient in practice.

Nadia: So overall, JailBound gives us a way to systematically probe model robustness by combining structured threat coverage with this sensitivity aware optimization.

Elias: Yeah, it’s a framework for evaluation, not a fix for safety itself; it’s about diagnosing where the model is locally sensitive versus where it has high utility in attacking.

Priya: It seems like they've moved beyond just checking if a model fails or not and are trying to map out exactly where those local sensitivities live.

Nadia: That’s what we saw, so we’ve seen how they built this framework and what the initial results showed regarding high value versus safety sensitivity states.

Elias: We should keep an eye on this kind of work because mapping those transition states is key to understanding how models operate under pressure.

Fazong Wu, Xin Wang, Ming Yang, †Zhenyong Zhang, ℔Xiaoming Wu

Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center, Qilu University of Technology (Shandong Academy of Sciences) · Shandong Provincial Key Laboratory of Industrial Network and Information System Security, Shandong Fundamental Research Center for Computer Science · Quan Cheng Laboratory

cs.CR

Submitted: 2026-11-05

Updated: 2026-11-05

Comments: Accepted by 40th Conference on Neural Information Processing Systems (NeurIPS 2026)

Code: https://github.com/FrazeWu/JailBound

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 90/100

The gist: As large language models (LLMs) are increasingly deployed as general-purpose conversational assistants…

Key concepts

Hierarchical Threat Taxonomy
This is a structured way to categorize potential jailbreak attempts based on risk categories, application domains, and specific attack types. This allows the framework to systematically cover a wide range of threats instead of just testing random prompts.
FOL-Guided Dual-Branch Search
This search method uses first-order loss (FOL) as a signal to explore two distinct areas of the attack landscape: one branch focuses on high-value, stable attacks, and the other focuses on locally sensitive states near the safe/unsafe boundary.
High-Value Branch (O-(e))
This branch favors attack embeddings that are locally stable and possess high utility for jailbreaking. It aims to find attacks that are robust and likely to succeed under matched settings, showing strong performance against baseline models.
Safety-Sensitivity Branch (O+(e))
This branch targets states near the safe-unsafe transition, favoring embeddings with high local sensitivity. This helps identify specific behavioral changes or perturbations that cause a model to flip its safety label, even if the attack isn't globally 'high-value'.

Terminology

Summary

As large language models (LLMs) are increasingly deployed as general-purpose conversational assistants, they must handle diverse user requests while reliably refusing harmful ones This paper presents JailBound, a unified jailbreak evaluation framework that combines automated benchmark construction with first-order loss (FOL)-guided dual-branch search It addresses the limitations of existing benchmarks by systematically extending coverage to new threat conditions and exploring local behavioral changes between safe refusal and unsafe response

Framework Overview

JailBound organizes threats by risk category, application domain, and attack type Conditioned on this taxonomy, a fine-tuned meta-attack generator synthesizes evaluation prompts across diverse threat conditions FOL serves as a local sensitivity signal for exploring high-value attack states and behavior-sensitive states The framework uses a hierarchical threat taxonomy over risk categories, application domains, and attack types

How the training signal is built

The framework begins with a hierarchical threat taxonomy that organizes jailbreak evaluation across risk categories, attack types, and application domains Based on this classification structure, we construct a large annotated jailbreak corpus and fine-tune Qwen2.5-14B as an automated meta-attack generator The generator produces structured meta-attack prompts that encode seed attack intent s and threat condition c To train the generator, we collect diverse open-source jailbreak datasets, re-annotate them under our taxonomy using Qwen3-72B-Embiggened, construct approximately 100k structured training instances, and fine-tune Qwen2. p. 35-14B as the meta-attack generator using quantized low-rank adaptation (QLoRA)

Where the gains come from

The core of JailBound involves a FOL-guided dual-branch search in continuous embedding space to explore both highly vulnerable regions and locally transition-sensitive states This strategy jointly explores two complementary regions of the attack landscape: high-value regions with strong jailbreak potential and transition-sensitive regions near the safe–unsafe transition The minus branch, O−(e), corresponds to the High-Value Branch, which favors locally stable attack embeddings with high attack utility The plus branch, O+(e), corresponds to the Safety-Sensitivity Branch, which favors transition-prone states with high local sensitivity

Evaluation Results and Insights

Under matched settings, the High-Value Branch improves average ASR over a ZOL-only baseline by 5.1 and 4.9 percentage points across the two evaluation settings Under local perturbations, states from the Safety-Sensitivity Branch exhibit higher behavioral flip rates than those from the High-Value Branch across all tested radii for JailBound and S-Eval, indicating greater local sensitivity of judge-assigned safety labels The Safety-Sensitivity states have a higher BFR at each representative radius for both JailBound and S-Eval Furthermore, the Safety-Sensitivity Branch achieves higher TSR than the High-Value Branch for every target reported in Table 17

Limitations and Scope

First, FOL is used as a local behavioral-sensitivity signal on the optimization model; it does not estimate a global distance to a model’s safety boundary Second, continuous optimization must be materialized into discrete text before evaluation, and this conversion reduces ASR by an average of 6.9 percentage points in our diagnostic experiment Third, the reported scores depend on the fixed external-judge threshold Finally, transfer patterns are observational: family similarity, deployment-time filtering, model scale, and alignment differences are not isolated as causal factors The authors treat JailBound as a controlled evaluation and diagnosis framework rather than a complete model of real-world safety risk

Conclusion

In this paper, we present JailBound, a jailbreak evaluation framework integrating hierarchical threat modeling, automated meta-attack generation, and first-order loss (FOL)-guided dual-branch search in embedding space By organizing threats across risk categories, application domains, and attack types, JailBound enables systematic evaluation under diverse threat conditions Its two optimization branches reveal complementary vulnerabilities by identifying high-value attack states and locally behavior-sensitive states Overall, these findings highlight the importance of combining structured threat coverage with sensitivity-aware optimization for a more comprehensive assessment of LLM jailbreak robustness

References

[1] Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, Sang T. Truong; DecodingTrust: A comprehensive assessment of trustworthiness in GPT models. p. 9 In Advances in Neural Information Processing Systems; 10

[2] Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu; HarmBench: A standardized evaluation framework for automated red teaming and robust refusal. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller; Proceedings of the 41st International Conference on Machine Learning; 21

[3] Tinghao Xie, Xiangyu Qi, Yi Zeng, Yangsibo Huang, Udari Sehwag, Kaixuan Huang; SORRY-Bench: Systematically evaluating large language model safety refusal. In International Conference on Learning Representations; 59937–59973

[4] Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones; Constitutional AI: Harmlessness from AI feedback. arXiv preprint arXiv:2212. p. 008073; 4

[5] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright; Training language models to follow instructions with human feedback. Advances in neural information processing systems; 35:27730–27744

[6] Liwei Jiang, Kavel Rao, Seungju Han, Allyson Ettinger, Faeze Brahman, Sachin Kumar; WildTeaming at scale: From in-the-wild jailbreaks to (adversarially) safer language models. Advances in Neural Information Processing Systems; 37:47094–47165

[7] Alexander Wei, Nika Haghtalab, Jacob Steinhardt; Jailbroken: How does LLM safety training fail? Advances in neural information processing systems; 36:80079–80110

[8] Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter; Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307. p. 015043; 2

[9] Patrick Chao, Alexander Robey, Edgar Dobriban; Jailbreaking black box large language models in twenty queries. In 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML); 9

[10] Yutao Mou, Shikun Zhang, Wei Ye; SG-Bench: Evaluating LLM safety generalization across diverse tasks and prompt types. Advances in Neural Information Processing Systems; 37:123032–123054

[11] Weikang Zhou, Xiao Wang, Limao Xiong, Han Xia, Yingshuang Gu, Mingxu Chai; EasyJailbreak: A unified framework for jailbreaking large language models. arXiv preprint arXiv:2403. p. 0 p. 012171; 1

[12] Xin Wang, Daniel Quevedo, Dongrun Li, Peng Cheng, Jiming Chen; Data security and privacy for ai-enabled smart manufacturing. Engineering; 52, 08 2025

[13] Xiaohan Yuan, Jinfeng Li, Dongxia Wang, Yuefeng Chen; S-Eval: Towards automated and comprehensive safety evaluation for large language models. Proceedings of the ACM on Software Engineering; 2(ISSTA):2136–2157

[14] Zhexin Zhang, Leqi Lei, Lindong Wu, Rui Sun, Yongkang Huang; SafetyBench: Evaluating the safety of large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers); 14

[15] Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring; Red teaming language models with language models. In Proceedings of the 2022 conference on empirical methods in natural language processing; 3419–3448

[16] Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke Zettlemoyer; Qlora: Efficient finetuning of quantized LLMs. Advances in neural information processing systems; 36:10088–10115

[17] Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce; JailbreakBench: An open robustness benchmark for jailbreaking large language models. Advances in Neural Information Processing Systems; 37:55005–55029

[18] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian; The Llama 3 herd of models. arXiv preprint arXiv:2407. p. 021783; 18

[19] Yuxiao Li, Alina Fastowski, Efstratios Zaradoukas, Bardh Prenkaj, Gjergji Kasneci; Analysing the safety pitfalls of steering vectors. In Findings of the Association for Computational Linguistics: ACL 2026; 19

[20] An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng; Qwen3 technical report. arXiv preprint arXiv:2505. p. 009388; 1

[21] Gemma Team, Morgane Riviere, Shreya Pathak; Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408. p. 000118; 21

[22] Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard; Gemma 3 technical report. arXiv preprint arXiv:2503. p. 019786; 21

[23] Marco Siino; Mistral at SemEval-2024 task 5: Mistral 7B for argument reasoning in civil procedure. In Proceedings of the 18th international workshop on semantic evaluation (SemEval-2024); 18

[24] Weilin Cai, Juyong Jiang, Fan Wang, Jing Tang, Sunghun Kim; A survey on mixture of experts in large language models. IEEE Transactions on Knowledge and Data Engineering; 37(7):3896–3915

[25] Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang; Yi: Open foundation models by 01.AI. arXiv preprint arXiv:2403.04652; 25

[26] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang; Judging LLM-as-a-judge with MT-Bench and chatbot arena. Advances in neural information processing systems; 36:46595–46623

[27] Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul; Zephyr: Direct distillation of LM alignment. arXiv preprint arXiv:2310. p. 016944; 1

[28] Daniil Moskovskiy, Sergey Pletenev, Alexander Panchenko; LLMs to replace crowdsourcing for parallel data creation? the case of text detoxification. p. 2 In Findings of the Association for Computational Linguistics: EMNLP 2024; 1

[29] Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin; ChatGLM: A family of large language models from GLM-130B to GLM-4 all tools. arXiv preprint arXiv:2406. p. 012793; 1

[30] Guan Wang, Sijie Cheng, Xianyuan Zhan, Xiangang Li, Sen Song; OpenChat: Advancing open-source language models with mixed-quality data. In International Conference on Learning Representations; 57021–57040

[31] Yuxin Wen, Neel Jain, John Kirchenbauer, Micah Goldblum, Jonas Geiping; Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discovery. Advances in Neural Information Processing Systems; 36:51008–51025

[32] Chuan Guo, Alexandre Sablayrolles, Hervé Jégou, Douwe Kiela; Gradient-based adversarial attacks against text transformers. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing; 5747–5757

Pruning redundant reasoning tokens

The paper introduces JailBound, a jailbreak evaluation framework that combines automated benchmark construction with first-order loss (FOL)-guided dual-branch search It organizes threats by risk category, application domain, and attack type This allows for systematic evaluation under diverse threat conditions The framework uses a hierarchical threat taxonomy over risk categories, application domains, and attack types

The meta-attack generator produces structured meta-attack prompts that encode seed attack intent s and threat condition c To train the generator, we collect diverse open-source jailbreak datasets, re-annotate them under our taxonomy using Qwen3-72B-Embiggened, construct approximately 100k structured training instances, and fine-tune Qwen2.5-14B as the meta-attack generator using quantized low-rank adaptation (QLoRA) The optimization objective is Latk(e) = ˜r(e) − γz∥z − z0∥2F − γu∥U − U0∥2F, which regularizes the learned soft prefix and editable embedding block toward their initializations The search explores both highly vulnerable regions and transition-sensitive regions near the safe–unsafe transition using two branches O±(e) = Latk(e) ± λ FOL(e)

Under matched settings, High-Value improves ASR over ZOL-only by 5.1 and 4.9 percentage points across the two evaluation settings Under local perturbations, states from the Safety-Sensitivity Branch exhibit higher behavioral flip rates than those from the High-Value Branch across all tested radii for JailBound and S-Eval The Safety-Sensitivity states have a higher BFR at each representative radius for both JailBound and S-Eval The High-Value Branch usually yields the strongest ASR, while the Safety-Sensitivity Branch remains intermediate

Table 9 reports the complete sweeps for JailBound and S-Eval, showing that BFR is higher for Safety-Sensitivity states across all tested radii Materialization reduces ASR from 37.1% to 30.3%, an average drop of 6. p. 69 percentage points Table 4 shows that OpenChat is the most vulnerable after optimization, with the highest High-Value (74. p. 78%), Safety-Sensitivity (60. p. 04%), and Avg. p. 7 (67.6%) scores The Safety-Sensitivity Branch achieves higher TSR than the High-Value Branch for every target reported in Table 17

The paper discusses the main limitations of the current study and possible extensions in Section 4.7 FOL is used as a local behavioral-sensitivity signal on the optimization model; it does not estimate a global distance to a model’s safety boundary Continuous optimization must be materialized into discrete text before evaluation, and this conversion reduces ASR by an average of 6. p. 8

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Automated benchmark construction allows for systematic extension of testing to new threat conditions by organizing threats by risk category, application domain, and attack type. This enables researchers to create a structured generation over a canonical threat space rather than relying on manually designed prompts.

  2. The FOL-guided dual-branch search strategy explores both high-value regions with strong jailbreak potential and locally transition-sensitive states near the safe–unsafe transition. This allows the system to identify not just successful attacks but also locally transition-sensitive states.

  3. The system can distinguish between attack success and local behavioral sensitivity by comparing the High-Value Branch, which favors locally stable attack embeddings with high attack utility, and the Safety-Sensitivity Branch, which favors transition-prone states with high local sensitivity.

  4. Materialization of continuous embedding states into executable text is performed to evaluate robustness under a unified protocol using an external judge. This allows for target-specific white-box evaluation while enabling cross-model transfer testing where the FOL-optimized attacks are not confined to the surrogate model in this evaluation.

  5. The system can characterize vulnerability breadth by calculating the Risk Coverage Rate (RCR), which measures whether at least one successful jailbreak is found in each risk category, providing a measure of how well the evaluation covers diverse harm dimensions.

Sources

Related papers