JailbreakOPT: Iterative Jailbreak Prompt Optimization with Macro-Action

summary

The gist

JailbreakOPT is a tool-assisted framework designed for improving iterative single-turn jailbreak prompt optimization by organizing diverse atomic jailbreak prompts into an attack tool library and…

In short

JailbreakOPT is a tool-assisted framework that improves iterative jailbreak prompt optimization by organizing atomic prompts into an attack tool library and composing them. It uses contextual Thompson sampling to decide which tools to use, learning from past attempts across episodes. This method finds stronger standalone attacks faster and is more robust against difficult AI models.

Key concepts

Attack Tool Library
This is a collection of pre-defined atomic jailbreak prompts, each acting as a specific 'tool' for the optimization process. Instead of searching randomly, the system selects and combines these known tools to build complex attack prompts.
Contextual Thompson Sampling
This is a method used to decide which attack tool to use next. It uses past results and current context (like the target model) to balance trying new tools (exploration) with using tools that have worked well so far (exploitation), making intelligent choices under uncertainty.
Intra-episode Optimization
This refers to the process of optimizing a single jailbreak attempt by composing atomic tools into a stronger prompt. It involves an orchestrator assigning actions, a mutator generating candidate prompts, and an evaluator checking the target model's response.
Inter-episode Learning (AHADP)
This is how the system learns from previous optimization episodes. It uses an Action-History–Augmented Decision Process to condition its tool selection on context signals like which tools were used before and what the target model is, allowing it to improve its strategy over time.

Terminology used across episodes

This episode discusses

The paper

JailbreakOPT: Iterative Jailbreak Prompt Optimization with Macro-Action · Read on arXiv

Ge Shi, *, Jun Yin, *, Donglin Xie, *, Fangyi Liu, *, Yucan Li, *

University of California, Davis University of California, Davis Department The Renmin University of China Independent Researcher Nankai University Cornell University The Chinese University of Hong Kong Shenzhen

Jailbreak attacks expose persistent safety weaknesses in large language models (LLMs), but existing stateless single-turn methods face a trade-off: hand-crafted prompts are expressive but static, while iterative prompt optimization can adapt but often relies on low-level mutations that require many target-model responses. We propose JailbreakOPT, a method that maintains a library of reusable prompt transformations (actions) and composes them into macro-actions for within-episode search, generating standalone attack prompts. To reuse experience across episodes, JailbreakOPT learns an action-selection policy using contextual Thompson sampling and outcomes from prior search paths. Experiments across multiple LLMs and benchmarks show that JailbreakOPT improves attack success rate (ASR) over single-action and iterative baselines. More broadly, exploration and exploitation over human-designed attack strategies can help probe residual vulnerabilities and identify failure patterns that may inform defenses aimed at broader classes of attacks.

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: I'm Nadia.

Elias: And I'm Elias, here with Priya.

Nadia: We're looking at the paper: "JailbreakOPT: Iterative Jailbreak Prompt Optimization with Macro-Action".

Elias: Here's the gist of the paper. JailbreakOPT is a tool-assisted framework designed for improving iterative single-turn jailbreak prompt optimization by organizing diverse atomic jailbreak prompts into an attack tool library and composing them via a unified intra-episode optimization abstraction to generate stronger standalone attack prompts.

Nadia: To facilitate experience reuse across multiple attack episodes, JailbreakOPT frames tool selection as a contextual bandit problem, employing contextual Thompson sampling to guide exploration and exploitation based on past outcomes.

Elias: Experiments demonstrate that JailbreakOPT enhances the Attack Success Rate (ASR) while simultaneously reducing the number of attacks required until success (No.A) compared to atomic single-turn attacks and existing iterative optimization baselines.

Nadia: The framework unifies representative iterative single-turn optimization methods under a common prompt-search abstraction, introduces intra-episode optimization with an attack tool library for composing atomic tools, and models inter-episode learning as an Action-History–Augmented Decision Process (AHADP) using contextual Thompson sampling to balance exploration and exploitation.

Paper summary: Nadia: So to recap, JailbreakOPT proposes a framework that organizes atomic jailbreak prompts into an attack tool library so you can compose them.

Elias: It uses this unified intra-episode optimization abstraction to generate stronger standalone prompts from those tools during each attack episode.

Priya: Before we get into the details of how it builds these chains, what’s the fundamental problem it's trying to solve in prompt optimization?

Nadia: The authors point out that existing iterative methods usually adapt within a single episode and don't transfer knowledge across different attack goals or targets.

Elias: They introduce inter-episode learning by framing this as contextual bandit learning using contextual Thompson sampling to guide tool selection based on past outcomes.

Priya: So the context signals they use for that learning include things like tool usage history and the specific target model identity?

Nadia: Yes, they condition this inter-episode adaptation on context signals such as tool usage history and the target model identity.

Elias: The framework sets up an orchestrator that assigns mutation actions to frontier elements, a mutator creates candidate prompts, and an evaluator gives feedback to update the search frontier.

Priya: That sounds like a lot of moving parts just to generate one new prompt candidate in each step of the process.

Nadia: It is complex, but they argue that this tool-augmented optimization substantially improves robustness when you are testing against difficult targets.

Conclusion: Nadia: Looking at JailbreakOPT, we see they’ve taken the idea of iterative optimization and made it more systematic by using these macro-actions and a tool library approach.

Elias: The authors are showing how you can reuse experience across multiple attack episodes, which is a big step because most methods don't transfer knowledge between different goals.

Priya: So if we think about this for the real world of finding vulnerabilities, what does that mean for someone who just wants to find weaknesses in an AI model?

Nadia: It means this approach is positioned as a scalable component for auto-red-teaming pipelines because it searches over a richer space of tool-composed prompts.

Elias: But they also admit their limitation, which is that the attack tool library and the order of tools are manually constructed, meaning performance is limited by how good those initial tools are.

Priya: That makes sense; if you build a bad set of tools into your library, the system can only compose better things from them.

Nadia: The paper shows that it’s not about finding a universal guarantee of safety, but it identifies concrete vulnerabilities under the tested configuration.

Elias: So for the practical side, this framework is useful for those who need to find specific weaknesses by reusing data across different testing scenarios.

More episodes

← Home