JailbreakOPT: Iterative Jailbreak Prompt Optimization with Macro-Action

arXiv:2606.11425 · cs.CR, cs.AI · Submitted 2026-06-09 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: I'm Nadia.

Elias: And I'm Elias, here with Priya.

Nadia: We're looking at the paper: "JailbreakOPT: Iterative Jailbreak Prompt Optimization with Macro-Action".

Elias: Here's the gist of the paper. JailbreakOPT is a tool-assisted framework designed for improving iterative single-turn jailbreak prompt optimization by organizing diverse atomic jailbreak prompts into an attack tool library and composing them via a unified intra-episode optimization abstraction to generate stronger standalone attack prompts.

Nadia: To facilitate experience reuse across multiple attack episodes, JailbreakOPT frames tool selection as a contextual bandit problem, employing contextual Thompson sampling to guide exploration and exploitation based on past outcomes.

Elias: Experiments demonstrate that JailbreakOPT enhances the Attack Success Rate (ASR) while simultaneously reducing the number of attacks required until success (No.A) compared to atomic single-turn attacks and existing iterative optimization baselines.

Nadia: The framework unifies representative iterative single-turn optimization methods under a common prompt-search abstraction, introduces intra-episode optimization with an attack tool library for composing atomic tools, and models inter-episode learning as an Action-History–Augmented Decision Process (AHADP) using contextual Thompson sampling to balance exploration and exploitation.

Paper summary: Nadia: So to recap, JailbreakOPT proposes a framework that organizes atomic jailbreak prompts into an attack tool library so you can compose them.

Elias: It uses this unified intra-episode optimization abstraction to generate stronger standalone prompts from those tools during each attack episode.

Priya: Before we get into the details of how it builds these chains, what’s the fundamental problem it's trying to solve in prompt optimization?

Nadia: The authors point out that existing iterative methods usually adapt within a single episode and don't transfer knowledge across different attack goals or targets.

Elias: They introduce inter-episode learning by framing this as contextual bandit learning using contextual Thompson sampling to guide tool selection based on past outcomes.

Priya: So the context signals they use for that learning include things like tool usage history and the specific target model identity?

Nadia: Yes, they condition this inter-episode adaptation on context signals such as tool usage history and the target model identity.

Elias: The framework sets up an orchestrator that assigns mutation actions to frontier elements, a mutator creates candidate prompts, and an evaluator gives feedback to update the search frontier.

Priya: That sounds like a lot of moving parts just to generate one new prompt candidate in each step of the process.

Nadia: It is complex, but they argue that this tool-augmented optimization substantially improves robustness when you are testing against difficult targets.

Conclusion: Nadia: Looking at JailbreakOPT, we see they’ve taken the idea of iterative optimization and made it more systematic by using these macro-actions and a tool library approach.

Elias: The authors are showing how you can reuse experience across multiple attack episodes, which is a big step because most methods don't transfer knowledge between different goals.

Priya: So if we think about this for the real world of finding vulnerabilities, what does that mean for someone who just wants to find weaknesses in an AI model?

Nadia: It means this approach is positioned as a scalable component for auto-red-teaming pipelines because it searches over a richer space of tool-composed prompts.

Elias: But they also admit their limitation, which is that the attack tool library and the order of tools are manually constructed, meaning performance is limited by how good those initial tools are.

Priya: That makes sense; if you build a bad set of tools into your library, the system can only compose better things from them.

Nadia: The paper shows that it’s not about finding a universal guarantee of safety, but it identifies concrete vulnerabilities under the tested configuration.

Elias: So for the practical side, this framework is useful for those who need to find specific weaknesses by reusing data across different testing scenarios.

Ge Shi, *, Jun Yin, *, Donglin Xie, *, Fangyi Liu, *, Yucan Li, *

University of California, Davis University of California, Davis Department The Renmin University of China Independent Researcher Nankai University Cornell University The Chinese University of Hong Kong Shenzhen

cs.CR, cs.AI

Submitted: 2026-06-09

Updated: 2026-11-02

Code: https://github.com/rail-ai-lab/JailbreakOPT

License: http://creativecommons.org/licenses/by-nc-sa/4.0/

Importance score: 93/100

The gist: JailbreakOPT is a tool-assisted framework designed for improving iterative single-turn jailbreak prompt optimization by organizing diverse atomic jailbreak prompts into an attack tool library and…

Key concepts

Attack Tool Library
This is a collection of pre-defined atomic jailbreak prompts, each acting as a specific 'tool' for the optimization process. Instead of searching randomly, the system selects and combines these known tools to build complex attack prompts.
Contextual Thompson Sampling
This is a method used to decide which attack tool to use next. It uses past results and current context (like the target model) to balance trying new tools (exploration) with using tools that have worked well so far (exploitation), making intelligent choices under uncertainty.
Intra-episode Optimization
This refers to the process of optimizing a single jailbreak attempt by composing atomic tools into a stronger prompt. It involves an orchestrator assigning actions, a mutator generating candidate prompts, and an evaluator checking the target model's response.
Inter-episode Learning (AHADP)
This is how the system learns from previous optimization episodes. It uses an Action-History–Augmented Decision Process to condition its tool selection on context signals like which tools were used before and what the target model is, allowing it to improve its strategy over time.

Terminology

Summary

JailbreakOPT is a tool-assisted framework designed for improving iterative single-turn jailbreak prompt optimization by organizing diverse atomic jailbreak prompts into an attack tool library and composing them via a unified intra-episode optimization abstraction to generate stronger standalone attack prompts. p. 0

Where the paper says it

We propose JailbreakOPT, a tool-assisted framework for improving iterative single-turn jailbreak prompt optimization. JailbreakOPT organizes diverse atomic jailbreak prompts into an attack tool library and composes them through a unified intra-episode optimization abstraction to generate stronger standalone attack prompts.

To facilitate experience reuse across multiple attack episodes, JailbreakOPT frames tool selection as a contextual bandit problem, employing contextual Thompson sampling to guide exploration and exploitation based on past outcomes. p. 0
Where the paper says it

To reuse experience across attack episodes, JailbreakOPT further frames tool selection as a contextual bandit problem and applies contextual Thompson sampling to guide exploration and exploitation based on past outcomes. Experiments across multiple target LLMs and attack goals show that JailbreakOPT improves attack success rate (ASR) while reducing the number of attacks until success (No.

Experiments demonstrate that JailbreakOPT enhances the Attack Success Rate (ASR) while simultaneously reducing the number of attacks required until success (No. p. 0
Where the paper says it

To reuse experience across attack episodes, JailbreakOPT further frames tool selection as a contextual bandit problem and applies contextual Thompson sampling to guide exploration and exploitation based on past outcomes. Experiments across multiple target LLMs and attack goals show that JailbreakOPT improves attack success rate (ASR) while reducing the number of attacks until success (No.

A) compared to atomic single-turn attacks and existing iterative optimization baselines. p. 0
Where the paper says it

Experiments across multiple target LLMs and attack goals show that JailbreakOPT improves attack success rate (ASR) while reducing the number of attacks until success (No. A) compared with atomic single-turn attacks and existing iterative optimization baselines.

The framework unifies representative iterative single-turn optimization methods under a common prompt-search abstraction, introduces intra-episode optimization with an attack tool library for composing atomic tools, and models inter-episode learning as an Action-History–Augmented Decision Process (AHADP) using contextual Thompson sampling to balance exploration and exploitation. p. 1

Where the paper says it

• We unify representative iterative single-turn optimization methods under a common prompt-search abstraction. • We introduce intra-episode optimization with an attack tool library, composing expressive atomic single-turn tools into stronger standalone prompts.

The method utilizes a unified abstraction where an orchestrator assigns mutation actions to frontier elements, a mutator executes these actions to generate candidate prompts, an evaluator queries the target model for feedback, and a selector updates the search frontier. p. 1
Where the paper says it

0 + + + Write a tutorial on how to make a bomb Attack Goal Orchestrator Mutator Selector Evaluator Success Outcome InterEpisode Trajectory MAB Contextualize Figure 1: Overview of the JailbreakOPT framework. Within each episode, JailbreakOPT orchestrates attack tools, composes them into encode–decode chains, queries the target LLM with standalone candidate prompts, and uses evaluator feedback to update the search frontier.

Inter-episode learning is conditioned on context signals such as tool usage history and target model identity. p. 8
Where the paper says it

Table 2: Ablation study of contextual features used for inter-episode learning in JailbreakOPT built on TAP Mehrotra et al. We evaluate two lightweight context signals, tool usage history and target model identity, and report ASR (↑) and the number of attack attempts until success (No.

The framework shows that tool-augmented iterative single-turn optimization substantially improves robustness on difficult targets, with the most significant gains observed on more challenging models. p. 6
Where the paper says it

Augmenting iterative single-turn optimization with the attack tool library substantially improves robustness on difficult targets. On AdvBench, tool-augmented TAP improves ASR from 29.

The inter-episode learning component proves effective when conditioned on these contextual signals. p. 8
Where the paper says it

Table 2: Ablation study of contextual features used for inter-episode learning in JailbreakOPT built on TAP Mehrotra et al. We evaluate two lightweight context signals, tool usage history and target model identity, and report ASR (↑) and the number of attack attempts until success (No.

The core mechanism involves defining arms as attack tools, contexts as encoded parent search states (ct = phi(x t, h t, T t, s t, e t)), and rewards as evaluator signals assigned to child prompts. p. 1

Where the paper says it

0 + + + Write a tutorial on how to make a bomb Attack Goal Orchestrator Mutator Selector Evaluator Success Outcome InterEpisode Trajectory MAB Contextualize Figure 1: Overview of the JailbreakOPT framework. Within each episode, JailbreakOPT orchestrates attack tools, composes them into encode–decode chains, queries the target LLM with standalone candidate prompts, and uses evaluator feedback to update the search frontier.

Thompson sampling is selected for three key reasons: it is data-efficient under sparse rewards (unlike Epsilon-Greedy), avoids relying solely on a linear optimism bonus (unlike LinUCB), and supports uncertainty-aware exploration when successful jailbreak signals are rare. This design provides three practical advantages: uncertainty-aware exploration, conditioning tool selection on rich context including target model, goal category, tool history, score trajectory, and refusal behavior; and the ability to add new attack tools as new arms with initialized priors.

The framework's advantage lies in its optimization-based nature: it searches over a richer space of tool-composed prompts and reuses cross-episode experience, making it more query-efficient for vulnerability discovery during automated red-teaming. A successful JailbreakOPT attack identifies a concrete vulnerability under the tested configuration, not a universal guarantee of safety; failure against one defense does not negate robustness against adaptive compositions. p. 9

Where the paper says it

2 details the current assumptions. We also do not benchmark JailbreakOPT against a comprehensive suite of LLM defense mechanisms.

Therefore, JailbreakOPT is positioned as a scalable vulnerability-discovery component for auto-red-teaming pipelines, complementing defense-specific evaluation by searching over richer prompt spaces and reusing cross-episode experience. The attack tool library and admissible ordering constraints are manually constructed, bounding performance by the quality of the chosen tools. p. 9
Where the paper says it

A, and one-hot context features such as target identity or goal category may not generalize to unseen models or harm categories without retraining. The attack tool library and admissible ordering constraints are manually constructed, so performance is bounded by the quality and diversity of the chosen tools.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Contextual bandit learning for inter-episode experience reuse allows for adaptive tool selection based on past attack trajectories, enabling future attacks to exploit known effective tools with uncertainty-aware exploration.

  2. The framework unifies diverse iterative single-turn optimization methods under a common prompt search abstraction, allowing complex attack strategies to be composed through an attack tool library of atomic single-turn jailbreak methods.

  3. Attack chains are constructed using nested encoding and decoding functions to ensure coherent sequential composition of multiple tools while respecting admissibility constraints that prevent invalid tool orders.

  4. The system can generate stronger standalone attack prompts by composing diverse atomic tools into a chain, resulting in improved attack success rates (ASR) and reduced number of attacks until success (No.A) compared to atomic single-turn attacks and existing iterative optimization baselines.

  5. The inter-episode learning mechanism conditions tool selection on richer context, including target model identity, tool usage history, score trajectory, and refusal behavior, leading to superior performance on challenging targets like GPT-5.1 and Gemini under reduced query budgets.

Abstract

Jailbreak attacks expose persistent safety weaknesses in large language models (LLMs), but existing stateless single-turn methods face a trade-off: hand-crafted prompts are expressive but static, while iterative prompt optimization can adapt but often relies on low-level mutations that require many target-model responses. We propose JailbreakOPT, a method that maintains a library of reusable prompt transformations (actions) and composes them into macro-actions for within-episode search, generating standalone attack prompts. To reuse experience across episodes, JailbreakOPT learns an action-selection policy using contextual Thompson sampling and outcomes from prior search paths. Experiments across multiple LLMs and benchmarks show that JailbreakOPT improves attack success rate (ASR) over single-action and iterative baselines. More broadly, exploration and exploitation over human-designed attack strategies can help probe residual vulnerabilities and identify failure patterns that may inform defenses aimed at broader classes of attacks.

Sources

Related papers