How Many Iterations to Jailbreak? Dynamic Budget Allocation for Multi-Turn LLM Evaluation
cs.LG
Submitted: 2026-05-07
Updated: 2026-09-24
Code: https://github.com/Shai128/dapro
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Evaluating and predicting the performance of large language models (LLMs) in multi-turn conversational settings is critical yet computationally expensive; key events---e.g., jailbreaks or successful
Terminology
Abstract
Evaluating and predicting the performance of large language models (LLMs) in multi-turn conversational settings is critical yet computationally expensive; key events---e.g., jailbreaks or successful task completion by an agent---often emerge only after repeated interactions. These events might be rare and, under any feasible computational budget, remain unobserved. Recent conformal survival frameworks construct reliable lower predictive bounds (LPBs) on the number of iterations to trigger the event of interest, but rely on static budget allocation that prohibits adaptivity in multi-turn setups. To address this, we introduce Dynamic Allocation via PRojected Optimization (DAPRO), a theoretically valid dynamic budget allocation framework for bounding the time-to-event in multi-turn LLM interactions. We prove that DAPRO satisfies the expected budget constraint and provides distribution-free, finite-sample coverage guarantees without requiring the conditional independence between censoring and event times assumed by prior conformal survival approaches. A key theoretical contribution is a novel coverage bound that scales with the square root of the mean censoring weight over only a subset of the samples rather than the worst-case weight, yielding tighter guarantees than prior work. Furthermore, DAPRO can be employed to obtain unbiased estimates of population-level evaluation metrics, such as the jailbreak rate, under limited computing resources. Comprehensive experiments across agentic task success, LLM jailbreaks, toxic content generation, and RAG hallucinations demonstrate that DAPRO achieves coverage closer to the nominal level with lower variance than static baselines, while not exceeding the budget constraint in expectation.
Sources
- Automating Customer Service using LangChain: Building custom open-source GPT Chatbot for organizations
- Watermark in the Classroom: A Conformal Framework for Adaptive AI Usage Detection
- Red Teaming Language Models with Language Models
- Conformal Survival Bands for Risk Screening under Right-Censoring
- Doubly Robust Conformalized Survival Analysis with Right-Censored Data
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned
- The Llama 3 Herd of Models
- Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs
- Gemma 3 Technical Report
- Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
- Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control
- Conformal Risk Control for Non-Monotonic Losses
- Leave-One-Out Stable Conformal Prediction
- A Conformal Prediction Score that is Robust to Label Noise
- SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Models
- Universal and Transferable Adversarial Attacks on Aligned Language Models
- Certifying LLM Safety against Adversarial Prompting
- SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks
- Active Evaluation Acquisition for Efficient LLM Benchmarking
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks