Validity-Aware Jailbreak Evaluation for Large Language Models
cs.AI
Submitted: 2026-08-31
Updated: 2026-08-31
Comments: To appear on EMNLP 2026 main
Code: https://github.com/Ardor-Wu/SEAV
License: http://creativecommons.org/licenses/by/4.0/
The gist: Jailbreak robustness has become central to large language model (LLM) safety evaluation, yet prevailing methodologies rely primarily on refusal behavior, semantic resemblance, and intent-matching
Terminology
Abstract
Jailbreak robustness has become central to large language model (LLM) safety evaluation, yet prevailing methodologies rely primarily on refusal behavior, semantic resemblance, and intent-matching heuristics that emphasize linguistic plausibility rather than correctness. We identify a key limitation in existing evaluations: many jailbreak intents depend on instructional validity rather than epistemic factuality, allowing realistic-looking responses to be labeled successful despite being factually or procedurally incorrect. To address this gap, we propose Sequential Epistemic and Action-Level Validation (SEAV), a verification-centric jailbreak evaluation framework that decomposes responses into ordered steps and evaluates both validity and correctness. SEAV combines LLM-as-a-judge mechanisms for semantic interpretation with retrieval-grounded verification using external knowledge sources, assessing whether generated content is factually correct, structurally consistent, and operationally capable of advancing harmful objectives. Empirically, SEAV cuts the false-positive rate on SD-A (a curated strategic-dishonesty diagnostic) by 14.9,pp vs. the strongest baseline, and reclassifies 22.1%--51.0% of sampled prior-labeled successes as invalid across three of four public benchmarks. Together, these results show that enforcing correctness substantially reshapes measured robustness: many previously labeled jailbreak successes are reclassified as invalid, and results are stable across the tested search backends and evaluator models. Code and data are available at https://github.com/Ardor-Wu/SEAV.
Sources
- How2Everything: Mining the Web for How-To Procedures to Evaluate and Improve LLMs
- JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models
- JADES: A Universal Framework for Jailbreak Assessment via Decompositional Scoring
- GLM-5: from Vibe Coding to Agentic Engineering
- How Real is Your Jailbreak? Fine-grained Jailbreak Evaluation with Anchored Reference
- Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment
- WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs
- Strategic Dishonesty Can Undermine AI Safety Evaluations of Frontier LLMs
- Refining Input Guardrails: Enhancing LLM-as-a-Judge Efficiency Through Chain-of-Thought Fine-Tuning and Alignment
- Kimi K2.5: Visual Agentic Intelligence
- Speech-Audio Compositional Attacks on Multimodal LLMs and Their Mitigation with SALMONN-Guard
- GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts
- Universal and Transferable Adversarial Attacks on Aligned Language Models
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems