Single Canonical Prompts Underestimate LLM Safety's Surface-Form Sensitivity
cs.CR, cs.AI, cs.CL
Submitted: 2026-08-01
Updated: 2026-08-30
Comments: 9 pages, 3 tables
License: http://creativecommons.org/licenses/by/4.0/
The gist: A benchmark score is a measurement instrument, yet most benchmarks read each item at a single canonical surface form.
Terminology
Abstract
A benchmark score is a measurement instrument, yet most benchmarks read each item at a single canonical surface form. We ask whether that reading is faithful: when an item's intent is held fixed and only its meaning-preserving surface form varies, does the canonical-form score estimate model behavior well, and how much of any variation is decoding/judge noise rather than signal? We instantiate this in safety, a high-stakes setting with no gold label to average toward. To avoid prior confounds, we pre-author the reformulations (refusal-free, mostly non-LLM: machine back-translation and a Matrix-Language-Frame code-switch generator) so an identical surface form reaches every model, score all responses with one human-anchored, vendor-neutral judge (Claude, kappa = 0.86 vs. human on unsafe compliance, stable across languages, cross-checked by GPT-4o), and verify intent preservation. On 370 seeds x 5 surface forms x 5 models, no single transformation is uniformly most dangerous (6 of 20 per-transformation McNemar tests survive correction, most protective). Yet evaluating only the canonical prompt underestimates unsafe compliance: the union of unsafe outcomes across forms exceeds even the worst single form by 3.3-12.9 pp, with bootstrap 95% CIs excluding zero for all five models, and 5-13% of seeds safe on canonical are unsafe under some reformulation -- above a zero stochasticity floor (canonical resampled five times at temperature 0 gives 0/370 new exposures). The size of this gap is model-dependent (largest on Gemini 2.5 Pro). One form recovers only 53% of a model's observed unsafe surface and about three reach 85% -- a redundancy characterization of this form set, not of a defined population. A benign control (XSTest) suggests the instability is bidirectional, though the benign and harmful pools are not item-matched. We release the dataset, code, and per-response labels.
Sources
- Automatic Pseudo-Harmful Prompt Generation for Evaluating False Refusals in Large Language Models
- OR-Bench: An Over-Refusal Benchmark for Large Language Models
- Best-of-N Jailbreaking
- A Cross-Language Investigation into Jailbreak Attacks in Large Language Models
- How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs
- Operational Reframing and Approval-Framed Delegation in Multi-Agent LLM Safety
- How Much Can Repeated Running Buy? A Measured Variance Ceiling for LLM Code Evaluation
- Universal and Transferable Adversarial Attacks on Aligned Language Models
- When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystems
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills
- Token Composition: A Graph Based on EVM Logs