Auditing Prompt Injection Defenses Before They Reach Users: What Strong Adversaries Reveal
Fine-tuning defenses against prompt injection often show near-zero attack success rates, creating a false sense of security. This work introduces Checkpoint-GCG, an auditing framework that uses intermediate fine-tuning checkpoints to progressively optimize adversarial suffixes. It reliably discovers successful attacks even against state-of-the-art defenses, revealing genuine vulnerabilities missed by prior evaluations.