AutoDojo: A Generative Benchmark for Evaluating Prompt Injection Defenses in LLM Agents
AutoDojo is an adaptive framework that uses a cheap, black-box attack to test prompt injection (IPI) defenses against LLM agents. It iteratively optimizes an injection strategy by observing success rates, finding that many existing defenses are brittle and offer limited protection when facing this dynamic threat.