LLMail-Inject: A Dataset from a Realistic Adaptive Prompt Injection Challenge
LLMail-Inject was a public challenge simulating adaptive indirect prompt injection attacks against an LLM-based email assistant. Participants used various strategies to bypass multiple defenses across different retrieval levels and LLMs. The resulting dataset provides crucial insights into the instruction-data separation problem, showing that end-to-end attacks are rare but highlight the need for robust, layered defenses against sophisticated adversaries.