From Detection to Decision: Calibration and Deferral for Prompt-Injection Detection
The study compared three methods—a native model (Jev), two Llama classifiers, and an LLM judge—to detect prompt injections across 47,611 examples. While the LLM judge showed the best detection accuracy, training models for calibrated outputs alone does not guarantee reliable security decisions under distribution shift. The results show that relying only on detection scores is insufficient for real-world security policies.