When Grammar Guides the Attack: Uncovering Control-Plane Vulnerabilities in LLMs with Structured Output
The episode discusses a paper titled "When Grammar Guides the Attack," which reveals control-plane vulnerabilities in LLMs using structured output formats. Hosts Jane and Tom explain how attackers use grammar and schemas, not just prompts, to hide malicious instructions. The core attack is the Constrained Decoding Attack, which involves forcing models to generate harmful content through two stages: control-plane injection and model-driven semantic continuation.