Decision Models as Policy Gates for Coding Agents: Explicit Policy Boundaries Improve Detection and Reduce False Alarms
cs.CL, cs.CR, cs.SE
Submitted: 2026-11-01
Updated: 2026-11-01
Comments: 15 pages, 2 figures. Dataset, rule texts and harness available at https://github.com/agentapprove/decision-policy-rules
Code: https://github.com/agentapprove/decision-polic
License: http://creativecommons.org/licenses/by/4.0/
The gist: Coding agents can destroy data, leak secrets and message the wrong person faster than a human can review.
Terminology
Abstract
Coding agents can destroy data, leak secrets and message the wrong person faster than a human can review. Could a decision model catch these risks at runtime? We evaluated 22 hosted decision models and two prompted general models as classifiers for a multi-rule policy engine. A synthetic dataset of 2,643 tool calls was labeled blind against 28 rules, with a sealed holdout. With a single shared preamble in both conditions, explicit boundaries written as examples and counterexamples raised recall by 5.5-26.1 percentage points on all 23 available models and reduced false positives on 21. Jev 1.14's recall rose from 89.3% to 98.2%, while unnecessary interruptions fell from 30.6% to 4.8% of 252 clean actions. With examples, its median latency was 120 ms at 0.46 per thousand requests. Strong models retained the recall gains when concrete names were replaced by generic descriptions; three largely retained them when the same boundaries were written as prose. On 16 new scenarios written after the initial studies, a follow-up improved recall on 22 of 23 models and reduced unnecessary interruptions on 21. We release the dataset, rule texts and harness.
Sources
- Evaluating Decision Models for Text Annotation in Computational Social Science
- Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
- Measuring the Permission Gate: A Stress-Test Evaluation of Claude Code's Auto Mode
- The Agent Incident Registry: Toward Preventing Repeated AI Agent Failures
- Evaluating System One Models for Agent Security Decisions: Reliability, Calibration, and Selective Automation
- CARE: Pre-Execution Command Verification for Shell-Executing LLM Agents
- ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback
- Benchmarking System One decision models against trained classifiers and language models for automated decision gates
- TypedBench: A Benchmark for Calibration, Framing Sensitivity, and Cost in System One Decision Models
- AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents
- GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning
- JevOut: Natural Context Can Flip Decision Models
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data