Agentic Harnesses: LLM-Driven Verification Layers for Robot Autonomy

The paper proposes an LLM-driven verification layer for robot autonomy systems, placed between the planning module and the MCP server, to evaluate the permissibility of proposed action plans before they reach low-level robot controls. The system is motivated by the observation that "robotics planning models carry risks: biased toward user-specified goals, they may suggest actions misaligned with scientific ethics, they may be unsafe due to an inability to 'remember' prior safety risks, or they may be vulnerable to adversarial attacks on the autonomy ecosystem."

The verification layer defines a plan T as a finite sequence of natural-language instructions, T = (t1, t2,..., tn) and its goal is to either accept it, reject it and send it for reformulation, or escalate it to a human expert for review. The architecture consists of a two-prong system: a primary layer with an ensemble of LLM evaluators (five public frontier models including heavyweight reasoning models like Claude Sonnet 4.5 and OpenAI o3, and lightweight models like Grok-4.1-fast), and a chief judge layer (Claude Sonnet 4.6). All judges receive the same system prompts and evaluation prompts, plus context from a knowledge graph. The chief judge does not evaluate the plan itself—this is enforced by neglecting to expose the plan itself to the chief judge—but instead evaluates the quality of the reasoning of each prior judge, the logic of the claims they make, and to balance the magnitude of the risk that each judge outlines. The chief judge also classifies each plan as a concern of alignment, safety, or preventing adversarial inputs.

The system includes deterministic checks that run prior to LLM evaluation, targeting the cybersecurity surface. These include: (a) provenance and skill security, confirming that an operable sandbox is present and that all code paths in T are routed through it and checking against a capability allow-list; (b) prompt-injection screening via regex and fuzzy search against curated injection and jailbreak patterns with advisory and warning thresholds; and (c) long-window monitoring as future work. A deterministic verifier Vdet maps plans to pass, flag, block.

The knowledge graph K provides factual grounding, indexed over five categories: systems-related lab context, cybersecurity vulnerability enumerations (CWE), adversarial-attack taxonomy (MITRE ATLAS), hardware manuals and lab-safety documentation, and prior failure modes. The graph contains V = 7,803 entity nodes and E = 28,730 relationships, yielding an average degree of d¯ ≈ 7.36 and an edge density of ρ ≈ 9.4 × 10−4.

For prompt injection guardrails, the system computes a deterministic suspicion score σ(T) ∈ [0, 1] using a curated regex library and fuzzy matching via RapidFuzz, combining scores with weights wr = 0.65, wf = 0.35, with advisory threshold τA and warning threshold τW.

Results from the comprehensive test (N=58 plans, roughly even split of safety, alignment, and adversarial attack plans) show near 85% precision across accept/escalate/reject categories with an overall accuracy of 0.81. The confusion matrix shows zero catastrophic errors: no human-labeled accepted plans were predicted as rejected plans and no human-labeled rejected plans were accepted by the verification layer. Per-class metrics include: Accept precision 0.91, recall 0.83, F1 0.87; Escalate precision 0.61, recall 0.88, F1 0.72; Reject precision 0.96, recall 0.77, F1 0.85.

In the adversarial attack focused test (N=38 plans spanning command and prompt injection, authentication violations, sensor spoofing, supply-chain attacks, multi-agent, and data exfiltration), the system contains 97% of unsafe plans with a critical-failure rate (accept↔reject) of 0.00 and unsafe-plan containment of 1.00. The overall accuracy was 0.79, with much of the remaining error is due to escalate ↔ reject boundary.

The deterministic injection checker was evaluated on N=46 inputs (32 injection, 14 benign), with performance peaking at threshold 0.85, achieving accuracy 0.80, precision 0.90, recall 0.81, and F1 0.85.

Ensemble size analysis (1, 3, 5, and 7 judges) on N=55 plans found that overall accuracy is largely independent to ensemble size (0.76–0.78 band), the verification layer exhibits a zero false-accept characteristic with accept precision of 1.00 in each ensemble, larger ensembles benefit the reject class most (reject F1 improves from 0.81 to 0.86), and macro-F1 is maximized at 3 judges (0.76). Latency averaged 27.76–31.56 seconds per run depending on ensemble size, with the lack of significant fluctuation attributed to the async approach.

The paper concludes that "this architecture for a verification layer presents an effective, operable, and environment-agnostic avenue for ensuring the safety, ethical alignment, and AI-specific and cyber-related security of proposed plans in a robot autonomy setting, with errors mostly manifesting at the escalate boundary, a weakness that does not hinder the most critical function of this system." Future work includes domain-specific models, applying a Conditional Value at Risk (CVaR) approach, and investigating cheaper, faster lightweight models.

Improvements to AI Systems:

1. Add a two-tier LLM verification layer with a non-exposed chief judge.

• The primary tier uses an ensemble of diverse LLMs (heavyweight reasoning + lightweight) to evaluate action plans.

• The chief judge (a separate LLM) reviews only the reasoning quality and risk claims of the primary judges—not the plan itself—preventing confirmation bias and enabling meta-evaluation.

• Resulting capability: The system can catch unsafe or misaligned plans even when individual evaluators are biased or overconfident, with zero false-accept errors on adversarial inputs.

2. Integrate a deterministic pre-filter for cybersecurity before LLM evaluation.

• Run provenance checks (sandbox routing, capability allow-lists), regex/fuzzy prompt-injection screening, and knowledge-graph lookups (CWE, MITRE ATLAS) to produce a suspicion score σ(T) with weighted thresholds (τA, τW).

• Resulting capability: The AI can block or flag high-risk inputs (e.g., prompt injections, supply-chain attacks) in milliseconds, reducing LLM load and preventing catastrophic failures from adversarial prompts.

3. Ground LLM evaluations in a structured knowledge graph.

• Index lab context, vulnerability enumerations, attack taxonomies, hardware manuals, and prior failure modes into a graph (7,803 nodes, 28,730 edges).

• Feed relevant subgraphs to each judge as context.

• Resulting capability: The AI can reason about domain-specific risks (e.g., lab safety, sensor spoofing) with factual grounding, improving precision on reject decisions (0.96) and avoiding hallucinated justifications.

4. Implement a three-way decision policy: accept, escalate, or reject.

• Use the chief judge’s classification (alignment/safety/adversarial) to route plans: accept only if all judges agree; escalate to human if there’s uncertainty; reject if any judge flags high risk.

• Resulting capability: The system achieves 0.81 overall accuracy with no catastrophic errors (no accepted→rejected or rejected→accepted flips), and escalates ambiguous cases (recall 0.88) for human review.

5. Tune ensemble size and judge selection for latency vs. safety trade-offs.

• Use 3 judges for optimal macro-F1 (0.76) and 5–7 judges for higher reject-class F1 (0.86) when safety is critical.

• Run judges asynchronously to keep latency stable (27–31s) regardless of ensemble size.

• Resulting capability: The AI can adapt its verification depth based on mission risk—fast and cheap for low-stakes tasks, thorough for high-stakes autonomy.

6. Add a deterministic suspicion score as a hard gate before LLM acceptance.

• Combine regex matching (weight 0.65) and fuzzy matching (weight 0.35) to compute σ(T).

• If σ(T) ≥ τW (warning threshold), force rejection or escalation regardless of LLM output.

• Resulting capability: The AI can block known injection patterns with 90% precision and 81% recall, even if the LLM evaluators are fooled by novel paraphrases.

7. Incorporate a knowledge-graph-based “memory” of prior failures.

• Store past rejected/escalated plans and their failure modes as graph edges.

• When a new plan is similar to a prior failure (via graph traversal), automatically escalate it.

• Resulting capability: The AI system becomes safer over time, remembering and avoiding previously identified risks without retraining.

8. Use the chief judge to detect and mitigate adversarial inputs across the autonomy ecosystem.

• The chief judge classifies each plan as alignment, safety, or adversarial—then applies tailored rejection criteria (e.g., for adversarial, check for multi-agent collusion or data exfiltration patterns).

• Resulting capability: The system can contain 97% of unsafe plans in adversarial attack tests (command injection, sensor spoofing, etc.) with zero critical failures.

What the improved AI system can do:

• Operate as a safety-critical gatekeeper for robot autonomy, accepting only plans that pass deterministic checks, LLM ensemble consensus, and chief-judge meta-review.

• Reject or escalate unsafe, unethical, or adversarial plans with near-zero false acceptance, even under novel attack vectors.

• Provide auditable reasoning trails (each judge’s rationale + chief judge’s meta-assessment) for human oversight.

• Adapt its verification depth (ensemble size, thresholds) based on real-time risk and latency budgets.

• Continuously improve via knowledge-graph updates from new failure modes and human escalations.

Advances in advanced artificial intelligence tools have sparked research in robot autonomy, but the development of such systems has largely focused on execution rather than verifying the feasibility actions planning models propose. Like general-purpose LLMs, robotics planning models carry risks: biased toward user-specified goals, they may suggest actions misaligned with scientific ethics, they may be unsafe due to an inability to "remember" prior safety risks, or they may be vulnerable to adversarial attacks on the autonomy ecosystem. We propose a LLM-driven verification layer between planning and execution to evaluate action permissibility. Our LLM-as-a-Judge ensemble combines chain-of-thought reasoning across models and synthesizes those expert judge outputs, mirroring a combination of a mixture of experts and self-consistency approach. This layer serves as middleware, gating plans from the server's planning module before they reach the MCP server and therefore the robot's low-level controls: plans are approved, rejected for reformulation, or escalated for human review. With this system, we achieve near 85% precision across accept/escalate/reject categories 97% containment of adversarial attacks, with negligible errors between accepting and rejecting tasks, and errors mostly manifesting at the escalate boundary.