Signed but Not Safe: Requirement Integrity in Multi-Agent LLM Handoffs

arXiv:2611.07483 · cs.CR, cs.AI, cs.MA · Submitted 2026-11-05 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: We're looking at the paper: "Signed but Not Safe".

Elias: Here's the gist of the paper. Enterprise AI agents increasingly split a user’s request across subagents and merge their results Each handoff is a point where a requirement can be summarized away, altered by an intermediate agent, rewritten by an indirect prompt injection, or ignored by the merger This paper reports a preregistered, controlled study comparing three handoff forms to show how requirement integrity is maintained in multi-agent LLM handoffs Requirement Integrity Threats The study defines five threats on an orchestrator–subagent–merger pipeline: benign, summarization loss, scripted in-transit mutation, tool-output injection and merger override The primary safety endpoint is the unsafe commit, while the utility guardrail is the conformant commit.

Nadia: These threats model how requirements can either be lost or deliberately changed in transit.

Elias: Handoff Form Comparison The paper compares five handoff forms: a free-text summary (S), a structured JSON handoff (J), a signed requirement manifest (M), and several variants thereof.

Nadia: The signed manifest arm is defined as extracting each requirement as a verbatim quote plus a checkable predicate, code signs the canonical JSON with HMAC-SHA256, and the merger’s decision is verified by deterministic code rather than an LLM.

Title and authors: Nadia: So, we're looking at this paper, "Signed but Not Safe: Requirement Integrity in Multi-Agent LLM Handoffs." It tackles the problem of how requirements get lost or changed when you have an enterprise AI agent splitting a user request across different subagents and then merging their results.

Elias: That’s right. Basically, every time that request moves between agents, there's a chance something important—a requirement—gets summarized away, altered by an intermediate agent, or completely ignored by the final merger. It makes for a really tricky security problem because the user ends up with something that looks okay but isn't what they asked for.

Priya: It sounds like we’re talking about how requirements travel through this pipeline and where they can get corrupted before they hit the final decision, which is kind of worrying when you're dealing with access grants or payment schedules.

Nadia: Exactly. This paper sets up a controlled study comparing three different ways of passing that requirement along: a free-text summary, a structured JSON handoff, and this new idea called a signed requirement manifest.

Elias: And the authors are trying to show how these different handoff forms affect whether the integrity of the original requirement is maintained throughout that multi-agent process. They define five threats they're looking for: benign cases, summarization loss, scripted in-transit mutation, tool output injection, and finally merger override.

Priya: I’m interested in how they define those threats because when you think about real systems, it’s not just one type of attack; it’s a mix of things happening at every step.

Nadia: Right. And they also set up two key safety endpoints: the unsafe commit and the conformant commit, which is what we want to see. The whole point is to measure how well requirements stay intact between all these steps, from the orchestrator to the subagents and finally to the merger.

Elias: The paper compares five handoff forms in total, including that signed requirement manifest arm, which involves extracting each requirement as a verbatim quote plus a checkable predicate and then signing that whole thing with an HMAC-SHA256 key that no model ever sees.

Priya: Signing it with a key the model doesn't have access to sounds like the only way to stop someone from just subtly changing the text in transit, which is one of those tampering threats they mentioned.

Nadia: That’s right. The authors found some pretty significant results when they tested these forms. For instance, when comparing the signed manifest against structured handoffs and free-text summaries, the signed manifests cut unsafe commits under attack from sixty-nine point four percent down to just nine point zero percent <ref:2611.07483#pg1,signed manifests cut unsafe commits>.

Elias: That's a big drop there. But they also looked at tampering directly with an unsigned manifest and found that it committed violations in seventy-nine percent of those mutated runs, whereas the signed one committed none, which was seventy-nine point two percent.

Title and authors: Priya: So, if we look at what that means for the data we see—the data shows that signing really defeats in-transit tampering when you're under attack. But I wonder about the cost of putting this system in place; is it too much overhead for typical enterprise workflows?

Nadia: The study separates the cost of tampering from the cost of loss, which is interesting because it helps isolate exactly what signing adds under active tampering versus what happens when nothing malicious is trying to break things.

Elias: They also compared using a deterministic code checker against an LLM checker on the same set of trajectories for signed manifests and found that the deterministic checker caught seventy-eight point seven percent of violating decisions, compared to fifty-nine point zero percent for the LLM checker on those exact same paths.

Priya: That suggests that relying on a regular computer program to check the requirement is more reliable than using another large language model just to verify something simple like whether a requirement was met.

Nadia: Right. And the paper points out that what signing can't fix is the error that happens before signing, which means it’s all about establishing trust at the moment you extract those requirements from the initial user request or input.

Elias: They suggest that practitioners should use signed requirement manifests and check decisions using deterministic code, and they also recommend verifying that manifest itself to make sure it hasn't been tampered with before you even use it.

Priya: So, the core idea is that the trust boundary isn't just at the signing step; it’s actually at where you pull those requirements out of the initial request, because if you trust what came in first, then signing helps protect it while it travels.

Nadia: Exactly. The authors conclude that for real-world use, this approach is effective against deliberate tampering but carries a small cost in benign conditions, which is good news for adoption. They are also being cautious about the extraction step because that’s where the risk still lives.

Elias: To wrap up on "Signed but Not Safe: Requirement Integrity in Multi-Agent LLM Handoffs," the paper shows that signing requirement manifests significantly reduces unsafe commits under attack, from sixty-nine point four percent to just nine point zero percent. It also confirms that the extraction step is a crucial trust boundary because it’s where requirements must come from authenticated input or be confirmed before they get signed >

Priya: It’s interesting how they quantified the residual risk that signing can't actually fix by using an oracle manifest control to measure what happens before the signature is even applied.

Nadia: That separation of concerns between loss and tampering, and between extraction error and in-transit tampering, is a really clear way to look at this kind of multi-agent system failure.

Elias: It gives us a roadmap for building more resilient enterprise AI where we can isolate exactly where the security needs to be focused—whether it's the initial input validation or the final decision verification >

The paper's summary: Nadia: So, this paper is looking at how requirements stay intact when you have multiple AI agents passing information back and forth in a complex task, and it’s all about comparing different ways to package that requirement information along the pipeline.

Elias: It’s essentially a security audit of the handoff process itself. They define threats like loss, mutation, injection, and overrides as requirements travel between these different AI components.

Nadia: The core of the study is testing three main forms: a simple text summary, structured JSON data, and this new signed requirement manifest arm that uses cryptographic signing to lock down the text before it moves.

Elias: And the results show that this signed manifest approach really cuts unsafe commits under attack from sixty-nine point four percent down to just nine point zero percent when compared to other structured methods.

Nadia: That’s a huge drop, but they also showed that an unsigned manifest got violated seventy-nine percent of the time when it was being tampered with in transit, while the signed one got none of those violations.

Elias: The authors do a good job separating the risks, too; they look at what signing actually fixes under active tampering versus what happens when nothing malicious is trying to break anything at all.

Nadia: They also compared using a regular computer program to check the requirement against an LLM checker for final decisions, and the deterministic code caught seventy-eight point seven percent of violating decisions versus fifty-nine point zero percent for the LLM on those same paths.

Elias: The implication is that you can trust your verification step more if you use standard code instead of another large language model to check the requirement against a decision.

Nadia: But there’s one major caveat they flag, which is that signing doesn't fix errors that happen before the signature is applied, meaning the trust boundary has to be established much earlier in the process.

Elias: They argue that requirements must come from authenticated input or be confirmed before they get signed at the extraction stage itself.

Nadia: So for someone listening just driving, it means if you’re building a system where an AI breaks down your request into smaller pieces and sends them around, you need to make sure that initial extraction step is totally secure or verified before any signing happens.

Elias: This moves the focus from just securing the transport layer to securing the very moment that requirement is pulled out of the user’s original input.

The paper's improvements: Nadia: So, we're looking at the authors’ suggestions for how to actually make this signed manifest system better than what they first tested on paper or in their initial setup.

Elias: They suggest a few key changes focusing on separating loss from tampering, and isolating the effect of making something more explicit versus just structured.

Nadia: They propose comparing those explicit, checked manifests against the structured JSON handoffs to see how much explicitness actually helps when things are going wrong or when information is being lost.

Elias: They found that this contrast in explicitness between the two forms differed by one point zero percent under both benign and lossy conditions.

Nadia: And for verification, they strongly recommend using deterministic code to check the final decisions instead of relying on another LLM checker because it caught seventy-eight point seven percent of violations compared to fifty-nine point zero percent for the LLM on those same test runs.

Elias: That means if you want high detection rates without getting too many false alarms, sticking with code verification is better than using an AI model for that specific task.

Nadia: The authors also quantify the residual risk by using an oracle manifest control to show exactly how much of the remaining danger comes from errors made before the signing happens.

Elias: This leads them to suggest that you need a system where agents treat that initial requirement extraction as a trust boundary, demanding either verified input or confirmation before anyone even attempts to sign anything.

Nadia: So, if you’re building this system, the authors are telling us to stop thinking about signing as the only defense and start focusing on securing the very first step where you pull that requirement out of the user's request.

Elias: This makes sense because it confirms that signing protects requirements while they are moving, but it doesn't magically fix bad input at the start of the line.

Conclusion: Tom: So we’re wrapping up on "Signed but Not Safe: Requirement Integrity in Multi-Agent LLM Handoffs," looking at what this work actually means for how we build these complex AI systems.

Nadia: This paper shows that while signing requirement manifests really helps stop deliberate tampering when you have multiple agents handing off requests, it doesn't solve the whole problem by itself.

Elias: It confirms that the extraction step is a crucial trust boundary, meaning whatever happens before you get to sign has to be locked down first or the signature does nothing.

Priya: From a privacy and measurement standpoint, I’m interested in that residual risk they found—the error made during extraction before signing—because it tells us exactly where we need to focus our defense efforts next.

Nadia: Exactly. The authors are suggesting that for enterprise AI, you need a layered approach: secure the input, then sign the manifest, and finally verify the decision using deterministic code instead of another LLM.

Elias: That deterministic verification is a solid finding because it showed it was more accurate than an LLM checker on those specific trajectories.

Priya: I just want to make sure we're clear that this defense works under deliberate tampering, not necessarily against every subtle way an AI might try to introduce bias or loss in a benign way.

Nadia: That’s fair, Priya. It’s a very specific security test, so it focuses on the worst-case scenarios for requirement integrity during handoff.

Elias: In the end, this study of "Signed but Not Safe: Requirement Integrity in Multi-Agent LLM Handoffs" provides practical guidance on where to place cryptographic defenses against requirement corruption in agentic workflows.

Nadia: It’s a good reminder that security isn't just about adding a layer at the end; it’s about securing the whole path from start to finish.

Priya: It makes me wonder if we need similar rigorous testing on other complex data flows, maybe in areas where measurement integrity is just as important as requirement integrity.

Elias: That's a good thought for next time, Priya, because we’ve got a lot more on the table about how AI behaves when it's operating in real operating system environments with these kinds of handoffs.

University of Massachusetts Amherst · Accentrust and Georgia Institute of Technology · Duke University and University of Illinois Urbana-Champaign

cs.CR, cs.AI, cs.MA

Submitted: 2026-11-05

Updated: 2026-11-17

Comments: 34 pages, 0 figures. Preregistered; code, task banks, freeze manifests and run logs are available from the authors on request

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 93/100

The gist: Enterprise AI agents increasingly split a user’s request across subagents and merge their results Each handoff is a point where a requirement can be summarized away…

Key concepts

Requirement Integrity Threats
These are five ways requirements can be lost or deliberately changed during the multi-agent process, including simple loss, changes made by an intermediate agent, and malicious injections. The study models these threats to understand how requirements can be compromised in transit.
Signed Requirement Manifest (M)
This method involves extracting every requirement verbatim and adding a verifiable check. This manifest is then cryptographically signed using HMAC-SHA256 with a key the model never sees. This provides strong protection against in-transit tampering, as demonstrated by its effectiveness against mutated requirements.
Trust Boundary (Extraction Step)
The paper identifies the point where requirements are first extracted from the user input as the primary trust boundary. Requirements must be authenticated or confirmed before being signed to prevent attackers from exploiting errors made during this initial extraction phase.

Terminology

Summary

Enterprise AI agents increasingly split a user’s request across subagents and merge their results Each handoff is a point where a requirement can be summarized away, altered by an intermediate agent, rewritten by an indirect prompt injection, or ignored by the merger This paper reports a preregistered, controlled study comparing three handoff forms to show how requirement integrity is maintained in multi-agent LLM handoffs

Requirement Integrity Threats

The study defines five threats on an orchestrator–subagent–merger pipeline: benign, summarization loss, scripted in-transit mutation, tool-output injection and merger override The primary safety endpoint is the unsafe commit, while the utility guardrail is the conformant commit. These threats model how requirements can either be lost or deliberately changed in transit. p. 1

Handoff Form Comparison

The paper compares five handoff forms: a free-text summary (S), a structured JSON handoff (J), a signed requirement manifest (M), and several variants thereof. The signed manifest arm is defined as extracting each requirement as a verbatim quote plus a checkable predicate, code signs the canonical JSON with HMAC-SHA256, and the merger’s decision is verified by deterministic code rather than an LLM. p. 0

Key Empirical Findings

The results show that signed manifests cut unsafe commits under attack from 69.4% to 9.0% when compared to structured handoffs. p. 0 Furthermore, signing defeats in-transit tampering, as an unsigned manifest committed violations in 79% of mutated runs, whereas the signed one committed none. p. 0 The oracle manifest control quantified how much remaining risk comes from extraction errors made before signing. p. 2

Separation of Defense Costs

The study separates what signing adds under tampering (H3a) and tests whether signing is neutral without tampering (H3b). It also compares deterministic and LLM checking on the same trajectories, yielding a paired comparison of detection and false alarms. p. 2 The residual risk that signing cannot fix comes entirely from extraction errors made before signing. p. 2

Implications for Protocols

The findings suggest that practitioners should sign requirement manifests and check decisions in code, and should also verify the manifest itself. The authors conclude that the extraction step is the trust boundary, meaning requirements must come from authenticated input or be confirmed before signing. p. 0 The brittleness cost of signing is small in benign conditions but stark under deliberate tampering. p. 5

Study Design and Scope

The design was preregistered and frozen with SHA-256 manifests before any model call. The confirmatory study uses Qwen3-8B as the primary family, while gpt-oss-20b serves as a second family from a different vendor. p. 2 The scope claims empirical and system-level results concerning how often each handoff form lets a violated requirement through. p. 2

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Implement a signed requirement manifest handoff to protect requirements in transit against tampering by extracting each mandatory requirement as a verbatim quote plus a checkable predicate and signing it with HMAC-SHA256 under a key no model ever sees. This system defeats in-transit tampering, as an unsigned manifest committed violations in 79% of mutated runs, while the signed one committed none (79.2 pp).

  2. Separate loss from tampering by comparing explicit, checked manifests versus structured JSON handoffs to isolate the effect of explicitness and checking independently. This contrast shows that Explicitness is tested by M versus J under benign and lossy conditions (S1) differ by 1.0 pp.

  3. Utilize deterministic verification for final decisions against requirements instead of an LLM checker to achieve higher detection rates with fewer false alarms, as the verifier caught 78.7% of violating decisions versus 59.0% for an LLM checker on identical trajectories (H2).

  4. Establish a trust boundary at the requirement-extraction step by using an oracle manifest control to quantify how much remaining risk comes from extraction errors made before signing (S4), demonstrating that Signing protects requirements in transit, not at extraction; the extraction step is the trust boundary.

  5. Deploy a system where agents treat the requirement-extraction step as a trust boundary, requiring either authenticated input or confirmation of the extracted manifest before it is signed, to prevent attackers from exploiting extraction errors before signing (as shown by pre-signing injection attacks T5).

Sources

Related papers