Towards Assurance Closure in AI-Native Large-Scale Agile Software Development

Author: Ricardo Britto (Ericsson, Sweden; Blekinge Institute of Technology, Sweden)

The paper addresses the unresolved verification-first assurance principle of the AI-Native Manifesto, which envisions "large-scale agile software development in which humans progressively shift from implementation work to supervisory control, while remaining responsible for intent, risk, policy, and exceptional decisions, as agents assume increasing engineering authority." The author develops this requirement as assurance closure: "the ability to establish what must be true, determine and obtain appropriate evidence, judge the credibility of that evidence, preserve its validity as the system changes, and use the resulting uncertainty to bound agent authority."

The paper is explicitly positioned as a position paper and research agenda rather than a proposal for a new verification method. The central research challenge is "to make the meta-assurance process that selects, connects, interprets, and maintains them sufficiently machine-operable for AI-native R&D. The underlying premise is that realizing the AI-native end-state requires more than better code generation: it requires assurance closure."

From a review of adjacent work (requirements engineering, formal methods, assurance cases, digital twins, runtime assurance, and industrial experience such as Amazon's ShardStore), the paper identifies six residual gaps ordered as a stepwise transformation toward greater delegated authority:

GAP 1—Autonomous specification adequacy. Before delegation, the system needs to know whether the available description of the intended behavior is good enough to act on. This goes beyond translating prose into formal language; requirements may be incomplete, mutually inconsistent, too weak to distinguish correct from incorrect behavior, or based on assumptions that have never been made explicit. The gap is the ability to detect that the current specification is not yet adequate for delegation and to identify what is missing before implementation proceeds.

GAP 2—Assurance strategy synthesis. Once behavior is clear, the system must decide what evidence should justify a particular change. Different properties call for different techniques: a local invariant may be suited to formal proof, an API contract to static or property-based checks, a distributed failure scenario to simulation or fault injection, and an operational property to runtime monitoring. What remains largely human work is constructing a proportionate assurance plan for each change: deciding which claims matter, how strongly they must be supported, which evidence can be reused, and where human judgment is still required.

GAP 3—Scalable assurance evidence generation. An assurance plan is only useful if the required evidence can actually be produced at AI-native speed. Existing methods already exist (theorem proving, model checking, static analysis, testing, fuzzing, simulation, emulation, executable reference models, digital twins, runtime monitors), but the unresolved issue is how an agentic system can instantiate these methods at the right scope and fidelity for the change at hand. The system may need to construct a focused executable environment that contains only the services, state, workloads, faults, and assumptions needed to challenge the affected behavior, and such environments must itself be credible for the claim being evaluated. The research problem is scalable, change-focused generation of assurance evidence, not any single mechanism such as a digital twin.

GAP 4—Evidence credibility and sufficiency. Producing evidence is not the same as having good reason to believe a claim. Agentic development adds a complication: the code, specification, tests, simulations, and evaluations may share the same model family, retrieved context, or mistaken interpretation. Several apparently independent artifacts may therefore repeat the same error. The gap is the ability to assess whether a body of evidence is relevant, sufficiently strong, sufficiently independent, and free of unresolved defeaters for the decision being made, which should not be reduced prematurely to a single synthetic confidence score.

GAP 5—Assurance validity under continuous change. Assurance is never permanent. A proof may depend on an interface assumption that later changes; a test may no longer represent production behavior; a simulation model may become stale. AI-native R&D raises "the required speed and granularity: each software or environment change should automatically reveal which claims, assumptions, models, tests, proofs, and observations are affected, which evidence remains reusable, and what must be regenerated. Without this capability, faster generation simply creates an assurance bottleneck downstream."

GAP 6—Assurance-governed delegation and supervisory control. This is a governance question: what should agents be allowed to do with the assurance available now? The ability to propose a change, edit code, merge, release, or deploy should depend on the current assurance state and the consequences of being wrong. Humans must retain enough visibility to understand unresolved uncertainty, intervene in exceptional cases, and reduce or revoke authority. The gap is not simply about more automation, but about evidence-based, reversible delegation with meaningful human supervision.

The paper proposes a high-level architecture in which "assurance becomes a continuous control function around agentic R&D," supported by a shared semantic assurance layer that "provides shared, machine-readable knowledge linking intent, claims, assumptions, risks, architecture, implementation, dependencies, assurance activities, evidence, provenance, defeaters, validity conditions, runtime observations, and validated engineering experience."

C1–Specification Assurance (addresses GAP 1): turns human intent, requirements, constraints, and known system behavior into machine-actionable obligations, but its key responsibility is to challenge them rather than merely formalize them. It identifies ambiguity, missing assumptions, conflicts, weak constraints, and important behavior that remains unspecified. Its output is a specification state that is sufficiently complete and explicit for the next level of delegation, together with any unresolved questions that still require human input.

C2–Assurance Strategy Synthesis (addresses GAP 2): given the affected claims, architecture, risk, and uncertainty, it decides what evidence should be produced and at what depth. It is deliberately method-neutral: its purpose is to construct and continuously adapt an assurance strategy rather than to privilege a specific technology.

C3–Assurance Execution Fabric (addresses GAP 3): turns that strategy into evidence by orchestrating heterogeneous tools and agents, scoping them to the affected parts of the system and constructing the required validation context. A central research concern is efficiency: the fabric should generate enough evidence to challenge the relevant claims without reproducing or re-verifying the entire system for every change.

C4–Evidence Adjudication (addresses GAP 4): interprets the evidence produced by C3 rather than treating tool success as proof of correctness. It reasons about relevance, coverage, provenance, method soundness, shared dependencies between evidence sources, uncertainty, and unresolved defeaters, and produces a structured assurance judgment that explains why the available evidence is or is not adequate for the decision under consideration.

C5—Continuous Assurance State (addresses GAP 5): maintains the dependencies between intent, claims, assumptions, system elements, assurance activities, evidence, and runtime observations. When something changes, it determines which assurances remain valid, which have become stale, and which parts of the assurance strategy must be rerun, aiming for incremental reassurance: preserve strong evidence when its assumptions still hold, while precisely invalidating the evidence that no longer applies.

C6–Delegation and Supervisory Control (addresses GAP 6): maps the current assurance state to concrete engineering permissions and escalation rules. An agent might be allowed to explore or prepare a change while prevented from merging; stronger evidence may permit merging but still require human approval for deployment. If evidence becomes invalid, authority should contract automatically. It exposes the remaining uncertainty and the reasons for restricting access to humans, enabling them to supervise intent, risk, and exceptions rather than re-performing all implementation work.

The architecture is a hypothesis about the capabilities required for assurance closure, not an implemented solution. The research agenda is organized around four questions:

RQ1–Specification adequacy for delegation: How can an agentic system determine that a specification is sufficiently complete, correct, consistent, and discriminating to justify a particular level of delegation? Research is needed to detect missing assumptions and weak constraints, generate counterexamples, compare independently derived interpretations, and decide when unresolved ambiguity requires human clarification rather than further autonomous work.

RQ2–Automated assurance strategy and evidence generation: Given a change, its affected claims, architecture, uncertainty, and consequence profile, how can the system construct a cost-effective portfolio of assurance activities and then instantiate it at the appropriate scope? This includes selecting among proof, analysis, testing, fuzzing, simulation, targeted executable environments, runtime monitoring, and human review, and determining the minimum credible context in which each selected technique can provide useful evidence.

RQ3–Evidence trustworthiness and sufficiency: How can evidence produced partly by agents be assessed for applicability, soundness, provenance, dependence on shared assumptions, coverage, and unresolved defeaters? The paper notes that agent-written tests, for example, are not automatically strong evidence; recent results show that increasing their volume may provide little improvement in repository-level issue resolution. Research should focus on when heterogeneous evidence genuinely corroborates a claim and when it merely reproduces the same interpretation through different artifacts.

RQ4–Continuous assurance and dynamic delegation: How can assurance dependencies be maintained incrementally so that change automatically invalidates stale evidence, triggers proportionate re-assurance, and adjusts agent authority? Delegation should be reversible: weakened evidence or violated assumptions should reduce permissible actions, while restored evidence may expand them. This also requires studying the information humans need to supervise the loop effectively without becoming a manual verification bottleneck.

Evaluation principle: A useful demonstrator should expose the same system to changes that differ in affected claims, uncertainty, and risk, deriving materially different and defensible strategies. Evaluation should consider "not only task success but also false assurance, defect and defeater discovery, assurance cost, unnecessary human intervention, re-assurance latency after a change, and the calibration between the assurance state and delegated authority."

The author concludes that the AI-native vision "ultimately depends on a change in the human role: people should be able to govern intent, acceptable risk, and exceptional decisions without having to reproduce every engineering action that agents perform. Verification-first assurance is therefore not a peripheral quality activity; it is a condition for responsible delegation. While existing verification and validation techniques provide many of the necessary building blocks, they do not yet provide assurance closure as an integrated, machine-operable capability. The central research challenge is to make the assurance reasoning around existing methods—what must be assured, what evidence is needed, whether the evidence should be believed, when it ceases to apply, and what authority it justifies—operate at the speed and scale of agentic development while preserving meaningful human control. Closing this gap is necessary for the full AI-native large-scale agile software development vision to move from increasingly capable code generation to trustworthy, bounded delegation of R&D authority."

Improved AI system: an assurance-governed agentic R&D platform

Rather than merely generating code faster, the system will treat assurance closure as a first-class control function. Concretely, it will:

• Challenge specifications before acting.

It will not just translate requirements into formal text. It will detect ambiguity, missing assumptions, inconsistencies, weak constraints, and unspecified behavior; generate counterexamples; compare independently derived interpretations of intent; and ask humans only when further autonomous work cannot resolve uncertainty.

Capability: refuse delegation until the specification is sufficiently complete and discriminating for the proposed level of authority.

• Synthesize a proportionate assurance strategy per change.

For each proposed change, it will decide which claims matter, which evidence is needed, which techniques are appropriate (proof, static analysis, testing, fuzzing, simulation, fault injection, runtime monitoring, human review), and at what depth. It will select a cost-effective portfolio rather than a fixed verification pipeline.

Capability: produce a defensible, method-neutral assurance plan that adapts to risk, affected architecture, and uncertainty.

• Generate evidence at the right scope and fidelity.

It will instantiate the assurance plan by constructing minimal credible environments—containing only the services, state, workloads, faults, and assumptions needed to challenge the affected behavior—without re-verifying the whole system for every change.

Capability: rapidly produce change-focused evidence via heterogeneous tools and agents while keeping evidence generation economical.

• Adjudicate evidence instead of trusting tool success.

It will treat “tests passed” or “proof found” as raw evidence, not as proof of correctness. It will reason about relevance, provenance, method soundness, coverage, shared dependencies between artifacts, and unresolved defeaters. Crucially, it will detect when code, tests, simulations, and evaluations share the same model family or mistaken interpretation and therefore repeat the same error.

Capability: provide a structured assurance judgment that explains why evidence is or is not sufficient, and flag false assurance.

• Maintain assurance validity under continuous change.

It will track dependencies between intent, claims, assumptions, architecture, implementation, assurance activities, evidence, and runtime observations. When any element changes, it will automatically determine which assurances still hold, which are stale, which evidence can be reused, and what must be regenerated.

Capability: incremental re-assurance—preserving strong evidence whose assumptions still hold and precisely invalidating only what no longer applies.

• Govern delegation dynamically and reversibly.

It will map the current assurance state to engineering permissions. It may allow an agent to explore or prepare a change, prevent merging when evidence is weak, permit merge with human approval for deployment, and automatically contract authority if evidence becomes invalid or assumptions are violated.

Capability: evidence-based, reversible delegation with meaningful human supervisory control—humans see the remaining uncertainty and the reasons for restrictions without becoming a manual verification bottleneck.

• Explain and supervise rather than black-box automate.

It will expose intent, risk, assumptions, evidence provenance, defeaters, and validity conditions in machine-readable and human-accessible form. Humans can govern intent, acceptable risk, and exceptional decisions without re-performing every engineering action.

• Evaluate itself on assurance outcomes, not just task success.

The improved system will be measured by false assurance, defect and defeater discovery, assurance cost, unnecessary human intervention, re-assurance latency after a change, and the calibration between the assurance state and delegated authority. A demonstrator will expose the same system to changes differing in affected claims, uncertainty, and risk, and derive materially different, defensible strategies.

In short: the improved AI system turns assurance from a peripheral quality activity into an integrated, machine-operable capability—enabling trustworthy, bounded delegation of R&D authority in large-scale agile software development.