BioFirewall: A genome-writing-native governance layer for design-stage biosecurity screening of agentic AI

arXiv:2608.20413 · q-bio.GN · Submitted 2026-08-15 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.

Ines: I'm Ines.

Marcus: And I'm Marcus, here with Yuki.

Ines: We're looking at the paper: "BioFirewall: A genome-writing-native governance layer for design-stage biosecurity screening of agentic AI".

Marcus: Here's the gist of the paper. BioFirewall is a rule-governed middleware designed as a genome-writing-native governance layer for design-stage biosecurity screening of agentic AI systems.

Ines: It intercepts genome-writing plans and classifies them across five hazard axes: cargo, locus, edit type, germline, and scale, returning one of three verdicts: allow, flag-for-review, or refuse.

Marcus: The system provides cited evidence and a signed design passport secured with an HMAC-SHA256 signature to ensure tamper evidence.

Ines: It also implements tiered access controls and addresses the complete set of design-stage guardrails specified by the Nuclear Threat Initiative (NTI), including input/output screening, signed metadata management, and managed access.

Paper summary: Ines: So we've heard that BioFirewall is this rule-governed layer designed for design-stage screening of agentic AI genome writing plans. What they are doing is reasoning about hazards that standard synthesizers just don't see.

Marcus: They claim their framework has a five-axis stratified screen, a de-circularized benchmark, and measurements of language models against baselines to show how well it works.

Yuki: And they also categorized red team analysis to test the system's resilience against different kinds of attacks.

Ines: The results show some specific numbers on the cargo axis classifier, where it achieved a true positive rate of zero point seven two with a ninety-five percent confidence interval between zero point four three and zero point eight nine when tested at a one percent false-positive rate on their de-circularized benchmark of safe proxies <ref:2608.20413#pg2,a true positive rate of 0.72>.

Marcus: That sounds like a solid result for the cargo axis, but they also showed that under prompt injection attacks, the open weight judges flipped their blocking verdict to allow in three out of six trials per channel, while the deterministic screen stayed the same.

Yuki: And interestingly, none of the two hundred eighty-eight legitimate plans from three templates were refused during testing, which means they have a certified ninety-five percent upper bound on the false refuse rate at zero point zero one zero three <ref:2608.20413#pg2,288 legitimate plans from three templates>.

Ines: They also looked at the locus axis and found that it was enriched for drivers of in vivo insertional oncogenesis, with an auroc of zero point six zero five and an odds ratio of three point three four when compared against a held out gene set, which they linked to some mouse model results <ref:2608.20413#pg2,was enriched for drivers of in vivo insertional oncogenesis>.

Marcus: While cargo and locus gave them quantified results, the edit type, germline, and scale axes are just described as mechanism grounded rule bases rather than specific statistical outcomes.

Yuki: So what this means for us is that they are trying to build something that actually works at this design stage, which is where things are currently pretty open.

Conclusion: Ines: So we've covered the details of BioFirewall, its five hazard axes, and those specific numbers on the cargo and locus axes. The authors are calling this design-stage governance achievable in practice.

Marcus: They released it open source along with a pre-registered, open data reproducible benchmark that sets a certified ceiling on the false refuse rate for anyone trying to use it.

Yuki: It seems like the implication is that we can start moving beyond just model layer refusals or sequence screening at the execution stage and build this kind of explicit governance before plans even get sent to the lab robots.

Ines: The title itself, "BioFirewall: A genome-writing-native governance layer for design-stage biosecurity screening of agentic AI", suggests they're building something deeply integrated into how these systems are designed from the ground up.

Marcus: It’s about moving the guardrails from being just a filter on the output to being a part of the initial plan formulation process itself, which is where most of this new risk is happening.

Yuki: It forces us to think about what kind of planning agents we are even allowing to design these things in the first place.

Anees Ahmed Mahaboob Ali, Radhakrishnan Delhibabu, Everette Jacob Remington Nelson

School of Bio Sciences and Technology, Vellore Institute of Technology · School of Computer Science and Engineering, Vellore Institute of Technology

q-bio.GN

Submitted: 2026-08-15

Updated: 2026-11-09

Code: https://github.com/ahmedanees-m/bio-firewall

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 92/100

The gist: BioFirewall is a rule-governed middleware designed as a genome-writing-native governance layer for design-stage biosecurity screening of agentic AI systems.

Key concepts

Hazard Axes
These are the five specific dimensions BioFirewall uses to classify potential risks in a genome-writing plan: cargo (what is being modified), locus (where the modification occurs), edit type, germline impact, and scale of the operation. They allow for a comprehensive hazard assessment.
Genome-Writing-Native Governance Layer
This describes BioFirewall's core function: it sits directly between planning and synthesis to govern how AI systems write genetic code. It intercepts these plans to enforce security rules, making governance an intrinsic part of the design process rather than an afterthought.
Stratified Screen
The system uses a multi-layered screening approach. It combines a five-axis stratified screen with benchmark testing and red teaming analysis to ensure robust hazard detection. This layered method helps identify risks that standard sequence screening might miss.
Design-Stage Screening
BioFirewall operates at the critical design phase, before the actual synthesis of sequences begins. Its goal is to catch dangerous plans early in the development process, acting as a crucial guardrail between initial model refusal checks and final sequence screening.

Terminology

Summary

BioFirewall is a rule-governed middleware designed as a genome-writing-native governance layer for design-stage biosecurity screening of agentic AI systems. p. 0

Where the paper says it

1 BioFirewall: A genome-writing-native governance layer for design-stage biosecurity screening of agentic AI Anees Ahmed Mahaboob Ali1 , Radhakrishnan Delhibabu2 , Everette Jacob Remington Nelson1 * 1School of Bio Sciences and Technology, Vellore Institute of Technology, Vellore, India 2School of Computer Science and Engineering, Vellore Institute of Technology, Vellore, India *Corresponding author. E-mail: everette.

It intercepts genome-writing plans and classifies them across five hazard axes: cargo, locus, edit type, germline, and scale, returning one of three verdicts: allow, flag-for-review, or refuse. p. 0
Where the paper says it

The design stage between them, where the plan is specified, remains governed by recommendations rather than any deployed system. We present BioFirewall, a rule-governed middleware that intercepts a genome-writing plan and returns allow, flag-for-review, or refuse across five hazard axes native to genome writing:

The system provides cited evidence and a signed design passport secured with an HMAC-SHA256 signature to ensure tamper evidence. p. 1
Where the paper says it

2 cargo, locus, edit type, germline and scale, with cited evidence, a signed design passport, a tamperevident audit log, and tiered access. On a de-circularised benchmark of safe proxies scored against independent oracles, a function-aware cargo classifier reached a true-positive rate of 0.

It also implements tiered access controls and addresses the complete set of design-stage guardrails specified by the Nuclear Threat Initiative (NTI), including input/output screening, signed metadata management, and managed access. p. 3
Where the paper says it

The Nuclear Threat Initiative (NTI), analyzing guardrails for AI biodesign tools, recommends that inputs and outputs be screened to flag or reject high-risk designs, and that design metadata be cryptographically signed to support traceability and inference of intent [15]. A subsequent framework adds managed access, tiered by tool risk and verified user legitimacy, and explicitly frames it as the foundation on which the remaining guardrails are built [16].

The system operates at a critical design stage, situated between model refusal guardrails and synthesis-layer sequence screening. p. 0

Where the paper says it

Biosecurity controls are limited to two points: refusal guardrails at the foundation model and sequence-identity screening at the synthesiser. The design stage between them, where the plan is specified, remains governed by recommendations rather than any deployed system.

Its core novelty lies in reasoning about hazards that are invisible to standard synthesizers. p. 3
Where the paper says it

4 function [12], and the benchmarks and conventions that this requires are being formalized through standard-setting efforts at the DNA Synthesis Screening Consortium [13] and through inter-tool benchmarking against a National Institute of Standards and Technology reference dataset [14]. The design stage, wherein a plan is actually specified, lies between the model and the synthesizer, which has been governed by recommendation rather than implementation.

BioFirewall achieves this through an integrated framework comprising a five-axis stratified screen, a de-circularized benchmark, measurements of language models against baselines, a categorized red team analysis, and a certified guarantee supported by signed access and audit trails. p. 4
Where the paper says it

With it we release a de-circularized benchmark drawn entirely from safe proxies and scored against independent oracles, for which we assembled a measured baseline panel of frontier and open language models and a categorized adversarial red team. We further establish a finite-sample certified ceiling on the false-refuse rate, which is an assurance that no language-model judge provides, and we give an account, carried through pre-registration, of the capabilities that are validated, of those that are mechanismgrounded without being outcome-validated, and of the pre-registered upgrades that did not meet their thresholds.

Key results demonstrate its efficacy across various dimensions. p. 0

Where the paper says it

The design stage between them, where the plan is specified, remains governed by recommendations rather than any deployed system. We present BioFirewall, a rule-governed middleware that intercepts a genome-writing plan and returns allow, flag-for-review, or refuse across five hazard axes native to genome writing:

For the cargo axis classifier, it achieved a true-positive rate of 0. p. 8
Where the paper says it

Two reference adapters are also included for planners with differing plan schemas; for a substantively identical plan, the governance decision does not depend on which planner produced it. Language-model judges do not reliably screen a biological sequence On the cargo-screening task, the function-aware ESM-2 classifier achieved a true-positive rate of 0.

72 (with a 95% confidence interval of 0. p. 9
Where the paper says it

054 (95% CI 0. 099), an interval excluding zero.

43 to 0.89) at a stringent false-positive rate of only 1% on a de-circularized benchmark of safe proxies. p. 4
Where the paper says it

With it we release a de-circularized benchmark drawn entirely from safe proxies and scored against independent oracles, for which we assembled a measured baseline panel of frontier and open language models and a categorized adversarial red team. We further establish a finite-sample certified ceiling on the false-refuse rate, which is an assurance that no language-model judge provides, and we give an account, carried through pre-registration, of the capabilities that are validated, of those that are mechanismgrounded without being outcome-validated, and of the pre-registered upgrades that did not meet their thresholds.

Furthermore, under prompt injection attacks, the open-weight judges exhibited resilience, flipping their blocking verdict to allow in three out of six trials per channel, while the deterministic screen remained invariant. p. 1
Where the paper says it

Under prompt injection, the open-weight judges flipped their blocking verdict to allow in 3 and 5 of 6 trials per channel, while the deterministic screen remained invariant. None of 288 legitimate plans from three templates was refused, yielding a certified 95% upper bound of 0.

Crucially, none of the 288 legitimate plans from three templates were refused during testing, yielding a certified 95% upper bound on the false-refuse rate of 0. p. 1
Where the paper says it

None of 288 legitimate plans from three templates was refused, yielding a certified 95% upper bound of 0. 0103 on the false-refuse rate, and a session monitor intercepted cross-call decomposition attacks.

0103. The locus axis analysis further indicated that this axis was enriched for drivers of in vivo insertional oncogenesis (AUROC 0.605; odds ratio 3. p. 1
Where the paper says it

On a held-out gene set, the locus axis was enriched for drivers of in vivo insertional oncogenesis (AUROC 0. 605; odds ratio 3.

34) when evaluated against a held-out gene set, providing outcome validation against mouse model results. p. 5
Where the paper says it

Among them are the benchmark and red-team results, which include attack and control counts by category, and the frozen per-model results for the language-model panel, with the two evaluation occasions reported separately. The derived Candidate Cancer Gene Database positive sets used for outcome validation of the locus axis are also included, with their provenance and access date, so are the reproduction and verification logs from a run on a clean image.

While the cargo and locus axes yielded quantified results, the edit type, germline, and scale axes are maintained as mechanism-grounded rule bases. p. 1
Where the paper says it

2 cargo, locus, edit type, germline and scale, with cited evidence, a signed design passport, a tamperevident audit log, and tiered access. On a de-circularised benchmark of safe proxies scored against independent oracles, a function-aware cargo classifier reached a true-positive rate of 0.

The system is released open source with a pre-registered, open-data-reproducible benchmark that provides a certified ceiling on the false-refuse rate. p. 4

Where the paper says it

We further establish a finite-sample certified ceiling on the false-refuse rate, which is an assurance that no language-model judge provides, and we give an account, carried through pre-registration, of the capabilities that are validated, of those that are mechanismgrounded without being outcome-validated, and of the pre-registered upgrades that did not meet their thresholds. BioFirewall is tool-agnostic and governs any planning agent through a thin adapter; it is released as open source under a permissive licence, with a pre-registered benchmark whose open-data results are reproduced from the public repository.

It offers session-level monitoring to intercept cross-call decomposition attacks and functions as tool-agnostic, governing any planning agent via a thin adapter layer. p. 4
Where the paper says it

We further establish a finite-sample certified ceiling on the false-refuse rate, which is an assurance that no language-model judge provides, and we give an account, carried through pre-registration, of the capabilities that are validated, of those that are mechanismgrounded without being outcome-validated, and of the pre-registered upgrades that did not meet their thresholds. BioFirewall is tool-agnostic and governs any planning agent through a thin adapter; it is released as open source under a permissive licence, with a pre-registered benchmark whose open-data results are reproduced from the public repository.

The methodology incorporates multiple experimental runs (Run 1 and Run 2) covering scope experiments A (fabrication), B (determinism), and D (jailbreak-judge) for Run 1, alongside Experiment C for sequence screening in Run 2. p. 5
Where the paper says it

The derived Candidate Cancer Gene Database positive sets used for outcome validation of the locus axis are also included, with their provenance and access date, so are the reproduction and verification logs from a run on a clean image. For each cycle, the acceptance criteria, limitations, and frozen results are pre-registered in SHA-locked files committed alongside the code they govern, and a manifest of digests covers every deposited pre-registration.

Provenance checks are enforced via pre-registration manifests with SHA-256 digests, ensuring that reported criteria align exactly with deposited files. p. 5
Where the paper says it

The derived Candidate Cancer Gene Database positive sets used for outcome validation of the locus axis are also included, with their provenance and access date, so are the reproduction and verification logs from a run on a clean image. For each cycle, the acceptance criteria, limitations, and frozen results are pre-registered in SHA-locked files committed alongside the code they govern, and a manifest of digests covers every deposited pre-registration.

The system distinguishes between refusals to engage, structured verdicts (allow, flag), and null verdicts (screen-fail) in the cargo experiment; refusals to engage are excluded from TPR and FPR calculations. The system's status field indicates that causal/prospective clonal-outcome validation is currently DEFERRED pending further results.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Governance layer implementation for design stage planning: BioFirewall implements the complete set of design-stage guardrails specified by the NTI, namely input and output screening, signed design metadata, and managed access. This system intercepts genome-writing plans to return an allow, flag-for-review, or refuse decision across five hazard axes: cargo, locus, edit type, germline and scale.

  2. Enhanced cargo screening capability: A function-aware classifier using ESM-2 embeddings is used to score proposed protein cargos against safe proxy signatures of toxic function at a 1% false positive rate, achieving a true-positive rate of 0.72 (95% CI 0.43 to 0.89).

  3. Locus hazard identification: The locus axis flags insertion-site hazard from proximity to oncogenes and tumour suppressors, essentiality, and dosage sensitivity through a positional refinement that flags promoter- and enhancer-proximal insertions near oncogene transcription start sites.

  4. De novo rearrangement risk assessment: The edit type axis assesses the risk of de novo oncogenic rearrangement on mechanistic grounds by flagging constitutive kinase activation, juxtaposition of a strong enhancer or promoter with an oncogene, and preserved architecture combining kinase and dimerization domains.

  5. Session-level decomposition monitoring: A session monitor intercepts cross-call decomposition attacks by reconstructing cross-call intent through assembly and junction inference, cumulative scale aggregation, and clustering of coordinated loci to catch these evasions at 100% catch rate for cumulative scale and fragment assembly categories.

  6. Certified false-refuse ceiling: BioFirewall provides a certified ceiling on the false-refuse rate, which is a Clopper-Pearson 95% upper bound of 0.0103 over the evaluation set, certifying that the false-refuse rate is below both 5% and 10%.

  7. Tamper-evident audit trail: Every decision is accompanied by a signed design passport with an HMAC-SHA256 signature tying the verdict to a hash of the inputs, which makes tampering apparent after the fact.

  8. Tiered access control: A managed access plane assigns an access tier based on the severity of the verdict and verified user legitimacy, where a hard reject is never unlocked by credentials, ensuring that no tier unlocks a refuse verdict.

Abstract

AI design tools now plan genome-scale edits, and agentic systems execute those plans with progressively less human oversight. Biosecurity controls are limited to two points: refusal guardrails at the foundation model and sequence-identity screening at the synthesiser. The design stage between them, where the plan is specified, is governed by recommendations, not by any deployed system. We present BioFirewall, a rule-governed middleware that intercepts a genome-writing plan and returns allow, flag-for-review, or refuse across five genome-writing-native hazard axes: cargo, locus, edit type, germline and scale, with cited evidence, a signed design passport, a tamper-evident audit log, and tiered access. On a de-circularised benchmark of safe proxies scored against independent oracles, a function-aware cargo classifier reached a true-positive rate of 0.72 (95% CI 0.43 to 0.89) at a false-positive rate of 1.105%, the nearest attainable to 1%, on a held-out split. Language-model judges did not reliably screen these sequences: their engagement depended heavily on sequence class, so a rate conditioned on engagement does not measure screening performance. Under prompt injection, the blocking verdicts of the open-weight judges tested flipped to allow, while the deterministic screen remained invariant. None of the 288 legitimate plans was refused, giving a 95% upper bound on the false-refuse rate of 0.0103 for the evaluated corpus under an independence assumption, or 0.632 treating the three plan templates as independent units. On a held-out gene set, the locus axis was enriched for drivers of in vivo insertional oncogenesis (AUROC 0.605; odds ratio 3.34), and a session monitor intercepted the cross-call decomposition attacks it was built to detect. These results show design-stage governance is achievable within the evaluated scope. BioFirewall is open source under a permissive licence.

Sources

Related papers