BioFirewall: A genome-writing-native governance layer for design-stage biosecurity screening of agentic AI
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.
Ines: I'm Ines.
Marcus: And I'm Marcus, here with Yuki.
Ines: We're looking at the paper: "BioFirewall: A genome-writing-native governance layer for design-stage biosecurity screening of agentic AI".
Marcus: Here's the gist of the paper. BioFirewall is a rule-governed middleware designed as a genome-writing-native governance layer for design-stage biosecurity screening of agentic AI systems.
Ines: It intercepts genome-writing plans and classifies them across five hazard axes: cargo, locus, edit type, germline, and scale, returning one of three verdicts: allow, flag-for-review, or refuse.
Marcus: The system provides cited evidence and a signed design passport secured with an HMAC-SHA256 signature to ensure tamper evidence.
Ines: It also implements tiered access controls and addresses the complete set of design-stage guardrails specified by the Nuclear Threat Initiative (NTI), including input/output screening, signed metadata management, and managed access.
Paper summary: Ines: So we've heard that BioFirewall is this rule-governed layer designed for design-stage screening of agentic AI genome writing plans. What they are doing is reasoning about hazards that standard synthesizers just don't see.
Marcus: They claim their framework has a five-axis stratified screen, a de-circularized benchmark, and measurements of language models against baselines to show how well it works.
Yuki: And they also categorized red team analysis to test the system's resilience against different kinds of attacks.
Ines: The results show some specific numbers on the cargo axis classifier, where it achieved a true positive rate of zero point seven two with a ninety-five percent confidence interval between zero point four three and zero point eight nine when tested at a one percent false-positive rate on their de-circularized benchmark of safe proxies <ref:2608.20413#pg2,a true positive rate of 0.72>.
Marcus: That sounds like a solid result for the cargo axis, but they also showed that under prompt injection attacks, the open weight judges flipped their blocking verdict to allow in three out of six trials per channel, while the deterministic screen stayed the same.
Yuki: And interestingly, none of the two hundred eighty-eight legitimate plans from three templates were refused during testing, which means they have a certified ninety-five percent upper bound on the false refuse rate at zero point zero one zero three <ref:2608.20413#pg2,288 legitimate plans from three templates>.
Ines: They also looked at the locus axis and found that it was enriched for drivers of in vivo insertional oncogenesis, with an auroc of zero point six zero five and an odds ratio of three point three four when compared against a held out gene set, which they linked to some mouse model results <ref:2608.20413#pg2,was enriched for drivers of in vivo insertional oncogenesis>.
Marcus: While cargo and locus gave them quantified results, the edit type, germline, and scale axes are just described as mechanism grounded rule bases rather than specific statistical outcomes.
Yuki: So what this means for us is that they are trying to build something that actually works at this design stage, which is where things are currently pretty open.
Conclusion: Ines: So we've covered the details of BioFirewall, its five hazard axes, and those specific numbers on the cargo and locus axes. The authors are calling this design-stage governance achievable in practice.
Marcus: They released it open source along with a pre-registered, open data reproducible benchmark that sets a certified ceiling on the false refuse rate for anyone trying to use it.
Yuki: It seems like the implication is that we can start moving beyond just model layer refusals or sequence screening at the execution stage and build this kind of explicit governance before plans even get sent to the lab robots.
Ines: The title itself, "BioFirewall: A genome-writing-native governance layer for design-stage biosecurity screening of agentic AI", suggests they're building something deeply integrated into how these systems are designed from the ground up.
Marcus: It’s about moving the guardrails from being just a filter on the output to being a part of the initial plan formulation process itself, which is where most of this new risk is happening.
Yuki: It forces us to think about what kind of planning agents we are even allowing to design these things in the first place.
Anees Ahmed Mahaboob Ali, Radhakrishnan Delhibabu, Everette Jacob Remington Nelson
School of Bio Sciences and Technology, Vellore Institute of Technology · School of Computer Science and Engineering, Vellore Institute of Technology
q-bio.GN
Submitted: 2026-08-15
Updated: 2026-11-09
Code: https://github.com/ahmedanees-m/bio-firewall
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 92/100
The gist: BioFirewall is a rule-governed middleware designed as a genome-writing-native governance layer for design-stage biosecurity screening of agentic AI systems.
Key concepts
- Hazard Axes
- These are the five specific dimensions BioFirewall uses to classify potential risks in a genome-writing plan: cargo (what is being modified), locus (where the modification occurs), edit type, germline impact, and scale of the operation. They allow for a comprehensive hazard assessment.
- Genome-Writing-Native Governance Layer
- This describes BioFirewall's core function: it sits directly between planning and synthesis to govern how AI systems write genetic code. It intercepts these plans to enforce security rules, making governance an intrinsic part of the design process rather than an afterthought.
- Stratified Screen
- The system uses a multi-layered screening approach. It combines a five-axis stratified screen with benchmark testing and red teaming analysis to ensure robust hazard detection. This layered method helps identify risks that standard sequence screening might miss.
- Design-Stage Screening
- BioFirewall operates at the critical design phase, before the actual synthesis of sequences begins. Its goal is to catch dangerous plans early in the development process, acting as a crucial guardrail between initial model refusal checks and final sequence screening.
Terminology
Summary
BioFirewall is a rule-governed middleware designed as a genome-writing-native governance layer for design-stage biosecurity screening of agentic AI systems. p. 0 1
BioFirewall: A genome-writing-native governance layer for design-stage
biosecurity screening of agentic AI
Anees Ahmed Mahaboob Ali1
, Radhakrishnan Delhibabu2
, Everette Jacob Remington Nelson1
*
1School of Bio Sciences and Technology, Vellore Institute of Technology, Vellore, India
2School of Computer Science and Engineering, Vellore Institute of Technology, Vellore, India
*Corresponding author. E-mail: everette. The design stage between them, where the plan is specified, remains governed
by recommendations rather than any deployed system. We present BioFirewall, a rule-governed middleware that intercepts a genome-writing
plan and returns allow, flag-for-review, or refuse across five hazard axes native to genome writing: 2
cargo, locus, edit type, germline and scale, with cited evidence, a signed design passport, a tamperevident audit log, and tiered access. On a de-circularised benchmark of safe proxies scored against
independent oracles, a function-aware cargo classifier reached a true-positive rate of 0. The Nuclear Threat
Initiative (NTI), analyzing guardrails for AI biodesign tools, recommends that inputs and outputs
be screened to flag or reject high-risk designs, and that design metadata be cryptographically
signed to support traceability and inference of intent [15]. A subsequent framework adds managed
access, tiered by tool risk and verified user legitimacy, and explicitly frames it as the foundation
on which the remaining guardrails are built [16].Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
The system operates at a critical design stage, situated between model refusal guardrails and synthesis-layer sequence screening. p. 0 Biosecurity controls are
limited to two points: refusal guardrails at the foundation model and sequence-identity screening
at the synthesiser. The design stage between them, where the plan is specified, remains governed
by recommendations rather than any deployed system. 4
function [12], and the benchmarks and conventions that this requires are being formalized through
standard-setting efforts at the DNA Synthesis Screening Consortium [13] and through inter-tool
benchmarking against a National Institute of Standards and Technology reference dataset [14]. The design stage, wherein a plan is actually specified, lies between the model and the synthesizer,
which has been governed by recommendation rather than implementation. With it we release a de-circularized
benchmark drawn entirely from safe proxies and scored against independent oracles, for which we
assembled a measured baseline panel of frontier and open language models and a categorized
adversarial red team. We further establish a finite-sample certified ceiling on the false-refuse rate,
which is an assurance that no language-model judge provides, and we give an account, carried
through pre-registration, of the capabilities that are validated, of those that are mechanismgrounded without being outcome-validated, and of the pre-registered upgrades that did not meet
their thresholds.Where the paper says it
Where the paper says it
Where the paper says it
Key results demonstrate its efficacy across various dimensions. p. 0 The design stage between them, where the plan is specified, remains governed
by recommendations rather than any deployed system. We present BioFirewall, a rule-governed middleware that intercepts a genome-writing
plan and returns allow, flag-for-review, or refuse across five hazard axes native to genome writing: Two reference adapters are also included for
planners with differing plan schemas; for a substantively identical plan, the governance decision
does not depend on which planner produced it. Language-model judges do not reliably screen a biological sequence
On the cargo-screening task, the function-aware ESM-2 classifier achieved a true-positive rate of
0. 054 (95% CI 0. 099), an interval excluding zero. With it we release a de-circularized
benchmark drawn entirely from safe proxies and scored against independent oracles, for which we
assembled a measured baseline panel of frontier and open language models and a categorized
adversarial red team. We further establish a finite-sample certified ceiling on the false-refuse rate,
which is an assurance that no language-model judge provides, and we give an account, carried
through pre-registration, of the capabilities that are validated, of those that are mechanismgrounded without being outcome-validated, and of the pre-registered upgrades that did not meet
their thresholds. Under prompt injection, the open-weight judges flipped their
blocking verdict to allow in 3 and 5 of 6 trials per channel, while the deterministic screen remained
invariant. None of 288 legitimate plans from three templates was refused, yielding a certified 95%
upper bound of 0. None of 288 legitimate plans from three templates was refused, yielding a certified 95%
upper bound of 0. 0103 on the false-refuse rate, and a session monitor intercepted cross-call
decomposition attacks. On a held-out gene set, the locus axis was enriched for drivers of in vivo
insertional oncogenesis (AUROC 0. 605; odds ratio 3. Among them are the
benchmark and red-team results, which include attack and control counts by category, and the
frozen per-model results for the language-model panel, with the two evaluation occasions reported
separately. The derived Candidate Cancer Gene Database positive sets used for outcome validation
of the locus axis are also included, with their provenance and access date, so are the reproduction
and verification logs from a run on a clean image. 2
cargo, locus, edit type, germline and scale, with cited evidence, a signed design passport, a tamperevident audit log, and tiered access. On a de-circularised benchmark of safe proxies scored against
independent oracles, a function-aware cargo classifier reached a true-positive rate of 0.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
The system is released open source with a pre-registered, open-data-reproducible benchmark that provides a certified ceiling on the false-refuse rate. p. 4 We further establish a finite-sample certified ceiling on the false-refuse rate,
which is an assurance that no language-model judge provides, and we give an account, carried
through pre-registration, of the capabilities that are validated, of those that are mechanismgrounded without being outcome-validated, and of the pre-registered upgrades that did not meet
their thresholds. BioFirewall is tool-agnostic and governs any planning agent through a thin
adapter; it is released as open source under a permissive licence, with a pre-registered benchmark
whose open-data results are reproduced from the public repository. We further establish a finite-sample certified ceiling on the false-refuse rate,
which is an assurance that no language-model judge provides, and we give an account, carried
through pre-registration, of the capabilities that are validated, of those that are mechanismgrounded without being outcome-validated, and of the pre-registered upgrades that did not meet
their thresholds. BioFirewall is tool-agnostic and governs any planning agent through a thin
adapter; it is released as open source under a permissive licence, with a pre-registered benchmark
whose open-data results are reproduced from the public repository. The derived Candidate Cancer Gene Database positive sets used for outcome validation
of the locus axis are also included, with their provenance and access date, so are the reproduction
and verification logs from a run on a clean image. For each cycle, the acceptance criteria,
limitations, and frozen results are pre-registered in SHA-locked files committed alongside the code
they govern, and a manifest of digests covers every deposited pre-registration. The derived Candidate Cancer Gene Database positive sets used for outcome validation
of the locus axis are also included, with their provenance and access date, so are the reproduction
and verification logs from a run on a clean image. For each cycle, the acceptance criteria,
limitations, and frozen results are pre-registered in SHA-locked files committed alongside the code
they govern, and a manifest of digests covers every deposited pre-registration.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Governance layer implementation for design stage planning: BioFirewall implements
the complete set of design-stage guardrails specified by the NTI, namely input and output screening, signed design metadata, and managed access.
This system intercepts genome-writing plans to return an allow, flag-for-review, or refuse decision across five hazard axes: cargo, locus, edit type, germline and scale. -
Enhanced cargo screening capability: A function-aware classifier using ESM-2 embeddings is used to score proposed protein cargos against safe proxy signatures of toxic function at a 1% false positive rate, achieving a true-positive rate of 0.72 (95% CI 0.43 to 0.89).
-
Locus hazard identification: The locus axis flags insertion-site hazard from proximity to oncogenes and tumour suppressors, essentiality, and dosage sensitivity through a positional refinement that flags promoter- and enhancer-proximal insertions near oncogene transcription start sites.
-
De novo rearrangement risk assessment: The edit type axis assesses the risk of de novo oncogenic rearrangement on mechanistic grounds by flagging constitutive kinase activation, juxtaposition of a strong enhancer or promoter with an oncogene, and preserved architecture combining kinase and dimerization domains.
-
Session-level decomposition monitoring: A session monitor intercepts cross-call decomposition attacks by reconstructing cross-call intent through assembly and junction inference, cumulative scale aggregation, and clustering of coordinated loci to catch these evasions at 100% catch rate for cumulative scale and fragment assembly categories.
-
Certified false-refuse ceiling: BioFirewall provides a
certified ceiling on the false-refuse rate,
which is a Clopper-Pearson 95% upper bound of 0.0103 over the evaluation set, certifying thatthe false-refuse rate is below both 5% and 10%.
-
Tamper-evident audit trail: Every decision is accompanied by a signed design passport with an HMAC-SHA256 signature tying the verdict to a hash of the inputs, which makes tampering apparent after the fact.
-
Tiered access control: A managed access plane assigns an access tier based on the severity of the verdict and verified user legitimacy, where
a hard reject is never unlocked by credentials,
ensuring thatno tier unlocks a refuse verdict.
Abstract
AI design tools now plan genome-scale edits, and agentic systems execute those plans with progressively less human oversight. Biosecurity controls are limited to two points: refusal guardrails at the foundation model and sequence-identity screening at the synthesiser. The design stage between them, where the plan is specified, is governed by recommendations, not by any deployed system. We present BioFirewall, a rule-governed middleware that intercepts a genome-writing plan and returns allow, flag-for-review, or refuse across five genome-writing-native hazard axes: cargo, locus, edit type, germline and scale, with cited evidence, a signed design passport, a tamper-evident audit log, and tiered access. On a de-circularised benchmark of safe proxies scored against independent oracles, a function-aware cargo classifier reached a true-positive rate of 0.72 (95% CI 0.43 to 0.89) at a false-positive rate of 1.105%, the nearest attainable to 1%, on a held-out split. Language-model judges did not reliably screen these sequences: their engagement depended heavily on sequence class, so a rate conditioned on engagement does not measure screening performance. Under prompt injection, the blocking verdicts of the open-weight judges tested flipped to allow, while the deterministic screen remained invariant. None of the 288 legitimate plans was refused, giving a 95% upper bound on the false-refuse rate of 0.0103 for the evaluated corpus under an independence assumption, or 0.632 treating the three plan templates as independent units. On a held-out gene set, the locus axis was enriched for drivers of in vivo insertional oncogenesis (AUROC 0.605; odds ratio 3.34), and a session monitor intercepted the cross-call decomposition attacks it was built to detect. These results show design-stage governance is achievable within the evaluated scope. BioFirewall is open source under a permissive licence.
Sources
- STELLA: Self-Evolving LLM Agent for Biomedical Research
- BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments
- International AI Safety Report 2026
- ABC-Bench: An Agentic Bio-Capabilities Benchmark for Biosecurity
- Agent-SafetyBench: Evaluating the Safety of LLM Agents
- ATBench: A Diverse and Realistic Agent Trajectory Benchmark for Safety Evaluation and Diagnosis
- Revamping Conformal Selection With Optimal Power: A Neyman--Pearson Perspective
- A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification
Related papers
- SeekRBP: Leveraging Sequence-Structure Integration with Reinforcement Learning for Receptor-Binding Protein Identification
- Genome-Factory: A Library for Tuning, Deploying, and Interpreting Genomic Foundation Models
- AVA: Agentic Variant Adjudication for Automated, Auditable Read-Level Review in Cancer Diagnostic Sequencing
- C3P: Contrastive promoter-protein pretraining yields representations capturing bacterial gene regulation
- Leveraging genomic deep learning models for the prediction of non-coding variant effects
- Adaptive Allele-Haplotype Incidence Indexing for Pangenome Queries: Representation, Reconstruction, and Benchmarking