PEN-STACK: A provenance-enforcing tool layer for language-model agents in genome writing

summary

The gist

PEN-STACK is an open tool layer designed to supply guaranteed provenance to language-model agents engaged in genome writing, functioning as a substrate rather than just an application.

In short

PEN-STACK is a tool layer ensuring guaranteed provenance for AI agents writing genomes. It enforces a 'no-fabrication' rule: every number must come from a validated tool call. This prevents models from inventing data, which is crucial for reliable biological design. The system uses scope-aware tools and safety checks to ensure outputs are grounded in verifiable evidence.

Key concepts

No-Fabrication Invariant
This core rule mandates that every quantitative value reported by an agent must originate directly from a validated tool call. If a model tries to state a number without this direct source, the system flags it as ungrounded and structurally unrepresentable, making fabrication impossible.
Scope-Aware Tools
PEN-STACK exposes twenty-two tools covering ten genome-writing design stages. These tools are context-aware, meaning they only operate on relevant parameters for a specific design step. This structure allows agents to interact with the system in a controlled, verifiable manner.
Design-Stage Biosecurity Screen
Before any protocol is sent out, a screening mechanism checks the design against safety standards. It maps decisions onto a predefined vocabulary (IBBIS Common Mechanism ScreenStatus) to immediately flag or refuse designs that might pose risks, such as enhancing pandemic transmissibility.
Grounding vs. Prompting
Instead of relying on models to guess necessary data through prompting, PEN-STACK requires agents to ground their answers in tool outputs. This means agents produce verifiable quantities by forcing them to use validated tools rather than just generating text based on a prompt.

Terminology used across episodes

This episode discusses

The paper

PEN-STACK: A provenance-enforcing tool layer for language-model agents in genome writing · Read on arXiv

Anees Ahmed Mahaboob Ali, Radhakrishnan Delhibabu, Everette Jacob Remington Nelson

School of Bio Sciences and Technology, Vellore Institute of Technology · School of Computer Science and Engineering, Vellore Institute of Technology

Writing a gene into a human genome is a coupled design problem across locus, writer enzyme, cargo and delivery vehicle. Language-model agents are widely applied to such problems, but genome writing has no grounded tool layer. We present PEN-STACK, an open tool layer with enforced provenance that exposes ten design stages as twenty-two scope-aware tools over SDK, Model Context Protocol and REST interfaces. It integrates chromatin, enzyme, off-target, delivery and immunogenicity evidence under one typed contract: every reported quantity originates from a validated tool, and anything out of scope returns an explicit known-unknown. Given no tools and a naive prompt, three model families fabricated 97.1% to 100.0% of 240 required quantities. Coaching against fabrication reduced this to 0 of 240 on each of two families and 5 of 240 on a 1.5B-parameter local model, but each measured zero carries a 95% upper bound of 1.58%, so prompting yields no model-independent guarantee. With tool access on the same forty goals, all three families fabricated nothing, but 81% and 84% of required quantities resolved through a tool call for the two larger models against 27% for the local model: grounding confers fabrication safety, not capability. A pre-emission biosecurity screen matched its own probe specification on all eight designs, a self-consistency check with unmeasured error rates. A human position-effect model validated on held-out chromosomes (Spearman rho = 0.571, n = 1,506) but not on the features served by default (rho 0.16), where the service withholds its validation flag. Ten claims were pre-specified in hash-locked specifications, though priority over scoring is not independently established; every outcome, including eight non-passes, is a machine-readable flag. Insertional-oncogenesis coverage is limited to documented loci. Code and three data resources are released openly.

Transcript

Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.

Ines: We're looking at the paper: "PEN-STACK: A provenance-enforcing tool layer for language-model agents in genome writing".

Marcus: Here's the gist of the paper. PEN-STACK is an open tool layer designed to supply guaranteed provenance to language-model agents engaged in genome writing, functioning as a substrate rather than just an application.

Ines: It enforces a no-fabrication invariant: every quantitative value reported must originate from a validated tool call, making ungrounded numbers structurally unrepresentable.

Marcus: This is achieved through mechanisms such as preventing candidate claims via errors on.as claim, freezing the nested provenance record to ensure immutability, and explicitly returning out-of-scope quantities as known unknowns.

Ines: The system exposes ten genome-writing design stages through twenty-two scope-aware tools, accessible via a software development kit, a Model Context Protocol server, and a Representational State Transfer interface.

Title and authors: Ines: So, we’re looking at this paper today called "PEN-STACK: A provenance-enforcing tool layer for language-model agents in genome writing." It sounds like they're trying to fix a big problem with how these AI models are used in biology.

Marcus: Yeah, it’s about making sure when an AI agent is designing a new genome, every single number it spits out actually comes from a tool you've already validated. It’s basically building this substrate so the model can't just make things up.

Yuki: From my side, I’m thinking about what that means for us. If these agents are making design plans for new DNA sequences, we need to know those quantities—like where to write something or what enzyme to use—are actually feasible and grounded in reality, not just plausible sounding text.

Ines: Exactly. The core idea is this layer enforces a "no-fabrication invariant." This means if the agent reports a quantity, it has to show the exact tool call that produced it. If you can't show that tool call, the claim is structurally unrepresentable, meaning it can't be true >

Marcus: It’s like making sure every claim in a report has its source line right next to it. They achieve this by stopping candidate claims with errors on something called ".as claim", and they freeze the nested record so you can’t change the history later, which keeps everything immutable >

Yuki: I wonder how that changes things for designing new life forms. Right now, we rely a lot on the model to bridge those gaps between design steps. If it can't invent a required component, it has to admit it doesn't know >

The paper's summary: Ines: The paper summarizes that they’ve built ten genome-writing design stages and twenty-two scope aware tools. These tools are all accessible through a software development kit, a Model Context Protocol server, and a representational state transfer interface >

Marcus: And the main result they show is how much fabrication happens without this layer. They found that in tests involving four planning goals needing six tool only quantities each—that’s twenty-four required quantities total—models fabricated between ninety point eight percent and ninety eight point eight percent of those numbers when just using a naive prompt >

Yuki: That high fabrication rate is concerning because it shows how much the models are guessing when they don't have a reliable source for the necessary design parameters >

Ines: Right, but then they show that if you drive the tools—meaning you force the agent to use those validated tools—the same models fabricated nothing on a four goal audit >

Marcus: That’s where the real difference is. Driving the validated tools results in every model family passing an audit when quantities have to go through those tools, and they even used a grounded zero as a comparison point >

Yuki: So, it suggests that if we give the agent access only to verified design steps, it stops inventing data points that are supposed to be fixed by the tools >

Ines: It really points toward grounding being better than just prompting or scaling up the model itself when you need verifiable biological quantities >

The paper's improvements: Marcus: Now let’s talk about what they suggest as improvements. One big one is grounding instead of prompting, which removes fabrication. The idea is that by requiring every quantity to "originate from a validated tool," agents can produce verifiable quantities, and the grounded arm, a four goal audit, warrants replication at the two hundred forty field scale >

Ines: That’s solid. They also implemented something called a type-enforced invariant which prevents claims. So if you try to generate a result that isn't from a tool call, it throws an error on ".as claim", and the generated sequence or structure remains just a proposal until writer verification scores it against measured data >

Yuki: That sounds like they’re trying to stop the models from asserting ungrounded facts as truth about biological systems >

Marcus: And they handle the things that are out of scope, too. They explicitly return quantities falling outside a tool’s scope card as known unknowns, and these are neither guessed nor omitted >

Ines: That’s important because it means they aren't making up data for parameters that the current tools can't handle; they just state what they don't know about >

Yuki: I like that explicit handling. It prevents them from inventing necessary data points, which is a big step toward more honest agent behavior >

Conclusion: Ines: So to wrap up, the PEN-STACK paper argues that grounding, not just prompting or making models bigger, removes fabrication by requiring a substrate for these agents >

Marcus: It provides that substrate as open and importable. They showed how driving validated tools stops fabrication on a four goal audit across different model families >

Yuki: For the wider field, this means if we build agent systems in biology, we can use this type-level solution as code to ensure the agents aren't just hallucinating necessary components >

Ines: And they showed that even with some noise, like a correlation of zero point five seven one for the human K562 head at exact-site resolution, the system still functions as intended >

Marcus: But they did have some limitations. They noted that while exact-site resolution is good, when you use coarser resolution features, like those served by default, the validation flag isn't returned reliably and you get a machine readable downgrade flag instead >

Yuki: That’s a fair limitation. It means if we rely on the defaults for speed or simpler analysis, we might not have that guaranteed provenance >

Ines: Yeah, so while it solves the fabrication problem by enforcing the tool layer structure, there's still a trade-off between resolution and certainty >

Marcus: It’s a lot of work to build this substrate, but if you can import it into other agentic biology systems, it could be very powerful for ensuring reliability >

More episodes

← Home