Building Barricades: Red-Teaming Agentic Biosynthesis Planning

summary

The gist

LLMs and agentic systems threaten to greatly lower knowledge barriers to producing small molecule toxins via biosynthesis…

In short

Researchers developed BioBulbul, an agentic system combining LLMs with biosynthesis planning and gene cluster design tools. This system uses chain-of-thought prompting to allow a simple natural language prompt like 'Synthesize [compound name]' to generate DNA sequences for synthesizable toxins. The finding highlights the risk of non-experts designing toxin pathways using these accessible AI tools.

Key concepts

Agentic System
An autonomous system where an LLM acts as a central agent, capable of selecting and executing multiple domain-specific tools sequentially to achieve a complex goal. BioBulbul exemplifies this by dynamically composing and running planning tools based on user input.
Chain-of-Thought Prompting
A prompting technique used to guide an LLM through a multi-step reasoning process. In this context, it allows the agent to break down a complex request (like synthesizing a compound) into sequential steps—planning, tool selection, and execution—to discover non-obvious pathways.
Biosynthesis Planning
The process of designing the biological pathway (the sequence of enzymes and reactions) required to create a specific small molecule. The system uses template-based and template-free tools to explore various potential reaction steps for toxin production.
Knowledge Barriers in Biosynthesis
The technical hurdles that historically prevented non-experts from designing biosynthetic pathways, such as identifying necessary enzymes or designing gene clusters. LLMs are beginning to lower these barriers by allowing non-experts to use natural language instead of needing expert knowledge.

Terminology used across episodes

This episode discusses

The paper

Building Barricades: Red-Teaming Agentic Biosynthesis Planning · Read on arXiv

Nicholas Roehner, Bryan Bartley, Jacob Beal, Caitlin Jagla, Troy Timmerman, Kemper Talley

RTX BBN Technologies

The rapid democratization of biotechnology, enhanced by large language models (LLMs) and agentic software, is eroding longstanding barriers that have limited biosynthesis planning to experts. We present BioBulbul, an agentic system that couples LLMs with open-source biodesign tools for retrobiosynthesis, enzyme selection, gene cluster design, and DNA assembly planning. Using simple natural language prompts, BioBulbul autonomously generates plausible biosynthesis pathways and corresponding gene cluster sequences for several known small molecule toxins, demonstrating that non-experts could feasibly access them using an agentic system. Our red-teaming experiments confirm the technical viability of LLM-driven toxin biosynthesis while highlighting three principal barriers: scarcity of open-source biodesign tools, limited structured enzymatic reaction data, and the complexity of many small molecule toxin pathways. We argue that these barriers are likely to diminish within a decade, expanding the biological threat landscape. To mitigate emerging risks, we recommend expanding DNA synthesis screening to include chokepoint enzymes, extending chemical supplier controls to precursor compounds, establishing continuous AI-assisted threat surveillance pipelines, and developing robust LLM guardrails. Coordinated investment across government, industry, and AI developers is essential to safeguard against the misuse of agentic biosynthesis planning.

Transcript

Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.

Ines: We're looking at the paper: "Building Barricades: Red-Teaming Agentic Biosynthesis Planning".

Marcus: Here's the gist of the paper. LLMs and agentic systems threaten to greatly lower knowledge barriers to producing small molecule toxins via biosynthesis, making it possible for nonexperts to invoke relevant biodesign tools and interpret their results The advent of LLMs is beginning to lower some of the technical barriers that have historically limited biosynthetic design to experts As increasing capabilities in biotechnology and synthetic biology have made bioengineering more accessible, these barriers to access have been falling Non-experts are hampered by knowledge barriers associated with tasks such as identifying the enzymes needed to synthesize a target small molecule (retrobiosynthesis) and designing gene clusters to express these enzymes A parallel phenomenon is emerging in biotechnology, one that will enable non-expert users to perform biosynthesis planning using simple natural language prompts and LLM agents that have been interfaced with biodesign tools If knowledge barriers continue to erode, there is a plausible risk that individuals without formal training could attempt to design and implement biosynthesis pathways for small molecule toxins Case Study of Agentic System Development BBN developed an agentic system, BioBulbul, that ties together LLMs with tools for biosynthesis planning and gene cluster design Like ChemCrow, BioBulbul leverages chain-of-thought prompting to dynamically compose and execute domain-specific software tools BioBulbul incorporates different types of tools for planning biosynthesis pathways The system utilizes template-based and template-free planning tools to discover non-obvious pathways that constitute potential blind spots in DNA synthesis screening With BioBulbul, a simple natural language prompt in the form of “Synthesize

compound name: ” is sufficient for the LLM agent to select and invoke tools for biosynthesis planning followed by genetic design The final result is a single spreadsheet with sequences for synthesizable DNA fragments that are compatible with a standard Gibson assembly protocol which LLMs are presently capable of generating as well Technical Barriers to Accessibility There are at least three barriers standing in the way of LLM-enabled biosynthesis of small molecule toxins The first and arguably least consequential of these barriers is the relative dearth of freely available open-source biodesign tools for biosynthesis planning and gene cluster design The second barrier to LLM-enabled toxin biosynthesis is the availability of suitable databases and training data for biodesign tools The third and currently strongest barrier to LLM-enabled toxin biosynthesis is the relative complexity of many known pathways (i.e., more than six reaction steps) and the enzymes that are often involved Practical Mitigation Strategies Given the certain erosion of the first two barriers with time and the indeterminate robustness of the third barrier, it is prudent to consider practical mitigations for the potential threat of LLM-enabled toxin biosynthesis Practical mitigations include: 1.

Ines: Screening purchases for control of potential hazards, which involves screening DNA synthesis orders for sequences of enzymes that are more likely than not to be involved in toxin biosynthesis A second pillar of industry-wide best practices could leverage existing control strategies used by chemical suppliers and extend them to include screening purchasers of commercially available precursor compounds necessary for the biosynthesis of small molecule toxins 2.

Marcus: Sustainable threat landscape surveillance, which involves developing an agentic AI-assisted human-in-the-loop workflow to detect and assess emerging biodesign tools and datasets before they become widely available This approach could involve automating the identification of emerging risks through continuous observation of scientific literature reviews, social media listening, and website change detection 3.

Ines: LLM guardrails to limit risk and enable detection, such as biasing an LLM to return pathways that are easily detected via screening due to the inclusion of controlled precursors or pathways that synthesize less toxic alternatives We recommend the pursuit of practical mitigations and safeguards spanning existing routes for both biomanufacturing and biodesign For biomanufacturing, these safeguards include expanding DNA synthesis screening pipelines to include sequences for “chokepoint” enzymes primarily involved in the biosynthesis of toxins and extending chemical-supplier controls to precursor compounds for toxin biosynthesis For biodesign, we recommend developing automated, potentially agentic approaches that focus on continuous threat surveillance to triage emerging biodesign tools and datasets before they become widely available We also recommend conducting further research into LLM guardrails to make malicious use of LLMs for biodesign less effective and easier to detect The ultimate success of these proposed mitigations will depend on continuing investment into biosecurity research and coordination between stakeholders in government, the biomanufacturing industry (including organizations such as the SBRC), and groups developing LLMs and biodesign tools By investing in practical mitigations, engaging stakeholders, and developing standards for screening and rapid reporting, the biosecurity community can greatly reduce the likelihood that small molecule toxins are synthesized by non-expert actors using advanced AI agents Acknowledgements This work was supported by the Defense Advanced Research Projects Agency (DARPA) Biological Technologies Office (BTO) under award number HR001124C0470 The views, opinions, and/or findings expressed are those of the author(s) and should not be interpreted as representing the official views or policies of DARPA or the U.S.

Paper summary: Ines: Building Barricades: Red-Teaming Agentic Biosynthesis Planning is essentially about testing how much an LLM agent can lower the knowledge barriers for designing biosynthesis pathways. The central thesis is that we need to assess this risk because LLMs are starting to make chemical synthesis accessible through text queries and standard lab equipment like Opentrons.

Marcus: They are concerned that if this trend continues, non-expert users could design small molecule toxins themselves using these AI agents interfaced with biodesign tools. It highlights a real parallel phenomenon emerging in biotechnology where planning can happen through simple natural language prompts.

Yuki: What matters is that the paper specifically assesses the degree to which these LLM agents could lower those barriers, looking at whether they can successfully design and implement pathways for small molecule toxins without expert knowledge.

Ines: The paper develops BioBulbul as their system to test this, showing how it ties together LLMs with planning and gene cluster design tools. It uses chain-of-thought prompting, which is a form of retrieval augmented generation or RAG, to enhance the LLM's response with specific biochemical and genetic information.

Marcus: They use this agentic workflow to demonstrate that a simple prompt like "Synthesize compound name" is sufficient for the LLM agent to select and invoke the necessary tools for planning biosynthesis followed by genetic design.

Yuki: The system uses both template-based planners, which rely on interpretable reaction rules designed by experts or extracted from databases, and template-free planners, which use black-box models trained on reaction databases. This shows they are testing different levels of AI assistance in the design process.

Ines: And they integrate tools from the Galaxy SynBioCAD pipeline for gene cluster design, covering selecting specific enzymes and designing libraries of gene clusters that vary enzyme expression using things like promoters and ribosome binding sites. That’s a comprehensive planning workflow.

Marcus: The implication is that this agentic capability means non-experts could potentially generate sequences for synthesizable DNA fragments compatible with a standard Gibson assembly protocol, which LLMs are already capable of generating as well.

Yuki: It points to the fact that the complexity of known pathways, sometimes involving more than six reaction steps and many enzymes, is being tackled by this AI approach. But they also acknowledge the barriers: there are still issues with open-source biodesign tools and suitable training data for these planning tools.

Conclusion: Ines: So, looking at "Building Barricades: Red-Teaming Agentic Biosynthesis Planning," the authors are showing that the capability to design biosynthesis pathways could become more accessible through agentic systems if LLM-driven tools mature.

Marcus: They are essentially building a red team for this capability, testing if simple natural language prompts combined with biodesign tools can lead to non-experts designing small molecule toxins. The paper suggests this is a plausible risk as knowledge barriers continue to erode.

Yuki: What this means for us in the wider field is that we have to seriously consider practical mitigations because the ability for non-expert actors to design these pathways could become a real threat. They are calling out the need for safeguards against this kind of misuse.

Ines: The authors suggest focusing on three main areas: first, screening DNA synthesis orders for sequences of enzymes more likely to be involved in toxin biosynthesis.

Marcus: Second, they propose sustainable threat landscape surveillance using an AI-assisted human-in-the-loop workflow to detect emerging biodesign tools and datasets before they become widely available.

Yuki: And third, they recommend looking into LLM guardrails that could bias the LLM to return pathways that are easily detected via screening, maybe by including controlled precursors or pathways that synthesize less toxic alternatives.

Ines: The paper strongly calls for coordination between government, biomanufacturing groups like the SBRC, and AI developers to develop standards for screening and rapid reporting.

Marcus: It really boils down to investing in practical safeguards across both biomanufacturing and biodesign so that non-expert actors don't end up synthesizing small molecule toxins using these advanced AI agents.

More episodes

← Home