Building Barricades: Red-Teaming Agentic Biosynthesis Planning
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.
Ines: We're looking at the paper: "Building Barricades: Red-Teaming Agentic Biosynthesis Planning".
Marcus: Here's the gist of the paper. LLMs and agentic systems threaten to greatly lower knowledge barriers to producing small molecule toxins via biosynthesis, making it possible for nonexperts to invoke relevant biodesign tools and interpret their results The advent of LLMs is beginning to lower some of the technical barriers that have historically limited biosynthetic design to experts As increasing capabilities in biotechnology and synthetic biology have made bioengineering more accessible, these barriers to access have been falling Non-experts are hampered by knowledge barriers associated with tasks such as identifying the enzymes needed to synthesize a target small molecule (retrobiosynthesis) and designing gene clusters to express these enzymes A parallel phenomenon is emerging in biotechnology, one that will enable non-expert users to perform biosynthesis planning using simple natural language prompts and LLM agents that have been interfaced with biodesign tools If knowledge barriers continue to erode, there is a plausible risk that individuals without formal training could attempt to design and implement biosynthesis pathways for small molecule toxins Case Study of Agentic System Development BBN developed an agentic system, BioBulbul, that ties together LLMs with tools for biosynthesis planning and gene cluster design Like ChemCrow, BioBulbul leverages chain-of-thought prompting to dynamically compose and execute domain-specific software tools BioBulbul incorporates different types of tools for planning biosynthesis pathways The system utilizes template-based and template-free planning tools to discover non-obvious pathways that constitute potential blind spots in DNA synthesis screening With BioBulbul, a simple natural language prompt in the form of “Synthesize
compound name: ” is sufficient for the LLM agent to select and invoke tools for biosynthesis planning followed by genetic design The final result is a single spreadsheet with sequences for synthesizable DNA fragments that are compatible with a standard Gibson assembly protocol which LLMs are presently capable of generating as well Technical Barriers to Accessibility There are at least three barriers standing in the way of LLM-enabled biosynthesis of small molecule toxins The first and arguably least consequential of these barriers is the relative dearth of freely available open-source biodesign tools for biosynthesis planning and gene cluster design The second barrier to LLM-enabled toxin biosynthesis is the availability of suitable databases and training data for biodesign tools The third and currently strongest barrier to LLM-enabled toxin biosynthesis is the relative complexity of many known pathways (i.e., more than six reaction steps) and the enzymes that are often involved Practical Mitigation Strategies Given the certain erosion of the first two barriers with time and the indeterminate robustness of the third barrier, it is prudent to consider practical mitigations for the potential threat of LLM-enabled toxin biosynthesis Practical mitigations include: 1.
Ines: Screening purchases for control of potential hazards, which involves screening DNA synthesis orders for sequences of enzymes that are more likely than not to be involved in toxin biosynthesis A second pillar of industry-wide best practices could leverage existing control strategies used by chemical suppliers and extend them to include screening purchasers of commercially available precursor compounds necessary for the biosynthesis of small molecule toxins 2.
Marcus: Sustainable threat landscape surveillance, which involves developing an agentic AI-assisted human-in-the-loop workflow to detect and assess emerging biodesign tools and datasets before they become widely available This approach could involve automating the identification of emerging risks through continuous observation of scientific literature reviews, social media listening, and website change detection 3.
Ines: LLM guardrails to limit risk and enable detection, such as biasing an LLM to return pathways that are easily detected via screening due to the inclusion of controlled precursors or pathways that synthesize less toxic alternatives We recommend the pursuit of practical mitigations and safeguards spanning existing routes for both biomanufacturing and biodesign For biomanufacturing, these safeguards include expanding DNA synthesis screening pipelines to include sequences for “chokepoint” enzymes primarily involved in the biosynthesis of toxins and extending chemical-supplier controls to precursor compounds for toxin biosynthesis For biodesign, we recommend developing automated, potentially agentic approaches that focus on continuous threat surveillance to triage emerging biodesign tools and datasets before they become widely available We also recommend conducting further research into LLM guardrails to make malicious use of LLMs for biodesign less effective and easier to detect The ultimate success of these proposed mitigations will depend on continuing investment into biosecurity research and coordination between stakeholders in government, the biomanufacturing industry (including organizations such as the SBRC), and groups developing LLMs and biodesign tools By investing in practical mitigations, engaging stakeholders, and developing standards for screening and rapid reporting, the biosecurity community can greatly reduce the likelihood that small molecule toxins are synthesized by non-expert actors using advanced AI agents Acknowledgements This work was supported by the Defense Advanced Research Projects Agency (DARPA) Biological Technologies Office (BTO) under award number HR001124C0470 The views, opinions, and/or findings expressed are those of the author(s) and should not be interpreted as representing the official views or policies of DARPA or the U.S.
Paper summary: Ines: Building Barricades: Red-Teaming Agentic Biosynthesis Planning is essentially about testing how much an LLM agent can lower the knowledge barriers for designing biosynthesis pathways. The central thesis is that we need to assess this risk because LLMs are starting to make chemical synthesis accessible through text queries and standard lab equipment like Opentrons.
Marcus: They are concerned that if this trend continues, non-expert users could design small molecule toxins themselves using these AI agents interfaced with biodesign tools. It highlights a real parallel phenomenon emerging in biotechnology where planning can happen through simple natural language prompts.
Yuki: What matters is that the paper specifically assesses the degree to which these LLM agents could lower those barriers, looking at whether they can successfully design and implement pathways for small molecule toxins without expert knowledge.
Ines: The paper develops BioBulbul as their system to test this, showing how it ties together LLMs with planning and gene cluster design tools. It uses chain-of-thought prompting, which is a form of retrieval augmented generation or RAG, to enhance the LLM's response with specific biochemical and genetic information.
Marcus: They use this agentic workflow to demonstrate that a simple prompt like "Synthesize compound name" is sufficient for the LLM agent to select and invoke the necessary tools for planning biosynthesis followed by genetic design.
Yuki: The system uses both template-based planners, which rely on interpretable reaction rules designed by experts or extracted from databases, and template-free planners, which use black-box models trained on reaction databases. This shows they are testing different levels of AI assistance in the design process.
Ines: And they integrate tools from the Galaxy SynBioCAD pipeline for gene cluster design, covering selecting specific enzymes and designing libraries of gene clusters that vary enzyme expression using things like promoters and ribosome binding sites. That’s a comprehensive planning workflow.
Marcus: The implication is that this agentic capability means non-experts could potentially generate sequences for synthesizable DNA fragments compatible with a standard Gibson assembly protocol, which LLMs are already capable of generating as well.
Yuki: It points to the fact that the complexity of known pathways, sometimes involving more than six reaction steps and many enzymes, is being tackled by this AI approach. But they also acknowledge the barriers: there are still issues with open-source biodesign tools and suitable training data for these planning tools.
Conclusion: Ines: So, looking at "Building Barricades: Red-Teaming Agentic Biosynthesis Planning," the authors are showing that the capability to design biosynthesis pathways could become more accessible through agentic systems if LLM-driven tools mature.
Marcus: They are essentially building a red team for this capability, testing if simple natural language prompts combined with biodesign tools can lead to non-experts designing small molecule toxins. The paper suggests this is a plausible risk as knowledge barriers continue to erode.
Yuki: What this means for us in the wider field is that we have to seriously consider practical mitigations because the ability for non-expert actors to design these pathways could become a real threat. They are calling out the need for safeguards against this kind of misuse.
Ines: The authors suggest focusing on three main areas: first, screening DNA synthesis orders for sequences of enzymes more likely to be involved in toxin biosynthesis.
Marcus: Second, they propose sustainable threat landscape surveillance using an AI-assisted human-in-the-loop workflow to detect emerging biodesign tools and datasets before they become widely available.
Yuki: And third, they recommend looking into LLM guardrails that could bias the LLM to return pathways that are easily detected via screening, maybe by including controlled precursors or pathways that synthesize less toxic alternatives.
Ines: The paper strongly calls for coordination between government, biomanufacturing groups like the SBRC, and AI developers to develop standards for screening and rapid reporting.
Marcus: It really boils down to investing in practical safeguards across both biomanufacturing and biodesign so that non-expert actors don't end up synthesizing small molecule toxins using these advanced AI agents.
Nicholas Roehner, Bryan Bartley, Jacob Beal, Caitlin Jagla, Troy Timmerman, Kemper Talley
RTX BBN Technologies
q-bio.OT
Submitted: 2026-11-05
Updated: 2026-11-05
Comments: 10 pages, 2 figures
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 83/100
The gist: LLMs and agentic systems threaten to greatly lower knowledge barriers to producing small molecule toxins via biosynthesis…
Key concepts
- Agentic System
- An autonomous system where an LLM acts as a central agent, capable of selecting and executing multiple domain-specific tools sequentially to achieve a complex goal. BioBulbul exemplifies this by dynamically composing and running planning tools based on user input.
- Chain-of-Thought Prompting
- A prompting technique used to guide an LLM through a multi-step reasoning process. In this context, it allows the agent to break down a complex request (like synthesizing a compound) into sequential steps—planning, tool selection, and execution—to discover non-obvious pathways.
- Biosynthesis Planning
- The process of designing the biological pathway (the sequence of enzymes and reactions) required to create a specific small molecule. The system uses template-based and template-free tools to explore various potential reaction steps for toxin production.
- Knowledge Barriers in Biosynthesis
- The technical hurdles that historically prevented non-experts from designing biosynthetic pathways, such as identifying necessary enzymes or designing gene clusters. LLMs are beginning to lower these barriers by allowing non-experts to use natural language instead of needing expert knowledge.
Terminology
Summary
LLMs and agentic systems threaten to greatly lower knowledge barriers to producing small molecule toxins via biosynthesis, making it possible for nonexperts to invoke relevant biodesign tools and interpret their results
Introduction and Problem Statement
The advent of LLMs is beginning to lower some of the technical barriers that have historically limited biosynthetic design to experts As increasing capabilities in biotechnology and synthetic biology have made bioengineering more accessible, these barriers to access have been falling Non-experts are hampered by knowledge barriers associated with tasks such as identifying the enzymes needed to synthesize a target small molecule (retrobiosynthesis) and designing gene clusters to express these enzymes A parallel phenomenon is emerging in biotechnology, one that will enable non-expert users to perform biosynthesis planning using simple natural language prompts and LLM agents that have been interfaced with biodesign tools If knowledge barriers continue to erode, there is a plausible risk that individuals without formal training could attempt to design and implement biosynthesis pathways for small molecule toxins
Case Study of Agentic System Development
BBN developed an agentic system, BioBulbul, that ties together LLMs with tools for biosynthesis planning and gene cluster design Like ChemCrow, BioBulbul leverages chain-of-thought prompting to dynamically compose and execute domain-specific software tools BioBulbul incorporates different types of tools for planning biosynthesis pathways The system utilizes template-based and template-free planning tools to discover non-obvious pathways that constitute potential blind spots in DNA synthesis screening With BioBulbul, a simple natural language prompt in the form of “Synthesize [compound name]” is sufficient for the LLM agent to select and invoke tools for biosynthesis planning followed by genetic design The final result is a single spreadsheet with sequences for synthesizable DNA fragments that are compatible with a standard Gibson assembly protocol which LLMs are presently capable of generating as well
Technical Barriers to Accessibility
There are at least three barriers standing in the way of LLM-enabled biosynthesis of small molecule toxins The first and arguably least consequential of these barriers is the relative dearth of freely available open-source biodesign tools for biosynthesis planning and gene cluster design The second barrier to LLM-enabled toxin biosynthesis is the availability of suitable databases and training data for biodesign tools The third and currently strongest barrier to LLM-enabled toxin biosynthesis is the relative complexity of many known pathways (i.e., more than six reaction steps) and the enzymes that are often involved
Practical Mitigation Strategies
Given the certain erosion of the first two barriers with time and the indeterminate robustness of the third barrier, it is prudent to consider practical mitigations for the potential threat of LLM-enabled toxin biosynthesis Practical mitigations include:
-
Screening purchases for control of potential hazards, which involves screening DNA synthesis orders for sequences of enzymes that are more likely than not to be involved in toxin biosynthesis A second pillar of industry-wide best practices could leverage existing control strategies used by chemical suppliers and extend them to include screening purchasers of commercially available precursor compounds necessary for the biosynthesis of small molecule toxins
-
Sustainable threat landscape surveillance, which involves developing an agentic AI-assisted human-in-the-loop workflow to detect and assess emerging biodesign tools and datasets before they become widely available This approach could involve automating the identification of emerging risks through continuous observation of scientific literature reviews, social media listening, and website change detection
-
LLM guardrails to limit risk and enable detection, such as biasing an LLM to return pathways that are easily detected via screening due to the inclusion of controlled precursors or pathways that synthesize less toxic alternatives
Conclusion and Recommendations
We recommend the pursuit of practical mitigations and safeguards spanning existing routes for both biomanufacturing and biodesign For biomanufacturing, these safeguards include expanding DNA synthesis screening pipelines to include sequences for “chokepoint” enzymes primarily involved in the biosynthesis of toxins and extending chemical-supplier controls to precursor compounds for toxin biosynthesis For biodesign, we recommend developing automated, potentially agentic approaches that focus on continuous threat surveillance to triage emerging biodesign tools and datasets before they become widely available We also recommend conducting further research into LLM guardrails to make malicious use of LLMs for biodesign less effective and easier to detect The ultimate success of these proposed mitigations will depend on continuing investment into biosecurity research and coordination between stakeholders in government, the biomanufacturing industry (including organizations such as the SBRC), and groups developing LLMs and biodesign tools By investing in practical mitigations, engaging stakeholders, and developing standards for screening and rapid reporting, the biosecurity community can greatly reduce the likelihood that small molecule toxins are synthesized by non-expert actors using advanced AI agents
Acknowledgements
This work was supported by the Defense Advanced Research Projects Agency (DARPA) Biological Technologies Office (BTO) under award number HR001124C0470 The views, opinions, and/or findings expressed are those of the author(s) and should not be interpreted as representing the official views or policies of DARPA or the U.S. Government This document does not contain technology or Technical Data controlled under either the U. p. 7 Government. This document does not contain technology or
Technical Data controlled under either the U. Government. This document does not contain technology or
Technical Data controlled under either the U. This document does not contain technology or
Technical Data controlled under either the U. International Traffic in Arms
Regulations or the U. This document does not contain technology or
Technical Data controlled under either the U. International Traffic in Arms
Regulations or the U.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
References
Appleton, Evan, Curtis Madsen, Nicholas Roehner, and Douglas Densmore. 2017. Any such screening would
need to be implemented in a manner consistent with FTC guidance and emerging
privacy legislation, with transparent notice to customers. Techniques from financial
fraud detection (Motie and Raahemi 2024) and consumer behavior analysis
(Adulyasak et al. " AI & society 39 (4): 2099-2128. Boiko, Daniil A. Boiko, Daniil A. , Robert MacKnight, and Gabe Gomes. " arXiv, 2023,
arXiv:2304. Bran, Andres M. "Augmenting large language models with chemistry
tools," Nature Machine Intelligence 6: 525-535. Das, Avisha, Amara Tariq, Felipe Batalini, Boddhisattwa Dhara, and Imon
Banerjee. Annual
Symposium Proceedings 2024 (2025): 339–48. Delépine, Baudoin, Thomas Duigou, Pablo Carbonell, and Jean-Loup Faulon. "RetroPath2. p. 8 Delépine, Baudoin, Thomas Duigou, Pablo Carbonell, and Jean-Loup Faulon. "RetroPath2. Sustainable success of this strategy would be made possible in part by
automating the identification of emerging risks. This includes adapting existing and
emerging tools built or fine-tuned for scientific literature reviews (Dennstädt et al. "
Systematic Reviews 13 (158). https://doi. "
Systematic Reviews 13 (158). https://doi. "
Systematic Reviews 13 (158). https://doi. "
Systematic Reviews 13 (158). https://doi. "
Systematic Reviews 13 (158). https://doi. https://doi. 1186/s13643-024-02575-4. 1186/s13643-024-02575-4. Gricourt, Guillaume, Philippe Meyer, Thomas Duigou, and Jean-Loup Faulon. " ACS Synthetic Biology 13 (8): 2276–2294. Hérisson, Joan et al. https://doi. 1038/s41467-022-32661-x. 1038/s41467-022-32661-x. Joel, Sathvik, Jie Wu, and Fatemeh Fard. https://doi. 1145/3770084. 1145/3770084. Kelwick, Richard, Laura Bowater, Kay H. Kelwick, Richard, Laura Bowater, Kay H. Yeoman, and Richard P. " FEMS Microbiology Letters 362 (16): fnv129. Kitchens, Brent, Jennifer L. Kitchens, Brent, Jennifer L. Claggett, and Ahmed Abbasi. "
Management Information Systems Quarterly 48 (3): 899–930. Lin, Anqi et al. Mikriukov, Andrei. "AI Tools for Automating Systematic Literature
Reviews. " In Proceedings of the 2025 International Conference on
Software Engineering and Computer Applications (SECA '25). Association
for Computing Machinery, New York, NY, USA, 25–30. https://doi. 1145/3747912. 1145/3747912. Millett, Piers et al. " Health Security 21 (6): 521–529. Motie, Soroor and Bijan Raahemi. " Expert Systems with Applications. 240: 122156. 240: 122156. Ross, Ron, Mark Winstead, and Michael McEvilley. "Engineering
Trustworthy Secure Systems. " NIST Special Publication NIST SP 800-
160v1r1. 800-160v1r1. Souly, Alexandra et al. " arXiv, 2025, arXiv:2510. Tu, Zhengkai et al. " Accounts of Chemical Research 58 (11): 1764-1775. Urbina, Fabio et al. " Nature Machine Intelligence 4: 189-191. Wilary, Damian M. Wilary, Damian M. and Jacqueline M. "ReactionDataExtractor 2. p. 9 and Jacqueline M. "ReactionDataExtractor 2. " Journal of Chemical Information and Modeling 63 (19): 6053-
6067. Zipf, George Kingsley. Zipf, George Kingsley. Human Behavior and the Principle of Least Effort; and
Introduction to Human Ecology. Human Behavior and the Principle of Least Effort; and
Introduction to Human Ecology. Cambridge, Mass: Addison-wesley Press,
1949.Design Automation in Synthetic Biology.
Cold Spring Harbor Perspectives in Biology 9 (4): a023978. Adulyasak, Y. p. 5Where the paper says it
Using AI to detect panic buying and improve products distribution amid pandemic.
AI & society 39 (4): 2099-2128. Boiko, Daniil A. p. 8Where the paper says it
Where the paper says it
Emergent autonomous scientific research capabilities of large language models.
arXiv, 2023, arXiv:2304.05332. Bran, Andres M. p. 8Where the paper says it
Augmenting large language models with chemistry tools,
Nature Machine Intelligence 6: 525-535. Das, Avisha, Amara Tariq, Felipe Batalini, Boddhisattwa Dhara, and Imon Banerjee. p. 8Where the paper says it
Exposing Vulnerabilities in Clinical LLMs Through Data Poisoning Attacks: Case Study in Breast Cancer.
AMIA Annual Symposium Proceedings 2024 (2025): 339–48. Delépine, Baudoin, Thomas Duigou, Pablo Carbonell, and Jean-Loup Faulon. p. 8Where the paper says it
RetroPath2.0: A retrosynthesis workflow for metabolic engineers. p. 8
Metabolic Engineering 45: 158-170. Dennstädt, F. p. 5Where the paper says it
Where the paper says it
Where the paper says it
Title and abstract screening for literature reviews using large language models: an exploratory study in the biomedical domain.
Systematic Reviews 13 (158). https://doi. p. 8Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Artificial Intelligence Methods and Models for Retro-Biosynthesis: A Scoping Review.
ACS Synthetic Biology 13 (8): 2276–2294. Hérisson, Joan et al. p. 8Where the paper says it
The automated Galaxy-SynBioCAD pipeline for synthetic biology design and engineering.
Nature Communications 13: 5082. https://doi.org/10.1038/s41467-022-32661-x. p. 8Where the paper says it
Where the paper says it
A Survey on LLM-based Code Generation for Low-Resource and Domain-Specific Programming Languages.
ACM Transactions on Software Engineering and Methodology 1049-331X. https://doi.org/10.1145/3770084. p. 8Where the paper says it
Where the paper says it
Where the paper says it
Promoting microbiology education through the iGEM synthetic biology competition.
FEMS Microbiology Letters 362 (16): fnv129. Kitchens, Brent, Jennifer L. p. 8Where the paper says it
Where the paper says it
Timely, Granular, and Actionable: Designing a Social Listening Platform for Public Health.
Management Information Systems Quarterly 48 (3): 899–930. Lin, Anqi et al. p. 8Where the paper says it
Bridging artificial intelligence and biological sciences: a comprehensive review of large language models in bioinformatics.
Briefings in Bioinformatics 26 (4): bbaf357. Mikriukov, Andrei. p. 9Where the paper says it
AI Tools for Automating Systematic Literature Reviews.
In Proceedings of the 2025 International Conference on Software Engineering and Computer Applications (SECA '25). Association for Computing Machinery, New York, NY, USA, 25–30. p. 9Where the paper says it
Where the paper says it
Where the paper says it
Beyond Biosecurity by Taxonomic Lists: Lessons, Challenges, and Opportunities.
Health Security 21 (6): 521–529. Motie, Soroor and Bijan Raahemi. p. 9Where the paper says it
Financial fraud detection using graph neural networks: A systematic review.
Expert Systems with Applications. 240: 122156. p. 9Where the paper says it
Where the paper says it
Engineering Trustworthy Secure Systems.
NIST Special Publication NIST SP 800-160v1r1. p. 9Where the paper says it
Where the paper says it
Poisoning attacks on LLMs require a near‑constant number of poison samples.
arXiv, 2025, arXiv:2510.07192. Tu, Zhengkai et al. p. 9Where the paper says it
ASKCOS: Open-Source, Data-Driven Synthesis Planning.
Accounts of Chemical Research 58 (11): 1764-1775. Urbina, Fabio et al. p. 9Where the paper says it
Dual use of artificial intelligence-powered drug discovery.
Nature Machine Intelligence 4: 189-191. Wilary, Damian M. p. 9Where the paper says it
Where the paper says it
ReactionDataExtractor 2.0: A Deep Learning Approach for Data Extraction from Chemical Reaction Schemes. p. 9
Journal of Chemical Information and Modeling 63 (19): 6053-6067. Zipf, George Kingsley. p. 9Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Future Research Directions
The degree to which such novel pathways can be implemented in practice remains underexplored More research is needed to ensure that there is high sensitivity and specificity in only returning biased pathways to determined threat actors, and such that the guardrails are robust to model finetuning and other methods to bypass them Collaboration with LLM developers to explore safeguard mechanisms, such as selective biasing, may be a fruitful research direction The scale and longevity of the risk posed by the AIdriven rapid evolution of biological design tools necessitates a systematic approach to threat landscape surveillance, vulnerability assessment, and hazard identification Potentially hazardous biodesign tools and datasets identified by this type of continuous observation would need triaging prior to full assessment
Recommendations for Action
We recommend the pursuit of practical mitigations and safeguards spanning existing routes for both biomanufacturing and biodesign For biomanufacturing, these safeguards include expanding DNA synthesis screening pipelines to include sequences for “chokepoint” enzymes primarily involved in the biosynthesis of toxins and extending chemical-supplier controls to precursor compounds for toxin biosynthesis For biodesign, we recommend developing automated, potentially agentic approaches that focus on continuous threat surveillance to triage emerging biodesign tools and datasets before they become widely available We also recommend conducting further research into LLM guardrails to make malicious use of LLMs for biodesign less effective and easier to detect The ultimate success of these proposed mitigations will depend on continuing investment into biosecurity research and coordination between stakeholders in government, the biomanufacturing industry (including organizations such as the SBRC), and groups developing LLMs and biodesign tools By investing in practical mitigations, engaging stakeholders, and developing standards for screening and rapid reporting, the biosecurity community can greatly reduce the likelihood that small molecule toxins are synthesized by non-expert actors using advanced AI agents
This work was supported by the Defense Advanced Research Projects Agency (DARPA) Biological Technologies Office (BTO) under award number HR001124C0470 The views, opinions, and/or findings expressed are those of the author(s) and should not be interpreted as representing the official views or policies of DARPA or the U.S. Government This document does not contain technology or Technical Data controlled under either the U.S. International Traffic in Arms Regulations or the U.S.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Improve agentic systems to dynamically compose and execute domain-specific software tools by leveraging chain-of-thought prompting and retrieval augmented generation to enhance LLM responses with highly specific biochemical and genetic information.
-
Enable non-expert users, defined as
someone with an introductory understanding of biology and laboratory techniques,
to perform biosynthesis planning including identifying enzymes needed for target small molecule synthesis (retrobiosynthesis) and designing gene clusters to express these enzymes using simple natural language prompts like “Synthesize [compound name]”. -
Integrate tools from the Galaxy SynBioCAD pipeline into agentic workflows to allow LLM agents to perform tasks such as selecting specific enzymes, designing genes to express those enzymes, and planning the assembly of biosynthesis gene clusters from smaller DNA fragments into constructs ready for expression.
-
Enhance pathway discovery by incorporating both template-based planners and template-free planners within the system so it can uncover
non-obvious pathways for synthesizing a molecule that constitute potential blind spots in screening that experts may overlook.
-
Develop automated, potentially agentic approaches focused on continuous threat surveillance to detect emerging biodesign tools and datasets before they become widely available, adapting concepts from cybersecurity to inform this workflow.
-
Implement LLM guardrails such that when prompted for a controlled toxin pathway, the LLM is biased to return pathways that are
easily detected via screening due to the inclusion of controlled precursors, pathways that are more complex and thus less likely to work, or pathways that synthesize less toxic alternatives.
Abstract
The rapid democratization of biotechnology, enhanced by large language models (LLMs) and agentic software, is eroding longstanding barriers that have limited biosynthesis planning to experts. We present BioBulbul, an agentic system that couples LLMs with open-source biodesign tools for retrobiosynthesis, enzyme selection, gene cluster design, and DNA assembly planning. Using simple natural language prompts, BioBulbul autonomously generates plausible biosynthesis pathways and corresponding gene cluster sequences for several known small molecule toxins, demonstrating that non-experts could feasibly access them using an agentic system. Our red-teaming experiments confirm the technical viability of LLM-driven toxin biosynthesis while highlighting three principal barriers: scarcity of open-source biodesign tools, limited structured enzymatic reaction data, and the complexity of many small molecule toxin pathways. We argue that these barriers are likely to diminish within a decade, expanding the biological threat landscape. To mitigate emerging risks, we recommend expanding DNA synthesis screening to include chokepoint enzymes, extending chemical supplier controls to precursor compounds, establishing continuous AI-assisted threat surveillance pipelines, and developing robust LLM guardrails. Coordinated investment across government, industry, and AI developers is essential to safeguard against the misuse of agentic biosynthesis planning.
Sources
- Emergent autonomous scientific research capabilities of large language models
- Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples
Related papers
- Physical disturbance expands the climatic envelope of annual plant dominance
- Current validation practice undermines surgical AI development
- QUAREP-LiMi: A community-driven initiative to establish guidelines for quality assessment and reproducibility for instruments and images in light microscopy
- FISH Omics Format: a standard enabling FAIR chromatin tracing data sharing across repositories
- Cancer genomics, targeted therapy to precision oncology: the treatment paradigm is a holistic approach to cancer therapeutics
- The Kinetic Basis of the Genetic Code