AI for Biologists: The Knowledge Boundaries of Foundation Models in the Age of Agentic Systems

summary

The gist

Artificial intelligence (AI) is changing how biological research is done by developing foundation models and agentic systems that are often difficult to trust without a clear understanding of their…

In short

The paper addresses how foundation models change biological research by creating agentic systems that are hard to trust without knowing their limits. It proposes five design elements—data, architecture, training scheme, evaluation scheme, and post-analysis—as the framework to define a model's knowledge boundary. The goal is to standardize these limits so researchers can reliably use AI outputs.

Key concepts

Knowledge Boundary
This defines the specific range of inputs (species, assays), tasks, and conditions where a foundation model's outputs are guaranteed to be reliable. It sets clear expectations for what the model knows and what it does not know.
Design Elements
These are five core components—data, architecture, training scheme, evaluation scheme, and post-analysis—that collectively determine the knowledge boundary of a foundation model. Understanding these elements is crucial for assessing a model's reliability.
Agent Routing
Agents need to know precisely where each foundation model's knowledge boundary lies so they can correctly route queries. This prevents errors by ensuring that an agent only asks a model questions within its proven, reliable operational limits.

Terminology used across episodes

This episode discusses

The paper

AI for Biologists: The Knowledge Boundaries of Foundation Models in the Age of Agentic Systems · Read on arXiv

Tian Cai, Andrew Kleist, Jaimin Patel, Duccio Malinverni, Alexander K. Taylor, Malvika Sharan, Renato Umeton, M. Madan Babu

Center of Excellence for Data-Driven Discovery, Department of Structural Biology, St. Jude Children’s Research Hospital · Department of Computer Science, University of California, Los Angeles

Artificial intelligence (AI) is changing how biological research is done. Biological foundation models trained on millions of nucleotides, protein, and single-cell measurements predict protein structures, mutation effects, and cellular responses to genetic perturbations. Agentic systems, AI assistants built on large language models, can run multistep analyses from plain-language requests and increasingly call foundation models as tools. Yet it is often unclear when these models' outputs can be trusted. We frame this as a model's knowledge boundary: the range of inputs, tasks, and conditions over which its outputs are reliable. This review explains how foundation models are built and proposes that five design elements (data, architecture, training scheme, evaluation scheme, and post-analysis) define this boundary. We examine evidence on where current models succeed or fail and present standardized descriptions of model limits so that researchers, and the agents acting for them, can use these tools reliably.

Transcript

Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.

Ines: We're looking at the paper: "AI for Biologists".

Marcus: Here's the gist of the paper. Artificial intelligence (AI) is changing how biological research is done by developing foundation models and agentic systems that are often difficult to trust without a clear understanding of their knowledge boundaries The paper frames this issue as defining a model’s knowledge boundary, which is the range of inputs, tasks, and conditions over which its outputs are reliable This review explains how foundation models are built and proposes that five design elements—data, architecture, training scheme, evaluation scheme, and post-analysis—define this boundary The goal is to present standardized descriptions of model limits so that researchers and agents can use these tools reliably.

Title and authors: Ines: So we're talking about this paper today, "AI for Biologists: The Knowledge Boundaries of Foundation Models in the Age of Agentic Systems." It's a big topic because it gets right to the core problem when we start letting these foundation models and agentic systems do research.

Marcus: Yeah, it tackles that trust issue head-on. It frames the whole thing around defining what a model actually knows, which they call the knowledge boundary—basically the limits of what inputs, tasks, and conditions it can handle reliably.

Yuki: I wonder how this impacts us in practice. If we don't know where those boundaries are, we risk letting agents pull results that are completely outside their competence.

Ines: Exactly. The paper proposes that these boundaries aren't just some abstract thing; they can be defined by five design elements: the data it was trained on, the architecture itself, how it was trained, how we evaluate it, and even what post-analysis methods we use to check the results.

Marcus: That sounds like a concrete way to stop relying on opaque outputs. They want standardized descriptions of these limits so that researchers and the agents using them know exactly what they're dealing with before they trust a result.

Ines: It moves us from just saying "this model works well here" to providing a structured description of *why* it works well here and not somewhere else.

Marcus: That sounds like it could prevent some really messy situations where an agent tries to use a variant score from a model that was never trained on population data, for example.

Yuki: That’s important because population genetics is so sensitive to those kinds of training biases, and knowing the boundary helps us understand why certain results might be unreliable for our specific species or cohort.

Ines: And they suggest we need to implement agent routing based on these model boundaries. So if an agent has a query, it needs to know which model's boundary lies where so it routes the query correctly.

Marcus: So it’s not just about building better models but about building better systems around them, making sure the system knows its own limitations and directs work properly.

Ines: Right. And they talk about closing that knowledge gap through iterative measurement loops. They suggest setting up self-driving laboratories where robotic platforms run experiments chosen by a model, and then the model returns those results for another round of testing.

Marcus: That’s a smart way to extend the boundary without just throwing more raw data at it, which is what they seem worried about. It’s about using new measurements to define where the knowledge still extends.

Yuki: I like that idea of iterative measurement because it grounds the model's knowledge in real-world, verifiable data points rather than letting it drift into unverified territory.

Ines: So, we're looking at five specific design elements to map out these boundaries and then using routing and closed loops to keep those boundaries as accurate as possible.

Marcus: It sounds like the practical application is moving toward making AI tools more accountable by forcing them to state their operational limits upfront.

Yuki: It gives us a framework to ask tough questions about the models we use, rather than just accepting their predictions at face value.

Ines: So, when we wrap up this discussion on "AI for Biologists: The Knowledge Boundaries of Foundation Models in the Age of Agentic Systems," it really boils down to standardizing how we describe model limits across five key design aspects.

Marcus: And the bigger implication is that this standardization should allow us to build agents that can actually operate reliably within those known constraints, instead of hallucinating answers outside them.

Yuki: For population genetics, this means being much more careful about where we apply predictions from these foundation models when they touch on evolutionary history or species-specific traits.

Ines: We’re moving toward a system where we know *when* to trust a prediction and *why*, based on the specific data, architecture, and training scheme behind it.

Marcus: It’s about creating a verifiable chain of logic for these complex biological predictions, which is something we really need right now.

Yuki: It gives us a language to discuss uncertainty in biology that isn't just vague; it’s tied directly to the model's design choices.

Ines: So, the paper suggests this structured approach to defining knowledge boundaries is essential for safely integrating foundation models into our research workflows.

Marcus: It sets a path forward for how we can design those agentic systems so they don't just run off into unsupported territory when they make a request.

Yuki: It’s about respecting the limits of the data and the architecture we feed them, which is a huge consideration when dealing with complex biological systems.

Ines: That’s what this paper sets up: defining those limits clearly so we can use these tools reliably in discovery rather than just for novelty.

The paper's summary: Ines: So, we're looking at this paper today, "AI for Biologists: The Knowledge Boundaries of Foundation Models in the Age of Agentic Systems." It’s about how to define exactly where these foundation models stop being trustworthy when they start acting like agents.

Marcus: Yeah, it tackles that trust problem head-on by focusing on what we call the knowledge boundary. It's basically drawing a line around what inputs and tasks a model can handle reliably before we let it do real research.

Yuki: I’m thinking about how this impacts us in the lab. If we don't know where those boundaries are, we risk an agent pulling out some result that’s just completely outside the model's competence.

Ines: Exactly. The paper proposes that these boundaries aren't just abstract ideas; they can be defined by five specific design elements. We’re talking about the data it was trained on, the architecture itself, how it was trained, how we evaluate it, and even what post-analysis methods we use to check the results.

Marcus: That sounds like a concrete way to stop relying on outputs that are just opaque or unclear. They want standardized descriptions of these limits so researchers and agents know exactly what they're dealing with before they trust the result.

Yuki: And for population genetics, knowing those boundaries is huge because we deal with really sensitive things like species-specific traits or evolutionary history. Knowing the training data species are important for us to understand why a model might fail on something new.

Ines: Right. They suggest we need to implement agent routing based on these model boundaries. So if an agent has a query, it needs to know which specific model's boundary lies where so it routes the query correctly and doesn't try to use the wrong tool for the job.

Marcus: That means we aren't just building better models; we’re building better systems *around* them, making sure those systems know their own limits and direct work properly instead of letting them run wild.

Yuki: And they talk about closing that knowledge gap through iterative measurement loops. It sounds like setting up self-driving laboratories where robotic platforms test experiments the model picks, and then the model feeds those results back for another round of testing.

Ines: That’s a smart way to extend a model’s knowledge boundary without just throwing more raw data at it; it's about using new measurements to define where the knowledge still extends.

Marcus: It sounds like the practical goal here is making AI tools more accountable by forcing them to state their operational limits right upfront, instead of just giving us a prediction and hoping for the best.

Yuki: It gives us a framework to ask tough questions about these models we use, rather than just accepting their predictions at face value.

Ines: So, the paper boils down to standardizing how we describe those limits across five key design aspects, using routing and closed loops to keep those boundaries as accurate as possible.

Marcus: It sets a path forward for how we can design those agentic systems so they don't just run off into unsupported territory when they make a request.

Yuki: For population genetics, this means being much more careful about where we apply predictions from these foundation models when they touch on evolutionary history or species-specific traits.

Ines: We’re moving toward a system where we know *when* to trust a prediction and *why*, based on the specific data and architecture behind it.

Marcus: It’s about creating a verifiable chain of logic for these complex biological predictions, which is something we really need right now.

Yuki: It gives us a language to discuss uncertainty in biology that isn't just vague; it’s tied directly to the model's design choices.

Ines: So, the paper suggests this structured approach to defining knowledge boundaries is essential for safely integrating foundation models into our research workflows.

Marcus: And the bigger implication is that this standardization should allow us to build agents that can actually operate reliably within those known constraints, instead of hallucinating answers outside them.

Yuki: It’s about respecting the limits of the data and the architecture we feed them, which is a huge consideration when dealing with complex biological systems.

Ines: That’s what this paper sets up: defining those limits clearly so we can use these tools reliably in discovery rather than just for novelty.

Marcus: The next thing to look at is how these boundary statements actually get implemented in the code, and whether that's even possible with the current architecture of these foundation models.

The paper's improvements: Marcus: So, we just talked about how the paper frames knowledge boundaries using five design elements—data, architecture, training scheme, evaluation scheme, and post-analysis—to define what a foundation model actually knows reliably.

Ines: Right. And now the paper moves on to suggesting how we can actually use this information practically to make these tools safer for biological research.

Marcus: They propose three big improvements starting with explicit knowledge boundary statements for each foundation model, which means releasing them with a structured description of their limits right there.

Yuki: That sounds really useful because it lets a biologist look at the data and say "I don't trust this result if it’s outside the species or conditions listed in their training set."

Ines: Exactly. It helps us decide whether to trust a result before an agent tries to use it. Then they suggest using those boundaries for agent routing.

Marcus: Agents need to know where each model’s boundary lies so they can correctly route a query, which stops errors from outside a model's competence from just propagating through the system.

Yuki: So, instead of an agent blindly asking Model A a question when it should be asking Model B, it has this map telling it exactly which tool to use.

Ines: It’s about preventing that kind of mistake where something trained without population data gets treated as authoritative just because the number looks good.

Marcus: And they talk about closing the knowledge gap through iterative measurement loops. They want us to combine self-driving laboratories with agents planning experiments and foundation models proposing candidates for testing.

Yuki: That sounds like a way to extend a model’s boundary by running new measurements rather than just feeding it more of the same old data.

Ines: It grounds the model's knowledge in real, verifiable biology, which is much better than letting it drift into unsupported territory on its own.

Marcus: So we’re moving away from just static descriptions of a model and toward these dynamic loops where the model actively seeks out new information to define its edges.

Yuki: That's about making the boundary an active thing, something that grows with good measurements rather than just a fixed line drawn in the sand.

Ines: It’s a big shift from passive tool use to active knowledge expansion guided by verifiable experimentation.

Marcus: The implication is that we can build much more robust research workflows where we have better control over what the AI is allowed to do.

Yuki: If we can define these boundaries clearly, it gives us a way to rigorously test the biological claims those models make in a way that’s actually supported by the species itself.

Ines: So, the improvements focus on making model limits explicit, using them for agent control, and using closed loops to actively grow those limits with new data.

Marcus: The next thing we need to figure out is the actual engineering challenge of implementing these dynamic routing and measurement loops in a way that’s computationally feasible for these massive foundation models.

Conclusion: Tom: So we've covered how the paper frames defining knowledge boundaries using five design elements—data, architecture, training scheme, evaluation scheme, and post-analysis—to set limits for foundation models acting as agents.

Marcus: And we discussed how these boundaries allow us to route queries correctly and use closed loops for iterative measurement to grow those limits dynamically.

Ines: It boils down to making the model's operational limits explicit so we know when a prediction is safe versus when it’s just guessing outside its training data.

Yuki: For population genetics, this framework gives us a way to connect what the AI says back to the actual history of the species and what our limited sample can actually support.

Marcus: It’s about creating accountability so we don't just take these model outputs at face value when they come from a model that doesn't know its limits.

Ines: We’re moving toward a system where we know *when* to trust a prediction and *why*, based on the specific data and architecture behind it.

Yuki: That gives us a language to discuss uncertainty in biology that isn't vague; it’s tied directly to the model's design choices.

Marcus: The practical challenge now is figuring out how we actually implement this boundary statement system within the code of these foundation models efficiently.

Ines: It’s about building systems that can self-report their limits without adding massive overhead to every single computation.

Yuki: And if we get that right, it opens up a lot more reliable ways for AI to contribute to discovery in our field.

Marcus: So, the paper "AI for Biologists: The Knowledge Boundaries of Foundation Models in the Age of Agentic Systems" gives us a clear blueprint for designing research systems that respect the actual capabilities and limitations of the models we use.

Ines: It’s about moving past just building bigger models to building smarter, more responsible ones that can operate safely within defined biological parameters.

Yuki: It's a necessary step for our field to integrate these powerful tools in a way that actually supports our understanding of evolution and species structure.

Marcus: Next time, we’ll be looking at how these boundary descriptions translate into the actual engineering challenges of agent routing and dynamic measurement loops.

More episodes

← Home