Do Perturbation Predictors Learn Gene Interactions? Diagnosing Extrapolation in Agentic Screening

arXiv:2610.18880 · cs.LG, cs.AI, q-bio.GN · Submitted 2026-10-01 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "Do Perturbation Predictors Learn Gene Interactions? Diagnosing Extrapolation in Agentic Screening".

Jane: The gist The difference between model prediction and an experimentally grounded additive reference, called additive disagreement (AD),

Tom: First, who's behind it and why it matters.

Title and authors: Tom: We talked about the title and authors, and the authors are Ziyuan Gao from the Department of Computer Science and Technology at the University of Cambridge, who set out to diagnose extrapolation in agentic screening.

Jane: The paper is really focusing on how we can tell when a model is making a big guess about an interaction effect it hasn't seen before.

Lu: They introduce this concept called additive disagreement or AD, which is essentially the difference between what the model predicts for a pair and what you get just by adding up their known single-gene effects.

Meng: So, if we put that in plain English, AD measures how much extra interaction effect the model introduces for a combination it hasn't observed. It directly targets that extrapolation component.

Tom: That’s right, and it’s not just about input novelty like kNN distance; AD directly targets the component that requires compositional generalization, which is what we think is important here.

Jane: And the paper sets up a comparison between this AD signal and other reliability signals like kNN distance and Mahalanobis distance to see which ones are actually better under compositional shift.

Lu: They compare it across eight different prediction methods spanning six modeling paradigms, including Ridge regression, MLP, Transformer, scGPT based predictors, and GEARS.

Tom: The big finding there is that these specialized neural predictors don't consistently beat the simple additive baseline on unseen gene combinations across all these models.

Jane: But the real impact comes when you look at what AD tells us about prediction errors; predictions with larger deviations from that additive reference tend to have larger errors.

Meng: That’s where it gets useful for deployment, because if a model has a high AD value, we know it’s asserting a strong interaction correction for something new and unsupported.

Tom: So, the paper is essentially saying that the source of error isn't just failing to model additive effects; it's introducing interaction corrections without any transferable evidence.

Jane: And that leads us into how they propose we can use this AD signal to improve our screening process.

The paper's summary: Tom: The paper summarizes the method by decomposing each model’s output into a component supported by observed single-gene effects and a residual interaction term that the model extrapolates without direct training signal.

Jane: This residual, which they call additive disagreement, AD, quantifies how strongly a prediction departs from an experimentally grounded additive reference using this formula: AD(A, B) equals one over G times the L1 norm of the difference between the model's prediction f theta of A and B and just adding up delta A plus delta B.

Lu: So, G is just the number of evaluated genes, and that quantity AD directly targets that component that requires compositional generalization with a clear biological interpretation as the mean magnitude of the model-implied departure from additivity.

Meng: This is different from generic uncertainty measures because kNN distance and Mahalanobis distance measure input novelty; they see if the genes themselves are new, but AD sees if their interaction is new.

Tom: And they evaluate this decomposition across eight prediction methods, comparing AD with eight alternative reliability signals spanning embedding-based, output-based, and ensemble-based approaches to see what works best.

Jane: The summary shows that additive disagreement reliably identifies high-error predictions when compared to other signals like prediction magnitude and gene-wise variance.

Lu: They found that additive disagreement is the most consistent indicator of prediction failure across all evaluated compositional settings, even when moving between different experimental contexts like Norman and Joung–Zhang.

Tom: That consistency is what makes it so compelling for use in an agentic screening setting where you need a reliable signal to decide whether to trust a prediction or not.

Jane: So, the paper summarizes that AD is the most consistent ranking signal across settings, which suggests it’s robust against shifts in how the data was organized.

The paper's improvements: Tom: The paper points out a couple of ways they can improve or use this method differently to make it more practical for real-world screening.

Jane: They suggest using additive disagreement specifically for selective abstention, which means replacing high-risk model predictions with the additive reference when AD is too large.

Lu: They found that by using AD for selective abstention, the system can reduce the mean absolute error by twelve percent, nineteen percent, and even thirty-eight percent at different thresholds on the Norman dataset.

Meng: That reduction in MAE means we’re not just flagging bad predictions; we’re actually making a measurable improvement in how accurate our screening decisions become when we use this method.

Tom: And they also found that for GEARS and GEARS+scGPT, routing through this AD signal substantially reduces their errors but generally still leaves them slightly worse than always using the additive reference.

Jane: So, even when we use the AD signal to guide us away from a prediction, it doesn't always beat just sticking with the simple additive reference completely.

Lu: They also found that in GEARS predictions, there are two recurring behaviors: either they produce large interaction corrections that move predictions far from the additive reference, or the corrections are smaller but still introduce unnecessary deviations.

Meng: The gene-level analysis showed that GEARS assigns nine point seven times larger corrections to the top twenty differentially expressed genes than it does to the rest of the transcriptome, suggesting these graph-based corrections often add noise instead of providing transferable interaction signal.

Tom: That points back to the conclusion that the main source of error isn't just a failure to model additive effects; it's introducing interaction corrections without transferable evidence.

Conclusion: Jane: So, to wrap up, this paper on "Do Perturbation Predictors Learn Gene Interactions? Diagnosing Extrapolation in Agentic Screening" shows that additive disagreement is the most consistent indicator of prediction failure because it directly targets the interaction-level extrapolation.

Tom: It means we can use AD to diagnose when a model asserts an interaction correction for a combination it has never observed, which helps us pinpoint exactly where the errors are coming from.

Lu: The paper shows that specialized neural predictors don't always beat simple additive baselines on unseen combinations, but the AD signal is still much better at identifying high-error predictions compared to generic uncertainty measures like kNN distance or Mahalanobis distance.

Meng: From a deployment standpoint, using this AD signal to guide decisions can reduce the mean absolute error by up to thirty-eight percent at a fifty percent abstention threshold on the Norman dataset, which is significant for optimizing our agentic screening workflow.

Tom: It’s a practical tool because it allows us to selectively replace high-risk model predictions with the additive reference, even if it doesn't always beat the absolute best performance of any single model.

Jane: This work on AD provides a solid foundation for building more reliable agentic screening pipelines by focusing on unsupported interaction extrapolation rather than just input novelty.

Lu: They point out that a limitation is that AD measures correction magnitude rather than biological correctness, and they suggest gene-level and pathway-level disagreement measures as a natural extension to this framework.

Meng: And the paper also notes that extending this framework to higher-order combinations will require richer references than just simple pairwise additivity.

Tom: So, while it doesn't solve everything, diagnosing extrapolation via additive disagreement gives us a concrete way to filter out the noise in these complex agentic screening models.

Ziyuan Gao

University of Cambridge

cs.LG, cs.AI, q-bio.GN

Submitted: 2026-10-01

Updated: 2026-10-01

Comments: Accepted at the NeurIPS 2026 Workshop ML4Molecules: Agentic Systems for Molecular Sciences

Code: https://github.com/zygao930/OOD

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 88/100

The gist: The gist The difference between model prediction and an experimentally grounded additive reference, called additive disagreement (AD), provides the most consistent indicator of prediction failure

Key concepts

Additive Disagreement (AD): This metric calculates the difference between a model's prediction for two genes and what their individual effects would predict if they acted independently (the sum of their single-gene effects). A large AD suggests the model is making an unsupported assumption about how those two genes interact.
Extrapolation Error: This refers to the model's failure when predicting results for gene pairs it has never seen during training. Standard methods often fail because neither gene is 'unfamiliar' alone; they only fail when their specific combination is novel, which AD helps detect.
Reliability Signals: These are metrics like kNN distance or ensemble uncertainty used to gauge how confident a model is in its prediction. The paper finds these signals are inconsistent across different datasets and models, making them less trustworthy than the simpler AD metric for compositional data.

Summary

The gist The difference between model prediction and an experimentally grounded additive reference, called additive disagreement (AD), provides the most consistent indicator of prediction failure across compositional settings because it directly targets interaction-level extrapolation rather than input proximity

How it works

The paper introduces a method to diagnose extrapolation in agentic screening by decomposing model predictions into an observed additive component and an extrapolated interaction component (Page 2). For an unseen pair of genes A and B, the prediction is formalized as fθ(A, B) = (δA + δB) + ˆIθ(A, B), where δA and δB denote the measured single-gene effects and ˆIθ(A, B) is the model-implied interaction correction (Page 2). This leads to the additive disagreement (AD) being defined as AD(A, B) = 1/G fθ(A, B) − (δA + δB)1, where G is the number of evaluated genes (Page 2). A large AD indicates that the model asserts a strong interaction correction for a combination it has never observed (Page 2).

Model Performance and Limitations

The study evaluates eight prediction methods across six modeling paradigms on two combinatorial perturbation datasets, Norman and Joung–Zhang, alongside the Adamson single-gene OOD setting (Page 2). The main findings show that specialised neural predictors do not consistently outperform simple additive baselines on unseen gene combinations (Page 3). Specifically, for GEARS and GEARS+scGPT, the results indicate they never outperform the additive baseline (0/75 on Norman, 0/44 on Joung–Zhang) (Page 3). Furthermore, the study found that generic reliability signals, including kNN distance, Mahalanobis distance, and ensemble uncertainty, are substantially weaker and less consistent than AD under compositional shift (Page 4).

Reliability Signal Efficacy

The analysis of reliability signals demonstrates the superiority of AD in identifying high-error predictions (Page 4). Table 2 shows that Additive disagreement, prediction magnitude, and gene-wise variance reliably identify high-error predictions, whereas kNN and ensemble-based signals are substantially weaker (Page 4). By using AD for selective abstention, the researchers found that it reduces MAE by 12%, 19%, and 38% at the top 50% failure threshold on Norman (Page 4). This suggests that compositional errors are more closely associated with unsupported interaction corrections than with input-space novelty (Page 4).

Failure Analysis and Interpretation

The failure analysis reveals two recurring behaviors in GEARS predictions: GEARS produces large interaction corrections that move predictions far from the additive reference or the corrections are smaller but still introduce unnecessary deviations (Page 4). Gene-level analysis showed that GEARS assigns 9.7× larger corrections to the top-20 DE genes than to the rest of the transcriptome, suggesting that these graph-based corrections often add noise rather than transferable interaction signal (Page 4). The paper concludes that the main source of error is not simply a failure to model additive effects, but the introduction of interaction corrections without transferable evidence (Page 4).

Cross-Dataset Consistency

The reliability signals show varying degrees of transferability across different OOD settings (Page 4). On Norman, AD ranks errors well across all models, with the strongest correlations observed for GEARS and GEARS+scGPT (Page 4). However, on Joung–Zhang, PM and GV become substantially more variable, while AD remains consistently positive across models and both compositional datasets (Page 4). This consistency suggests that AD is the most consistent ranking signal across settings (Page 4).

Decision Utility

The final analysis tests the decision-level utility of reliability signals via a conservative fallback analysis where high-risk predictions are replaced by the additive reference (Page 4). For GEARS and GEARS+scGPT, routing substantially reduces their errors but generally remains slightly worse than always using the additive reference (Page 4). The study establishes that current signals do not fully capture when fallback is beneficial (Page 4).

Future Directions

Limitations include the requirement for observed singlegene effects for both constituent perturbations and the fact that AD measures correction magnitude rather than biological correctness (Page 9). The paper suggests that Gene-level and pathway-level disagreement measures are a natural extension (Page 9). Extending the framework to higher-order combinations will require richer references than simple pairwise additivity (Page 9). The decomposition may also generalize to other compositional scientific settings, including drug combinations and multimodal interventions (Page 9).

Declaration of LLM Usage

Large language models were used solely as writing and coding assistants during the preparation of this work (Page 19).

Ethics Statement

This work studies the reliability of perturbation-response predictors using publicly available, deidentified Perturb-seq datasets (Page 19). It involves no human subjects, no personally identifiable information, and no data collection (Page 19). This study is diagnostic in nature: by identifying when predictions rely on unsupported interaction extrapolation, it aims to reduce wasted experimental resources and misleading biological interpretations in downstream screening pipelines (Page 19). We are not aware of foreseeable harmful applications arising directly from this work (Page 19).

References

[35] Arthur E Hoerl and Robert W Kennard. Ridge regression: biased estimation for nonorthogonal problems (Page 7).

[36] Diederik P Kingma and Max Welling. Auto-encoding variational Bayes (Page 7).

[34] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need (Page 7).

[32] Britt Adamson, Thomas M Norman, Marco Jost, May Y Cho, James K Nuñez, Yuwen Chen, Jacqueline E Villalta, Luke A Gilbert, Max A Horlbeck, Marco Y Hein, Ryan A Pak, Andrew N Gray (Page 9).

[33] Julia Joung et al. A transcription factor atlas of directed differentiation (Page 9).

[5] Yusuf Roohani, Kexin Huang, and Jure Leskovec. Predicting transcriptional outcomes of novel multigene perturbations with gears (Page 7).

[7] Haotian Cui, Chloe Wang, Hassaan Maan, Kuan Pang, Fengning Luo, Nan Duan, and Bo Wang. scGPT: toward building a foundation model for single-cell multi-omics using generative ai (Page 7).

[9] Charlotte Bunne et al. How to build the virtual cell with artificial intelligence: Priorities and opportunities (Page 7).

[15] Yinuo Qu et al. Crispr-gpt for agentic automation of gene-editing experiments <ref:

Improvements for AI systems

Bold header: Implementation of Additive Disagreement (AD) as a core reliability signal for agentic workflows. This allows systems to simultaneously compute the additive reference, δA + δB, and compare it with the model output using only single-gene effect estimates. This enables autonomous perturbation screening by immediately flagging predictions that rely on unsupported interaction extrapolation.
Bold header: Development of a selective prediction mechanism based on AD for resource optimization. By using Additive disagreement to reduce MAE by 19% at 20% abstention and by 38% at 50% abstention, the system can replace high-risk model predictions with the additive reference, effectively guiding decisions in agentic perturbationscreening workflow.
Bold header: Creation of a cross-dataset reliability assessment module. This module utilizes AD to maintain consistency across different experimental contexts, noting that AD remains informative across all evaluated compositional settings and is the most consistent indicator of prediction failure.
Bold header: Integration of graph-based structure analysis for model diagnostics. The system can diagnose failures by observing that GEARS's errors are driven by the co-expression graph, as it shows co-expression degree correlates with both predicted interaction magnitude (ρ = 0.633) and prediction error (ρ = 0.626).
Bold header: Dynamic adaptation of reliability thresholds based on evaluation metrics. The system can be configured to use AD signals that are calibrated for specific goals, recognizing that AD performs strongly under full-transcriptome MAE on Norman but transfers poorly to Top20 error.

Abstract

High-throughput perturbation screens increasingly use machine-learning models to predict the effects of gene combinations that were never measured. A difficult case arises when both genes in a test pair were seen individually during training, but their combination was not. In this setting, standard OOD methods often fail because neither gene appears unfamiliar on its own. We therefore compare each model prediction with a simple additive prediction obtained by summing the two observed single-gene effects. The difference between them, which we call additive disagreement (AD), measures how much extra interaction effect the model introduces for an unseen combination. Across eight prediction methods and three Perturb-seq datasets, specialised neural predictors do not consistently outperform this simple additive baseline. More importantly, predictions with larger deviations from the additive baseline tend to have larger errors. Conventional output-based and embedding-based confidence signals are inconsistent across datasets and models, whereas AD remains informative across all evaluated compositional settings. We further study how this signal can guide decisions in an agentic perturbation-screening workflow. The code is available at https://github.com/zygao930/OOD.

Sources

Benchmarking Transcriptomics Foundation Models for Perturbation Analysis : one PCA still rules them all