Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets

summary

The gist

Protein-protein interaction (PPI) databases do not faithfully reflect biological realities; instead, they are influenced by study and technical biases that distort certain protein and interaction…

In short

This research investigates how biases in protein-protein interaction (PPI) databases lead machine learning models to learn shortcuts instead of true biological signals. The authors characterize these biases across multiple datasets and propose an open pipeline using integer linear programming to mitigate them through similarity-aware splitting and bias-minimizing negative sampling.

Key concepts

Dataset Biases
These are systematic distortions in PPI databases, such as self-interactions or taxonomic identity, that cause machine learning models to rely on these easy shortcuts instead of learning genuine biological relationships. The paper analyzes how different data sources like HIPPIE and PDB affect the prevalence and strength of these shortcuts.
Integer Linear Programming (ILP)
ILP is a mathematical optimization technique used here to create precise strategies for data splitting and negative sampling. It allows researchers to formulate problems where they can simultaneously minimize undesirable outcomes, such as data loss or distributional differences between positive and negative samples.
Shortcut Learning
Shortcut learning occurs when a machine learning model finds an unintended, simple pattern in the data—like protein identity or topology—that correlates strongly with the correct answer. The paper shows that these shortcuts are prevalent in PPI datasets and can be amplified by certain sampling methods.
G-AUDIT Framework
This is a proposed framework for systematically quantifying potential biases within PPI attributes. It measures how detectable an attribute is from the input data and its mutual information with the prediction label, helping researchers pinpoint specific shortcuts like functional relatedness.

Terminology used across episodes

This episode discusses

The paper

Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets · Read on arXiv

Friedrich-Alexander-Universität Erlangen-Nürnberg · Technical University of Munich · IEO European Institute of Oncology IRCCS

Protein-protein interaction (PPI) databases do not faithfully reflect biological realities. Instead, they are influenced by study and technical biases that distort certain protein and interaction attributes. Machine learning models can exploit these as learning shortcuts if the negative dataset is not constructed with care. So far, the shortcuts introduced during PPI dataset construction have only been examined in isolation. Here, we systematically characterize both reported and, to our knowledge, previously unreported biases in PPI datasets that lead machine learning models to learn shortcuts instead of biological signal. We analyze HIPPIE, IntAct, and STRING, dedicated PPI databases, as well as two datasets derived from 3D-structural information in the Protein Data Bank (PDB). We show that random data splitting introduces strong topological shortcuts. When train-test protein overlap is removed, the resulting datasets still retain usable shortcuts stemming from self-interactions, taxonomic identity, and functional relatedness, whose prevalence interestingly depends on the data source. We further show that sampling negatives from a set of high-confidence non-interactors, an intuitively appealing choice, can amplify the shortcut stemming from functional relatedness. To detect and mitigate these biases, we provide an open Nextflow pipeline that combines similarity-aware, data-loss-minimizing dataset splitting with bias-minimizing negative sampling, both formulated as integer linear programs. Its key concept of quantifying biases to minimize them through optimization-based negative sampling can, in principle, be extended to any machine learning problem where the pool of negative candidates is much larger than the positives and is thus of interest also beyond PPI prediction.

Transcript

Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.

Ines: We're looking at the paper: "Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets".

Title and authors: Ines: So we're looking at this paper today called "Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets". It seems like the authors are saying that these protein interaction databases aren't actually perfect pictures of reality, and they are influenced by biases from how they were built.

Marcus: Yeah, it sounds like they're pointing out that machine learning models can get tricked into learning these shortcuts instead of the real biological connections. It’s a warning about taking results at face value without checking the data construction.

Ines: Exactly. The paper starts by showing that these databases have biases that distort certain protein and interaction attributes, and they show how machine learning models exploit those biases if you don't build the negative dataset carefully > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg1>

Marcus: They look at a few specific databases like HIPPIE, IntAct, and STRING, plus some datasets from the Protein Data Bank that come from three dee structures. And they find that just splitting data randomly can introduce really strong topological shortcuts > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg2>

Yuki: It's interesting to hear about how these biases manifest because from a population genetics standpoint, we know that the sample size and source matter a huge amount for what we can actually infer about evolution > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg3>

Ines: Right. They show that even if you remove the overlap between your training and testing proteins, there are still usable shortcuts left from things like self-interactions, taxonomic identity, and functional relatedness > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg2>

Marcus: And they also find that sampling negatives from a set of high-confidence non-interactors can actually make the shortcut coming from functional relatedness even stronger > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg3>

Yuki: That suggests that if you're only looking at functional relatedness, you might be picking up artifacts about pathways and complexes instead of the actual individual protein pairs > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg3>

Title and authors: Ines: The authors then provide a whole pipeline, an open Nextflow pipeline, that uses integer linear programs to do two things: similarity-aware splitting to minimize data loss while controlling the sizes of your training, validation, and test sets > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg3>

Marcus: And they also have this bias-minimizing negative sampling method using integer linear programs to jointly reduce the distributional differences between what you sample as negatives and your positives > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg3>

Yuki: So, if I'm trying to use this for something like predicting interactions between new genes, would these methods help me get closer to the true biological signal rather than just learning statistical tricks > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg3>

Ines: The results show that even a Random Forest baseline can achieve near perfect performance in the worst-case scenario, depending on which dataset you're using > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg2>

Marcus: They also identify topology as the attribute that is most detectable across all these different datasets, and they say it has the highest utility for performance improvements > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg2>

Yuki: It sounds like topology is a really robust feature because it seems to hold up regardless of whether you're looking at sequence, species, or functional grouping > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg3>

Ines: They also point out that the self-interaction shortcut is dominant in structural datasets like PDB Dimers, where eighty percent of all positive interactions there are actually self-interactions > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg2>

Marcus: That's a big finding for structural data because it means you can't just trust the interaction counts without accounting for those self-interactions first > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg2>

Yuki: It does make sense if the structure itself is heavily biased toward homo dimers, which limits how much we can use that structural information to improve other datasets > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg2>

Title and authors: Ines: The authors say their most rigorous mitigation strategy, the integer linear program based sampling, still leaves some bias in PDB Dimers and PINDER because of those high self-interaction proportions > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg3>

Marcus: So they're not claiming this is a perfect fix, which is realistic, but they're showing you how to get the least biased datasets possible with this specific combination of splitting and sampling > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg3>

Yuki: It implies that if we want to use structural data effectively, we need to be very careful about what kind of interactions we're expecting it to show us > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg2>

Ines: So, the big implication is that reported performance figures for these models are only trustworthy if you look at the specific splitting and sampling strategy they used to get those results > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg1>

Marcus: And it also means the choice of your starting PPI database matters just as much as how you train the model, which is a bit frustrating because you can't always switch databases easily > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg1>

Yuki: It shifts the focus from just getting high accuracy numbers to understanding *why* those numbers are high or low depending on the underlying data source > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg3>

Ines: To wrap up, this paper with its detailed analysis of HIPPIE, IntAct, and STRING shows us how to systematically identify these shortcuts and a concrete pipeline to minimize them through optimization > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg3>

Marcus: It gives us a tool the community can use to apply these bias-aware procedures to their own data, which is a big deal for making more reliable predictions > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg3>

Title and authors: Yuki: I think the main risk they flag is over-interpreting a dataset as unbiased after you've done one mitigation step, which is something we need to keep in mind when applying these techniques > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg1>

Ines: And that’s where the real work lies, figuring out how to quantify those biases with frameworks like G-AUDIT and then minimizing them through optimization > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg3>

Marcus: It points toward a future where we can use this kind of optimization for any problem where you have a huge pool of negative candidates, like predicting domain-domain or protein-ligand interactions > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg3>

Yuki: It also suggests that if we want to connect these PPI findings to broader evolutionary patterns, we need to be more rigorous about the sample support because functional relatedness can be a major confounder > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg3>

Ines: So, this paper is less about a single perfect result and more about providing a structured way to audit your data for hidden biases so you can trust the resulting machine learning predictions more > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg1>

Marcus: It's a lot of practical advice for anyone working with large biological cohorts who are worried about batch effects or selection bias, and it’s not just theoretical stuff > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg3>

Yuki: The G-AUDIT framework sounds like a good way to start quantifying what's going on before you even try to fix it, which is a smart approach for population genetics > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg3>

Ines: We'll stick with this paper as a guide for how we need to look at the data we feed into these sequence and structure analysis models > Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets <ref:2609.10193#pg1>

The paper's summary: Ines: So what we're looking at today is this paper by Ines and Marcus about protein interaction databases, and they’re basically saying these datasets aren't perfect mirrors of biology.

Marcus: They’re focusing on how the way these databases are built introduces biases that can trick machine learning models into learning shortcuts instead of the actual biological signal.

Ines: Exactly. The authors systematically look at HIPPIE, IntAct, and STRING, and even some data from the Protein Data Bank that comes from three dee structures. They show how just splitting the data randomly creates these strong topological shortcuts for the AI to latch onto.

Marcus: That’s a key part of it. They found that even if you remove the overlap between your training proteins and your testing proteins, those shortcuts still exist because of things like self-interactions or how closely related two species are taxonomically.

Ines: And they found something interesting—when you sample negatives from a set of high-confidence non-interactors, that actually amplifies the shortcut coming from functional relatedness.

Marcus: So what this means for us is that if we only focus on functional relationships, we might be picking up artifacts about broad pathways and complexes instead of the specific protein pair interactions we’re really looking for.

Ines: The authors then build a whole pipeline, an open Nextflow pipeline, that uses integer linear programs to tackle this problem head-on.

Marcus: It sets up similarity-aware splitting that minimizes data loss while giving us control over how big our training, validation, and test sets are.

Ines: And they have this bias-minimizing negative sampling method using integer linear programs too, which tries to jointly reduce the distributional differences between the negatives and your positives.

Marcus: It’s an optimization approach to making sure your training data stays as unbiased as possible by matching specific statistical properties like self-interaction counts or GO biological process indices.

Ines: The results show that even a basic Random Forest model can get near perfect performance in the worst-case scenario, depending on which database you're using.

Marcus: They also point out that topology is the attribute that seems to be most detectable across all these different datasets, and it has the highest utility for improving performance.

Ines: They highlight that in structural datasets like PDB Dimers, eighty percent of the positive interactions are actually self-interactions, which shows how much those structural data sources are inherently biased toward homo dimers.

Marcus: It’s a big takeaway for anyone looking to use structural information; you can't just trust the interaction counts without accounting for those self-interactions first.

Ines: The authors admit that their most rigorous mitigation strategy still leaves some residual bias in PDB Dimers and PINDER because of those high self-interaction proportions.

Marcus: It’s realistic to say they aren't claiming a perfect fix, but they are showing a concrete way to get the least biased datasets possible using this combination of splitting and sampling.

Ines: So, the main point is that reported performance numbers are only trustworthy if you look at exactly what splitting and sampling strategy produced those results.

Marcus: And it also means the choice of your starting protein interaction database matters just as much as how you train the model, which is a bit frustrating because you can't always switch databases easily.

Ines: It shifts the focus from just getting high accuracy numbers to understanding *why* those numbers are high or low depending on the underlying data source.

Marcus: This paper points toward using frameworks like G-AUDIT to quantify biases before you try to fix them, which is a smart way for us as data scientists to start auditing our own datasets.

Ines: The goal is providing a structured way to identify these hidden biases so we can trust the resulting machine learning predictions more.

Marcus: This work has implications for any prediction task where you have a massive pool of negative candidates, like predicting domain-domain or protein-ligand interactions.

The paper's improvements: Tom: So we're looking at how Ines and Marcus suggest ways to actually improve this research on protein interaction datasets, moving beyond just identifying the problems.

Marcus: Basically, they propose a whole new approach using optimization techniques to actively fight those biases instead of just noticing them after the fact.

Ines: They introduce a G-AUDIT framework to systematically measure if any attribute can be used as a shortcut by looking at how detectable it is from the input data and its relationship with the output label.

Marcus: So, this means we’re not just guessing what’s wrong; we’re quantifying specific biases like self-interactions or taxonomy based on their detectability.

Ines: Then they formulate similarity-aware dataset splitting as an integer linear program to minimize data loss while giving us precise control over the sizes of our training, validation, and test sets.

Marcus: That’s a big step up from just doing a random split or some simple graph partitioning because we get tailored configurations.

Ines: And they suggest this bias-minimizing negative sampling method using integer linear programs to jointly reduce the distributional differences between your negatives and your positives.

Marcus: This is about matching statistical properties, like the self-interaction count or the mean GO biological process index, between what we sample as negatives and what our positives actually look like.

Ines: The paper shows that when you apply these mitigation strategies—the splitting and sampling—a Random Forest baseline can hit near perfect performance in the most biased settings for certain datasets.

Marcus: But they also found that topology is the attribute that is most detectable across all datasets, and it seems to have the highest utility for getting better results.

Ines: They showed how different data sources affect shortcut magnitude, pointing out that structural data from the Protein Data Bank stores a fundamentally different population of interactions.

Marcus: That helps researchers decide whether they should even augment their protein interaction datasets with structural information based on the specific bias of that source.

Ines: The authors admit their most rigorous method still leaves residual bias in PDB Dimers because of those high self-interaction proportions, which is a fair caveat.

Marcus: So the implication is that we need to be very careful when we use these structural data sources; they come with their own built-in biases that optimization can only partially fix.

Ines: The authors also emphasize that reported performance figures are only trustworthy if you look at the specific splitting and sampling strategy they used to get those results.

Marcus: It’s a practical warning for us: don't just look at the accuracy number; look at the entire pipeline that generated it.

Ines: This work sets up a tool for the community to apply these bias-aware procedures to their own data, which is really helpful for moving forward.

Marcus: It suggests that if we want to connect these findings to broader evolutionary patterns, we need more rigor about our sample support because functional relatedness can still be a major confounder.

Ines: The next step they suggest is using this concept of quantifying existing shortcuts with G-AUDIT and then minimizing them through optimization for other types of problems than just PPI prediction.

Conclusion: Tom: So we're wrapping up this session by summarizing what we just covered about "Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets."

Marcus: Basically, the paper showed us that protein databases have hidden biases, and they developed an optimization pipeline to actively fight those biases.

Ines: It’s important to remember that reported performance figures for these models are only trustworthy if you look at the exact splitting and sampling strategy they used to get those results.

Marcus: And it also means the choice of your starting protein interaction database matters just as much as how you train the model, which is a bit frustrating because you can't always switch databases easily.

Yuki: I think this really shifts the focus from just chasing accuracy numbers to understanding *why* those numbers are high or low depending on the underlying data source.

Ines: It’s about using tools like G-AUDIT to quantify biases before you even try to fix them, which is a smart approach for us as computational biologists.

Marcus: This work has implications for any prediction task where you have a massive pool of negative candidates, like predicting domain-domain or protein-ligand interactions.

Yuki: It also suggests that if we want to connect these PPI findings to broader evolutionary patterns, we need more rigor about our sample support because functional relatedness can still be a major confounder.

Ines: So the big picture here is that we need a structured way to audit our data for hidden biases so we can trust the resulting machine learning predictions more.

Marcus: This paper gives us a tool the community can use to apply these bias-aware procedures to their own data, which is a big deal for making more reliable predictions across the board.

Yuki: It’s not just theoretical stuff; it’s practical advice for anyone working with large biological cohorts who are worried about batch effects or selection bias in their samples.

Ines: We'll stick with this paper as a guide for how we need to look at the data we feed into those sequence and structure analysis models.

Marcus: It’s a lot of work, but it’s necessary if we want our AI predictions to actually reflect biological reality rather than just learning statistical tricks.

Yuki: We'll keep an eye on this approach as they apply it to other pair-input prediction tasks like gene-gene interactions or protein-ligand interactions down the line.

More episodes

← Home