One Method Does Not Fit All: A Large-Scale Benchmark of Imputation Methods for Single-Cell RNA Sequencing Data

arXiv:2603.24626 · q-bio.GN, cs.LG, stat.ML · Submitted 2026-03-25 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.

Ines: We're looking at the paper: "One Method Does Not Fit All".

Marcus: Here's the gist of the paper. We present a comprehensive benchmark of 15 scRNA-seq imputation methods spanning 7 methodological categories, including traditional and DL-based methods.

Ines: Methods are evaluated across 30 datasets from 10 experimental protocols on 6 downstream analyses.

Marcus: Results show that traditional methods, such as model-based, smoothing-based, and low-rank matrix-based methods, generally outperform DL-based methods, including diffusion-based, generative adversarial network-based, graph neural network-based, and autoencoder-based methods.

Ines: In addition, strong performance in numerical gene expression recovery does not necessarily translate into improved biological interpretability in downstream analyses.

Title and authors: Marcus: Let's talk about the title and who wrote this, which is "One Method Does Not Fit All: A Large-Scale Benchmark of Imputation Methods for Single-Cell RNA Sequencing Data." The title itself tells you the main point upfront.

Ines: It’s a direct statement that there isn't a universal solution for imputing zeros in single-cell data. It sets the stage for why we need this kind of comparison study.

Yuki: From my perspective, it reinforces that biological questions are nuanced; you can't just run the same imputation algorithm and expect the same answer across all species or tissues.

Marcus: The authors are Yuichiro Iwashita, Ahtisham Fazeel Abbasi, Koichi Kise, Andreas Dengel, and Muhammad Nabeel Asim. They’re coming from a mix of computer science departments and research centers that work on artificial intelligence and genomics.

Ines: That mix is interesting because it means they’re looking at the problem from both the computational modeling side and the actual biological data side.

Yuki: I wonder what this means for future population studies; can we trust imputed data when we try to track subtle genetic variation across a large cohort?

Marcus: It suggests that before you run a big imputation pipeline, you need to know which specific downstream task—clustering, trajectory analysis, or maybe just numerical recovery—is your absolute priority.

The paper's summary: Ines: So what does the paper actually summarize? It’s basically showing fifteen imputation methods in six categories against thirty different datasets and protocols to see how they perform across six tasks.

Marcus: They break it down into traditional methods like model based, smoothing based, and low rank matrix based, as well as deep learning approaches like diffusion based, GAN based, graph neural network based, and autoencoder based.

Yuki: And the key finding they push is that traditional methods generally perform better than the deep learning ones in terms of pure numerical recovery of gene expression.

Ines: But here’s the catch they throw in: high performance on numerical recovery doesn't automatically mean better biological interpretability downstream, like when you look at marker gene analysis or cell type annotation.

Marcus: They stress that performance varies a lot depending on the dataset, the protocol used to generate it, and which specific analysis you are running. No single method is consistently best across all scenarios.

Yuki: This is important because if we take a result from one study and apply it to another population with different sequencing depth or capture noise, the results might be totally unreliable.

The paper's improvements: Ines: The paper suggests a few key improvements for how we think about imputation selection. First, you have to select strategies tailored precisely to your analytical objectives rather than just picking the method with the highest overall score.

Marcus: That’s right; they recommend traditional methods like scImpute and MAGIC when you need high fidelity in tasks that require preserving biological meaning, like tracking trajectories or annotating cell types.

Yuki: And for tasks where numerical accuracy is paramount, maybe when you're just trying to get a good estimate of the raw expression levels, WEDGE might be more appropriate.

Ines: They also point out that we need a better way to check for instability; they found that methods using only read counts without unique molecular identifiers, like in SMART-seq datasets, are much less stable.

Marcus: And they noted a specific weakness with some deep learning models; scMultiGAN showed poor correlation with bulk RNA sequence data at both the pseudo bulk and cell line levels, even though it recovered the ground truth data numerically.

Yuki: So the improvement isn't just about finding a better algorithm; it’s about building a framework that tells you what kind of data—read counts versus UMIs—will break which method.

Conclusion: Ines: Wrapping up this whole benchmark, the paper concludes that choosing an imputation method needs to be carefully tailored to the specific downstream task and dataset characteristics. That task-specific evaluation is the most important lesson here.

Marcus: So, for practical application, they recommend scImpute and MAGIC for things like marker gene analysis or trajectory analysis because they seem better at keeping the biological signal intact.

Yuki: For those of us working with large cohorts, this means we have to be very careful about which imputation method we use if we want to make claims about cell types or development pathways across different samples.

Ines: And for me, it’s a reminder that numerical recovery alone isn't enough; you have to check the biological signal preservation alongside the raw numbers.

Marcus: I think this benchmark gives us a clear map of trade-offs between numerical accuracy and biological fidelity across all these different methods. It shows that performance is not uniform.

Yuki: It’s a lot to take in, but it grounds our work in reality; we can't just assume one technique will solve every single data problem without checking the specific conditions first.

Ines: Absolutely. We’ve seen how different approaches handle different noise profiles, and that context is everything when you’re trying to interpret what your scRNA-seq actually tells you about the biology.

RPTU University Kaiserslautern-Landau · German Research Center for Artificial Intelligence (DFKI GmbH) · Graduate School of Informatics, Osaka Metropolitan University · intelligentX GmbH

q-bio.GN, cs.LG, stat.ML

Submitted: 2026-03-25

Updated: 2026-11-05

Code: https://github.com/Winnie09/imputationBenchmark

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 75/100

The gist: We present a comprehensive benchmark of 15 scRNA-seq imputation methods spanning 7 methodological categories, including traditional and DL-based methods.

Key concepts

scRNA-seq Imputation Methods
These are mathematical techniques used to fill in missing gene expression values in single-cell RNA sequencing data. Because scRNA-seq data is inherently sparse (many zero counts), imputation methods estimate the missing values based on existing information, aiming to create a more complete and accurate representation of the true cell state.
Downstream Analyses
These are the biological questions researchers ask after imputing the data. The study evaluates performance in six tasks: numerical gene expression recovery, cell clustering, differential expression (DE) analysis, marker gene identification, trajectory analysis, and cell type annotation. Success depends on how well the imputation preserves these biological signals.
Traditional vs. DL-based Methods
Traditional methods rely on established statistical models like smoothing or matrix factorization. Deep Learning (DL)-based methods use neural networks (like GANs or GNNs) to learn complex, non-linear relationships in the data. The benchmark compares these two paradigms to see which is better for different imputation goals.
Technical Noise and Dropout
Technical noise refers to errors introduced during sequencing, such as read counts without unique molecular identifiers (UMIs). Dropout refers to the natural biological phenomenon where some genes are expressed at zero in a cell. The study investigates how methods handle these technical issues when evaluating performance across different sequencing protocols.

Terminology

Summary

We present a comprehensive benchmark of 15 scRNA-seq imputation methods spanning 7 methodological categories, including traditional and DL-based methods. p. 0 Methods are evaluated across 30 datasets from 10 experimental protocols on 6 downstream analyses. p. 0 Results show that traditional methods, such as model-based, smoothing-based, and low-rank matrix-based methods, generally outperform DL-based methods, including diffusion-based, generative adversarial network-based, graph neural network-based, and autoencoder-based methods. p. 1 In addition, strong performance in numerical gene expression recovery does not necessarily translate into improved biological interpretability in downstream analyses. p. 1 Furthermore, method performance varies substantially across datasets, protocols, and downstream analyses. p. 1

The evaluation covers 6 key tasks: numerical gene expression recovery, cell clustering, DE analysis, marker gene analysis, trajectory analysis, and cell type annotation. p. 3 For instance, in numerical gene expression recovery (Fig. p. 3 1), scTsI, PBLR, and WEDGE achieve the best overall performance with medians of LND values consistently close to zero. p. 3 In cell clustering (Fig. p. 7 3), MAGIC and WEDGE show the best coherency performance across UMAP visualizations. p. 9 In DE analysis (Fig. p. 9 7), AcImpute achieves the best overall performance, with the highest DE enrichment analysis performance across 3 datasets and the best effect size analysis performance, while scMultiGAN shows the worst DE enrichment performance on sc 10x 5cl. p. 7 In marker gene analysis (Fig. p. 1 8), scImpute and MAGIC consistently show the best performance. p. 6 In trajectory analysis (Fig. p. 1 10), PbImpute, scImpute, scLRTC, CPARI, and Bubble consistently preserve the temporal ordering of cells across 2 datasets. p. 9 In cell type annotation (Fig. p. 1 11), MAGIC achieves the best overall performance across the 6 datasets. p. 7

Table 4 summarizes the performance of the 15 imputation methods across 6 downstream tasks, showing that traditional methods, such as scImpute, MAGIC, and WEDGE, show the best or moderately high overall performance across the 6 tasks. p. 9 Conversely, DL-based methods are more variable. p. 6 The trade-off between numerical gene expression recovery performance and biological signal preservation is a critical consideration in the design and selection of imputation methods. p. 1 The study concludes that the choice of imputation method should be carefully tailored to specific downstream tasks and dataset characteristics to ensure optimal performance. p. 3 For instance, scImpute and MAGIC are recommended for tasks that require the preservation of biologically meaningful information, such as marker gene analysis, trajectory analysis, and cell type annotation, while WEDGE may be more suitable for tasks that prioritize numerical gene expression recovery. p. 3 While traditional methods show relatively better performance across a wide range of tasks, the performance of DL-based methods is more variable. p. 0 The study notes that the absence of UMIs in SMART-seq datasets leads to higher instability for many methods because they use only read-counts without UMIs, which can lead to increased technical noise. p. 7 Furthermore, scMultiGAN shows poor correlation with bulk RNA-seq data at both the pseudo-bulk and cell line levels despite its moderate numerical recovery of ground truth data. p. 7 The study acknowledges that imputation does not universally improve downstream task performance, particularly in datasets with sufficient sequencing depth. p. 1

Table 5 presents 30 unique benchmark datasets spanning 10 scRNA-seq data extraction protocols, 12 cell lines, 11 tissues from human and mouse samples, and 6 disease conditions. p. 1 The curated dataset collection covers a wide range of sizes and sparsity rates to evaluate imputation methods under different number of cells, genes, and dropout conditions. p. 1 Protocols introduce distinct technical noise profiles and dropout patterns that can substantially influence imputation performance. p. 2 This protocol-level heterogeneity is addressed by selecting 26 datasets obtained using 10 different protocols. p. 1 In addition to real datasets, 4 simulated datasets with known ground truth are used to compare the performance of imputation methods under different dropout rates. p. 9 The simulated datasets are generated using the Splatter package, which allows controlled simulation of scRNA-seq data with known ground truth. p. 7 Each simulated dataset contains 2,000 cells and 10,000 genes, and 4 simulated datasets with varying dropout rates of 30. p. 979%, 50.62%, 70.12%, and 89.61% are generated, which reflect a range of dropout conditions commonly observed in real scRNA-seq experiments. p. 2 Following Hou et al. p. 9, corresponding bulk RNA-seq datasets are obtained for 3 real scRNA-seq datasets to evaluate imputation performance at the cell population level. p. 6 After QC and data splitting, dropout events are artificially introduced into each split set of real scRNA-seq datasets since it is not possible to distinguish between true biological zeros and technical dropout events in real scRNA-seq datasets. p. 2 To introduce dropout events, 10% of non-zero expression values in each split dataset are selected and masked as zero expression values, with the original non-zero expression values used as ground truth for evaluation. p. 2

and run with default hyperparameters except for scIDPMs [48] and scMultiGAN [51], whose hyperparameters are adjusted to utilize GPU acceleration. p. 7 Downstream tasks are performed using the Scanpy [58] package in Python, except for DE analysis which is performed using the MAST [55] and limma [59] package in R, trajectory analysis which is performed using the TSCAN [114] pagkage in R, and cell type annotation which is performed using the scGPT [61] package in Python and 1D-CNN implemented with PyTorch [120] package in Python. p. 1 The evaluation metrics are calculated on top of the scikit-learn [121] package in Python. p. 9 All visualizations are created using the ggplot2 [122] package in R. p. 2

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Develop a method that selects imputation strategies tailored to specific analytical objectives, as Our findings provide practical guidance for selecting imputation methods tailored to specific analytical objectives suggests. This improved system can guide users to choose between traditional methods like scImpute and MAGIC for preserving biologically meaningful information (marker gene analysis, trajectory analysis) or WEDGE for numerical gene expression recovery.

  2. Implement a multi-method comparison engine that evaluates performance across 6 downstream tasks using task-specific evaluation measures such as MAE, ARI, IoU, and MCC. This system will allow researchers to assess the trade-off between numerical accuracy and biological fidelity by comparing metrics like LND for numerical recovery against UMAP visualization for cell clustering coherency.

  3. Create a protocol robustness assessment module that identifies when imputation performance varies substantially across datasets and protocols. This system can highlight instability in methods like scIGANs or scGNN on data generated by read-counts (SMART-seq, SMART-seq2, Fluidigm C1) compared to those using unique molecular identifiers (UMIs).

  4. Integrate a feature selection module for marker gene analysis that uses DE analysis results. This system can identify genes whose expression levels significantly differ between clusters and rank them based on LFC or test statistics derived from MAST or Wilcoxon rank-sum tests, ensuring the identified marker genes are biologically relevant to cluster distinctions.

  5. Develop a trajectory inference validation tool that checks for preservation of continuous expression gradients. This tool will compare inferred pseudotime scores (POS) and Kendall’s rank correlation coefficient (KRCC) against true cellular development time labels, flagging methods like scIDPMs and stDiff that collapse subtle expression gradients into discrete states.

  6. Build a cell type annotation quality control system using 1D-CNN or scGPT. This system can assess the macro accuracy, precision, recall, and F1 score of cell type predictions across different datasets to determine which imputation methods provide the strongest evidence for biological context without significantly outperforming the masked baseline in sufficient sequencing depth.

Sources

Related papers