Simple models outperform single-cell foundation models on downstream benchmarks
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.
Ines: I'm Ines.
Marcus: And I'm Marcus, here with Yuki.
Ines: We're looking at the paper: "Simple models outperform single-cell foundation models on downstream benchmarks".
Marcus: Here's the gist of the paper. Parameter-free representations outperform single-cell foundation models on downstream benchmarks The simple, interpretable pipelines obtain SOTA or near SOTA performance across multiple benchmarks commonly used to evaluate single-cell foundation models, including outperforming foundation models on out-of-distribution tasks involving novel cell types and organisms absent from the training data These results suggest that much of the biologically relevant structure present in current scRNA-seq benchmarks is already accessible through low-complexity linear representations Our work highlights the need for more stringent evaluation protocols and provides strong evidence that simple representations in terms of normalized gene expression counts capture many of the key aspects of cell identity contained in scRNA-seq cell atlases Pruning redundant reasoning tokens The paper investigates whether similar performance can be achieved without utilizing computationally intensive deep learning-based representations The authors systematically compared the performance of simple, interpretable pipelines to the reported performance of large scale single-cell foundation models on common downstream tasks They find that by carefully choosing pre-processing steps and normalization procedures, it is possible to achieve SOTA or near-SOTA performance using simple representations where cells are viewed as vectors in gene expression space This motivates a unifying interpretation: much of the biologically relevant structure present in current scRNA-seq benchmarks is already accessible through low-complexity linear representations How the training signal is built The study employs simple, interpretable pipelines that rely on careful normalization and linear methods to obtain SOTA or near SOTA performance across multiple benchmarks For cross-species cell-type annotation, the authors used a “transfer learning framework in which labels from one species (source species) were transferred and evaluated on embeddings from a different species (target species)” to analyze a spermatogenesis dataset The scTOP method quantifies cellular identity by projecting a cell’s gene expression profile onto a defined cell type space using normalized gene expression values transformed via log2(x + 1) and subsequent ranking and z-scoring For human cell annotation, the pipeline is extended by introducing two additional denoising steps: ANOVA-based gene selection and Principle Component Analysis (PCA) The final representation of the cell is obtained by projecting its standardized vector onto the first d principal directions derived from PCA Where the gains come from The performance of these pipelines often exceeds that of foundation models, despite the fact that they require orders-of-magnitude less computational resources and have almost no free parameters For cross-species annotation, scTOP achieves consistently higher macro F1-scores than foundation models across all eight species tested In the Tabula Sapiens 2.0 cell type classification task, the performance obtained with this pipeline closely matches (and in some cases exceeds) that reported for large foundation models The key insight behind local classifiers is that disease states are not globally separable, but locally distinguishable This strategy demonstrates the power of this approach, achieving a macro F1-score of 0.862 on the SARS-CoV-2 dataset
48: Manifold analyses for more datasets In manifold analyses for other spermatogenesis datasets, the results are similar to the human dataset, indicating that the near-linearity manifold feature remains true across datasets For four Tabula sapiens datasets chosen to represent different scores on our pipeline, we see that after using scTOP the four tissues displayed have a correlation of around 0.
Title and authors: Ines: So, what are they actually doing in terms of methodology here? They use a few specific pipelines. For cross-species cell type annotation, they employ something called scTOP which quantifies cellular identity by projecting the gene expression profile onto a defined cell type space using normalized gene expression values transformed via log base two of x plus one and then ranking and z-scoring.
Marcus: That scTOP method is quite specific. And for human cell annotation, they extend that pipeline by adding two extra denoising steps: an ANOVA based gene selection and then principal component analysis, or PCA.
Yuki: The structure of the analysis seems to be moving from just a projection into a more refined process, using PCA to reduce dimensions before getting that final representation for classification. It’s like filtering out the noise step by step before making the final call on cell identity.
Ines: They also looked at how this works in cross-species settings by using a transfer learning framework where labels from one species, the source species, were transferred and then evaluated on embeddings from a different species, the target species. They applied this to a spermatogenesis dataset.
Marcus: And they used this whole setup to analyze cell identity for human cells too. It’s about testing if these simple linear methods can handle the complexity of distinguishing between different cell states across those different biological contexts.
Yuki: The goal seems to be proving that these low-complexity models don't just work on one specific dataset, but they maintain performance when you test them on multiple benchmarks and even out-of-distribution scenarios involving entirely new organisms.
The paper's summary: Ines: One of the main improvements they point to is that these simple pipelines require orders of magnitude less computational resources and have almost no free parameters compared to foundation models. That’s a huge deal for practical application, especially when you’re working with limited compute.
Marcus: And they demonstrate that for cross-species annotation specifically, scTOP achieves consistently higher macro F1 scores than the foundation models they were comparing it to across all eight species tested. That's a very concrete statistical win.
Yuki: That finding supports the idea that this approach has a lot of real predictive power, not just theoretical interest. It shows that for mapping cell types between different organisms, this linear approach is statistically superior to the large models they built.
Ines: They also found that on a specific task called Tabula Sapiens two point zero cell type classification, the performance obtained with their pipeline closely matches and in some cases exceeds what was reported for those large foundation models on that particular task <ref:2602.16696#pg1>.
Marcus: The paper highlights another key insight about local classifiers: disease states aren't globally separable; they are locally distinguishable, which is a powerful concept when you’re looking at how different conditions manifest in specific regions of the data.
Yuki: That local distinguishability idea connects back to the broader picture, suggesting that biological variation might be structured in a way that simple, localized mathematical operations can capture effectively.
The paper's improvements: Ines: So, wrapping up the main points of "Simple models outperform single-cell foundation models on downstream benchmarks," it boils down to this: you don't need these huge deep learning representations to get good results on many standard tasks in single-cell biology.
Marcus: The paper strongly suggests that carefully chosen normalization and linear methods can yield SOTA or near SOTA performance, and they show this holds even when testing on novel cell types outside the original training data.
Yuki: For me, what’s interesting is the implication for how we think about biological structure itself; it seems much of what we need to understand cellular identity is already present in these low-complexity linear representations.
Ines: I agree. The study suggests that simple representations in terms of normalized gene expression counts capture many of the key aspects of cell identity found in scRNA-seq atlases, which is a strong argument for using these methods more broadly across the field.
Marcus: And they highlight a real need for more stringent evaluation protocols because it’s clear that we haven't fully tested how robust these simple representations are against truly novel biological samples.
Yuki: It opens up new avenues for how we can analyze complex biological data, moving away from relying solely on massive models toward methods that are both interpretable and computationally efficient.
Conclusion: Ines: So to wrap up, this paper on "Simple models outperform single-cell foundation models on downstream benchmarks" shows that you don't need these huge deep learning representations for many common tasks in single-cell biology.
Marcus: Exactly what they show is that simple pipelines, using careful normalization and linear methods, can get SOTA or near SOTA performance across multiple benchmarks.
Yuki: It really suggests that much of the biological structure we're looking at in these datasets is already accessible through these low-complexity linear representations.
Ines: The key finding is that even when testing on novel cell types or organisms completely absent from the training data, these simple models still perform well. That’s a big deal for generalization.
Marcus: For us working with large cohorts, this means we might not always need to throw massive compute at every problem; we can try these simpler methods first and see if they get us there.
Yuki: From a population genetics view, it supports the idea that the underlying patterns governing cell identity aren't overly complex in a way that requires deep learning to uncover.
Ines: They actually highlight the need for better evaluation protocols because we need to be more rigorous about how we measure if these simple representations are truly capturing biological reality.
Marcus: And they provide strong evidence that these simple representations, based on normalized gene expression counts, capture many of the key aspects of cell identity found in those big cell atlases.
Yuki: It’s interesting how this connects to the broader history of species; it implies a fundamental organization that persists across different biological contexts.
Ines: So, "Simple models outperform single-cell foundation models on downstream benchmarks" is pointing toward a more efficient and accessible way to approach cell type classification.
Marcus: Yeah, it gives us a clear direction for how we might design better pipelines for our next batch of genomic data analysis.
Yuki: And it makes me wonder what other types of biological questions these simple linear methods can help us answer when we look at different organisms.
Huan Souza, *, Pankaj Mehta, ^2, †
Department of Physics, Boston University · Faculty of Computing and Data Science, Boston University
q-bio.GN, cs.LG, q-bio.QM
Submitted: 2026-02-18
Updated: 2026-11-06
Code: https://github.com/czi-ai/transcriptformer
Project page: https://chanzuckerberg.github.io/cz-benchmarks
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 78/100
The gist: Parameter-free representations outperform single-cell foundation models on downstream benchmarks The simple…
Key concepts
- Simple, Interpretable Pipelines
- These are methods that use basic linear math and careful pre-processing (like normalization) instead of complex deep learning models. They treat cell gene expression as vectors in a mathematical space, making the results easier to understand and implement without needing massive amounts of computational power or many parameters.
- scTOP Method
- This is a specific linear method used for cross-species cell type annotation. It works by projecting a cell's gene expression profile onto a defined space using normalized values. This allows the system to classify cells into known types by finding the best fit within this constructed reference basis.
- Manifold Analysis
- This involves studying the underlying geometric structure of single-cell data. By comparing standard distances (Euclidean) with distances calculated along curves (geodesic), researchers can determine if the complex cell data actually lies on a simple, low-dimensional linear subspace, confirming that biological variation is often simpler than it appears.
- Pointwise Mutual Information (PMI)
- PMI is a statistical measure used to quantify the relationship between two variables—in this case, transcription factor activity and gene expression probability. It helps uncover coherent patterns of how regulatory elements control gene expression without needing complex, high-order modeling techniques.
Terminology
Summary
Parameter-free representations outperform single-cell foundation models on downstream benchmarks The simple, interpretable pipelines obtain SOTA or near SOTA performance across multiple benchmarks commonly used to evaluate single-cell foundation models, including outperforming foundation models on out-of-distribution tasks involving novel cell types and organisms absent from the training data These results suggest that much of the biologically relevant structure present in current scRNA-seq benchmarks is already accessible through low-complexity linear representations Our work highlights the need for more stringent evaluation protocols and provides strong evidence that simple representations in terms of normalized gene expression counts capture many of the key aspects of cell identity contained in scRNA-seq cell atlases
Pruning redundant reasoning tokens
The paper investigates whether similar performance can be achieved without utilizing computationally intensive deep learning-based representations The authors systematically compared the performance of simple, interpretable pipelines to the reported performance of large scale single-cell foundation models on common downstream tasks They find that by carefully choosing pre-processing steps and normalization procedures, it is possible to achieve SOTA or near-SOTA performance using simple representations where cells are viewed as vectors in gene expression space This motivates a unifying interpretation: much of the biologically relevant structure present in current scRNA-seq benchmarks is already accessible through low-complexity linear representations
How the training signal is built
The study employs simple, interpretable pipelines that rely on careful normalization and linear methods to obtain SOTA or near SOTA performance across multiple benchmarks For cross-species cell-type annotation, the authors used a “transfer learning framework in which labels from one species (source species) were transferred and evaluated on embeddings from a different species (target species)” to analyze a spermatogenesis dataset The scTOP method quantifies cellular identity by projecting a cell’s gene expression profile onto a defined cell type space using normalized gene expression values transformed via log2(x + 1) and subsequent ranking and z-scoring For human cell annotation, the pipeline is extended by introducing two additional denoising steps: ANOVA-based gene selection and Principle Component Analysis (PCA) The final representation of the cell is obtained by projecting its standardized vector onto the first d principal directions derived from PCA
Where the gains come from
The performance of these pipelines often exceeds that of foundation models, despite the fact that they require orders-of-magnitude less computational resources and have almost no free parameters For cross-species annotation, scTOP achieves consistently higher macro F1-scores than foundation models across all eight species tested In the Tabula Sapiens 2.0 cell type classification task, the performance obtained with this pipeline closely matches (and in some cases exceeds) that reported for large foundation models The key insight behind local classifiers is that disease states are not globally separable, but locally distinguishable This strategy demonstrates the power of this approach, achieving a macro F1-score of 0. p. 9 4D, performance obtained with our pipeline closely matches (and in some cases exceeds) that
reported for large foundation models, including TranscriptFormer. The mean macro F1-score across
tissues is 0.Where the paper says it
Manifold analyses for more datasets
In manifold analyses for other spermatogenesis datasets, the results are similar to the human dataset, indicating that the near-linearity manifold feature remains true across datasets For four Tabula sapiens datasets chosen to represent different scores on our pipeline, we see that after using scTOP the four tissues displayed have a correlation of around 0.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Instead of using computationally intensive deep learning-based representations like TranscriptFormer, simple linear pipelines relying on careful normalization and linear methods can achieve SOTA or near SOTA performance across multiple benchmarks. This allows for obtaining
SOTA or near SOTA performance across multiple benchmarks commonly used to evaluate single-cell foundation models.
-
The system can perform cross-species cell type annotation by using a simple parameter free linear-algebra based method called scTOP to construct a reference basis and classify new cells by projecting their gene expression profiles onto this basis. This pipeline achieves
consistently higher macro F1-scores than foundation models
on cross-species transfer tasks. -
For cell type classification tasks, the system can employ a pipeline combining scTOP preprocessing with ANOVA based gene selection, PCA, and logistic regression for human cell annotation. This approach shows that
macro F1-scores exceed 0.8
across many tissues in the Tabula Sapiens 2.0 dataset without requiring deep learning for classification. -
The system can identify cells infected by SARS-CoV-2 by augmenting the pipeline with an unsupervised clustering step using the Leiden algorithm on PCA transformed data, which discovers
natural subdivisions in the data based purely on transcriptional similarity.
This enables local classifiers to detectcontext-dependent signatures of infection
within biologically coherent regions. -
The system can quantify gene target relationships by calculating a pointwise mutual information PMI score between transcription factor activity and gene expression probability, which recovers
coherent and biologically meaningful TF–gene association patterns
without higher-order modeling. This allows for identifying regulatory structure encoded inthe observable conditional co-expression patterns of normalized scRNA-seq data.
-
The system can assess the geometric structure of the transcriptional data manifold by comparing Euclidean distances with geodesic distances computed via Isomap, which shows that
for the spermatogenesis dataset this correlation is greater than 0.9,
suggesting thatthe submanifold corresponding to real cells is well-approximated by a linear subspace.
Abstract
Single-cell RNA sequencing (scRNA-seq) data exhibit strong and reproducible statistical structure. This has motivated the development of large-scale foundation models, such as TranscriptFormer, that use transformer-based architectures to learn a generative model for gene expression by embedding genes into a latent vector space. These embeddings have been used to obtain state-of-the-art (SOTA) performance on downstream tasks such as cell-type classification, disease-state prediction, and cross-species learning. Here, we ask whether similar performance can be achieved without utilizing computationally intensive deep learning-based representations. Using simple, interpretable pipelines that rely on careful normalization and linear methods, we obtain SOTA or near SOTA performance across multiple benchmarks commonly used to evaluate single-cell foundation models, including outperforming foundation models on out-of-distribution tasks involving novel cell types and organisms absent from the training data. Our findings highlight the need for rigorous benchmarking and suggest that the biology of cell identity can be captured by simple linear representations of single cell gene expression data.
Sources
- Benchmarking and Evaluation of AI Models in Biology: Outcomes and Recommendations from the CZI Virtual Cells Workshop
- A twenty-first century statistical physics of life
Related papers
- SeekRBP: Leveraging Sequence-Structure Integration with Reinforcement Learning for Receptor-Binding Protein Identification
- Genome-Factory: A Library for Tuning, Deploying, and Interpreting Genomic Foundation Models
- AVA: Agentic Variant Adjudication for Automated, Auditable Read-Level Review in Cancer Diagnostic Sequencing
- C3P: Contrastive promoter-protein pretraining yields representations capturing bacterial gene regulation
- Leveraging genomic deep learning models for the prediction of non-coding variant effects
- Adaptive Allele-Haplotype Incidence Indexing for Pangenome Queries: Representation, Reconstruction, and Benchmarking