Benchmarking Recent Single-Cell Synthetic Data Generation Methods on the OneK1K Dataset

Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.

Ines: We're looking at the paper: "Benchmarking Recent Single-Cell Synthetic Data Generation Methods on the OneK1K Dataset".

Marcus: Here's the gist of the paper. This work benchmarks synthetic single-cell RNA-seq data generation methods—scGFT, scLDM, and Cell2Sentence—on the donor-split OneK1K dataset to study the trade-off between biological utility and privacy risk.

Ines: The study evaluated three distinct generative architectures: Single-cell Graph Fourier Transform (scGFT), Single-cell Latent Diffusion Models (scLDM), and Cell2Sentence on OneK1K dataset

9: .

Marcus: The OneK1K dataset consists of single-cell RNA sequencing data from approximately 1.2 million peripheral blood mononuclear cells (PBMCs) collected from 982 individual donors

9: .

Ines: For the benchmarking analysis, the researchers restricted the modeling phase to highly variable genes (HVGs), which capture the main axes of biological variation

2: .

Paper summary: Ines: So, we're looking at this paper right now called "Benchmarking Recent Single-Cell Synthetic Data Generation Methods on the OneK1K Dataset". Marcus It’s basically setting up a test to see which methods for creating synthetic single-cell RNA sequencing data are best when you have donor information involved. Ines The authors are comparing three main approaches: scGFT, scLDM, and Cell2Sentence. Yuki And they're doing this using the OneK1K dataset, which is a donor-split set of single-cell data from about one point two million peripheral blood mononuclear cells collected from nine hundred eighty-two different donors.

Marcus: Right, so the core thesis here is that generating synthetic data for privacy-aware sharing needs to balance two things: how biologically useful the synthetic data is, and how much privacy risk it introduces to the original donors. Ines The paper claims they benchmark these three methods across statistical fidelity, how well they preserve biological structure downstream, and membership inference vulnerability. Yuki That's interesting because it connects the technical generation stuff to what’s happening in the actual human population genetics.

Marcus: They focus a lot on highly variable genes or HVGs when they generate synthetic data because making a full transcriptome synthetic is just too much computational work, so they evaluate how you handle those non-HVG genes instead. Ines That's where it gets crucial; the paper finds that how you handle those non-HVG values is a central determinant of privacy risk. Yuki So, if you just copy the non-HVG values directly from the training cells, that leads to severe leakage in membership inference attacks.

Ines: That’s exactly what they found; copying them directly is not viable for privacy Yuki But they show that using a Poisson-based reconstruction for those non-HVG genes substantially reduces the risk of membership inference. Marcus And they also looked at scGFT and scLDM, and found that both of those models actually do a better job preserving the underlying biological structure than just using an independent Poisson baseline. Ines That suggests that these generative modeling approaches can be effective while still keeping privacy profiles near random under the right settings.

Yuki: So, what does this mean for us when we think about using these synthetic datasets to study population genetics? Marcus It means that when we're creating synthetic data for sharing, we can't just pick any method; you have to evaluate it on multiple axes—fidelity and privacy risk simultaneously. Ines The paper points out that the results aren't always correlated; a model might look good on one metric but drift badly on another. Yuki That’s important because it shows we need to be careful when we rely too much on just one score, whether it's biological fidelity or privacy protection.

Marcus: And they have a pretty clear takeaway about the trade-off; for instance, with scGFT, increasing the number of Fourier components to twenty-five reduces leakage down to a near-random privacy profile compared to the baseline Ines That gives us some control over that trade-off—you can tune complexity and manage privacy. Yuki But they also mentioned a limitation there; since they focused on HVGs, you’re not actually generating the full transcriptome, so you have to reconstruct those non-HVG genes separately.

Ines: Right, that reconstruction step is a key point; it shows that decisions made outside of the main modeling space can really affect both how well the full transcriptomic data is preserved and how much privacy you have. Marcus Overall, this paper on "Benchmarking Recent Single-Cell Synthetic Data Generation Methods on the OneK1K Dataset" sets up a framework for evaluating these synthetic generation paradigms not just by their utility scores but by their actual donor-level privacy risk. Yuki It makes it clear that we need to look at robustness and interpretability across these different metrics when choosing a method to handle sensitive single-cell data.

Ines: So, in conclusion, this work shows that both scGFT and scLDM substantially exceed the Poisson baseline in capturing joint structure that independent sampling cannot achieve Marcus The authors motivate a broader evaluation perspective for research that prioritizes robustness across multiple metrics. Yuki It emphasizes that you need to examine not only which generator performs best on a single score but how specific modeling choices shift the balance between biological utility and donor-level privacy throughout the whole process.

Conclusion: Ines: So, what did we actually cover? Basically, they're taking these three methods—scGFT, scLDM, and Cell2Sentence—and testing them on the OneK1K dataset to see how they balance creating useful synthetic data with keeping the donor information private.

Marcus: Yeah, the main authors are Buse Giledereli and Computer Engineering Boğaziçi University. They're looking at how these different ways of making synthetic single-cell RNA sequencing data stack up against each other when you’re worried about privacy risks for the original donors.

Yuki: And what it means for us in population genetics is that we can't just trust one way to generate this synthetic data; we need to check both how accurate the biology is and how much risk we introduce to the individuals in the cohort.

Ines: Exactly. The paper shows that scGFT and scLDM are actually better at keeping the real biological structure when you look downstream compared to a simple Poisson baseline.

Marcus: But it also shows that handling those non-HVG genes is a huge deal; copying them directly from the training data causes serious leakage, but using a Poisson replacement significantly cuts down on membership inference risk.

Yuki: That means for us studying human populations, the choice of how you handle those genes outside the main model space actually matters a lot for privacy.

Ines: It really does. They show that you can control this trade-off by tweaking parameters, like how many Fourier components you use in scGFT to reduce leakage down to a near-random profile.

Marcus: So, the authors are pointing out that we need to look at these synthetic datasets through multiple lenses—utility, behavior in feature space, and privacy risk—not just one score.

Yuki: That’s what I want to emphasize. It makes sense that we have to consider how these modeling choices shift the balance between getting a good biological representation and protecting donor privacy across the whole process.

Ines: Right, so this benchmarking framework is useful because it lets us see those trade-offs in real terms for both the different models and our data handling decisions.

Marcus: And it sets up a clear direction for future work, focusing on making these evaluations more robust by testing both HVG-focused and full-gene behavior.