How Private Are DNA Embeddings? Inverting Foundation Model Representations of Genomic Sequences

summary

The gist

DNA foundation models can be susceptible to model inversion attacks when their embeddings are shared via Embeddings-as-a-Service (EaaS) frameworks…

In short

The study tested DNA foundation models for privacy risks when their embeddings are shared via Embeddings-as-a-Service. It found that per-token embeddings allow near-perfect sequence reconstruction, while mean pooling offers only partial protection. Tokenization strategy significantly impacts resilience, with variable BPE tokens in DNABERT-2 being more secure than fixed tokenizers.

Key concepts

Model Inversion Attack
This is an attempt by an adversary to reverse-engineer the original sensitive input data (like a DNA sequence) from the model's output or its learned numerical representations (embeddings). The attack works by training a separate 'inversion model' designed to predict the original input based on the embedding.
Per-token Embeddings
This strategy involves storing and sharing embeddings for every single token in a sequence separately. The study found this method provides virtually no privacy protection because a simple model can reconstruct the entire sequence with near-perfect accuracy, making it highly vulnerable.
Mean-Pooled Sequence Embeddings
Instead of sharing individual tokens, this method averages all the token embeddings in a sequence to create one single vector representing the whole sequence. This offers partial resilience against inversion attacks, but meaningful reconstruction is still possible for shorter sequences and specific models.
Tokenization Strategy
This refers to how a foundation model breaks down an input sequence (like DNA) into smaller pieces (tokens). The paper suggests that variable-length tokenizers, such as BPE in DNABERT-2, introduce alignment ambiguity that makes the sequence much harder to reconstruct than fixed-length tokenization.

Terminology used across episodes

This episode discusses

The paper

How Private Are DNA Embeddings? Inverting Foundation Model Representations of Genomic Sequences · Read on arXiv

University of Tuebingen

Genomic foundation models are increasingly used to share dense representations rather than raw DNA sequences. This is particularly pursued for mean-pooled representations that are later used for downstream prediction tasks. Whether this practice protects the privacy of the underlying sequence is largely unknown. We evaluated model-inversions against zero-shot embeddings from DNABERT-2, Evo 2, and Nucleotide Transformer v2 (NTv2). Substantial reconstruction remains possible: a single multi-length Length-Conditioned Query Decoder reconstructed NTv2 sequences with mean Levenshtein similarity from 0.98 at 10 nucleotides to 0.68 at 100, and Evo 2 sequences from nearly 1.00 at 15 nucleotides to 0.67 at 100. DNABERT-2 showed higher resistance (0.56 to 0.51), although it remained above the random baseline at short lengths. Therefore, we further evaluated, whether a linkage attack similar to Homer's attack is possible based on the mean-pooled representations of human DNA sequences. At SNP-enriched length-25 loci and known coordinates, Evo 2 and NTv2 reached 99% identification at 32 loci and DNABERT-2 at 256 loci. With alignment-based coordinate recovery, the corresponding 99% budgets were 64 and 128 loci for Evo 2 and NTv2, while DNABERT-2 did not reach the reported identification thresholds. Random loci required larger budgets. These findings show that mean-pooled embeddings can leak sequence and identity, with risk depending on sequence length, representation, locus informativeness, and coordinate information.

Transcript

Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.

Ines: We're looking at the paper: "How Private Are DNA Embeddings? Inverting Foundation Model Representations of Genomic Sequences".

Marcus: Here's the gist of the paper. DNA foundation models can be susceptible to model inversion attacks when their embeddings are shared via Embeddings-as-a-Service (EaaS) frameworks, necessitating urgent privacy-aware design prior to widespread deployment The study evaluates the resilience of DNA foundation models to model inversion attacks where adversaries attempt to reconstruct sensitive training data from model outputs Our results show that per-token embeddings allow nearperfect sequence reconstruction across all models Model Inversion Attack Setup Model inversion attacks aim to recover original input sequences from their embedded representations by training an inversion model as a decoder network The study focuses on training-based attacks where the inversion model is optimized to minimize the reconstruction loss L = R(x, I(M(x))) The evaluation involves three DNA foundation models: DNABERT-2, Evo 2, and Nucleotide Transformer v2 (NTv2) Embedding Strategy Comparison The research compares two embedding sharing strategies: per-token embeddings and mean-pooled sequence embeddings Per-token embeddings provide virtually no privacy protection, allowing near-perfect reconstruction using a simple MLP across all three foundation models Mean pooling offers partial but insufficient protection, as meaningful partial reconstruction remains possible for shorter sequences and specific models Tokenization and Resilience The tokenisation strategy of the foundation model is a suspected determinant of privacy, with DNABERT-2’s Byte Pair Encoding (BPE) coinciding with substantially stronger resilience to inversion attacks compared to NTv2’s fixed 6-mer and Evo 2’s single-nucleotide tokenisers This difference is hypothesized to stem from variable-length BPE tokens introducing alignment ambiguity, which hinders inversion Sequence Length Trade-offs The privacy risk of sequence length presents a trade-off where longer sequences are more sensitive but harder to reconstruct from mean-pooled embeddings because averaging discards positional information Shorter sequences carry less genomic content but are substantially easier to invert, implying no single safe sequence length exists Correlation as a Diagnostic Tool The correlation between pairwise embedding distances and sequence similarities serves as a reliable predictor of reconstruction quality across models and sequence lengths A stronger correlation indicates that the embedding space preserves sequence-level structure, facilitating reconstruction This metric can be used as a lightweight diagnostic for assessing privacy risk without a full attack evaluation Generalizability to Real Data The vulnerability of DNA foundation model embeddings extends to individual level genomic data, as reconstruction performance on sequences derived from the 1000 Genomes Project closely mirrors results obtained on hg38 This confirms that the attack scenario generalises to private genomes The study concludes that per-token embeddings offer no meaningful privacy protection, while mean pooling provides partial resilience that varies substantially across models and sequence lengths The findings motivate the development of embedding-level privacy defences for the safe adoption of EaaS paradigms in genomics Future work should explore defenses such as differential privacy or embedding perturbation to mitigate identified vulnerabilities Key Findings Summary 1.

Ines: Per-token embeddings allow near-perfect reconstruction using a simple MLP, demonstrating that the vulnerability is inherent to the embeddings rather than dependent on large attack models 2.

Marcus: Mean pooling provides partial but insufficient protection, with meaningful partial reconstruction possible for shorter sequences and specific models 3.

Ines: BPE tokenisation in DNABERT-2 coincides with considerably greater reconstruction difficulty, suggesting tokenisation design warrants further investigation as a factor in privacy-aware model design 4.

Paper summary: Marcus: Moving into the details of this paper on "How Private Are DNA Embeddings? Inverting Foundation Model Representations of Genomic Sequences," they focus on comparing two specific sharing strategies for those embeddings, <ref:2603.06950#pg2>.

Ines: They are comparing per-token embeddings, where each token from the foundation model's tokeniser gets its own vector, against mean-pooled sequence embeddings, which is just averaging all those position-specific vectors together to get one fixed-size representation.

Yuki: I wonder how that averaging affects the information we keep about the sequence structure itself versus just a general summary.

Marcus: Well, they found that per-token embeddings preserve positional information and the full per-token structure of what the foundation model outputs, as shown in equation three for sequences of length l, where n is the number of tokens produced by the tokeniser <ref:2603.06950#pg3>.

Ines: That means that when you use those per-token embeddings, you have all that positional information locked in, which makes it super vulnerable to reconstruction.

Yuki: So the mean pooling is supposed to be the defense because it discards some of that specific positional detail, making it less reconstructible.

Marcus: They test this across three models: DNABERT-two Evo two and Nucleotide Transformer v2, showing how different architectures handle these two embedding strategies differently in terms of privacy <ref:2603.06950#pg2>.

Ines: A really interesting result they highlight is the tokenisation strategy itself being a big factor; for instance, DNABERT-two uses Byte Pair Encoding or BPE, and they found that this method coincides with substantially stronger resilience to inversion attacks compared to Nucleotide Transformer v2's fixed six-mer tokenisers <ref:2603.06950#pg3>.

Yuki: That suggests the way a model decides how to break down a sequence into tokens is actually a key part of its privacy profile, which is something I haven't thought about before.

Marcus: And they also looked at the trade-offs with sequence length; they found that longer sequences are more sensitive because there's more data, but they’re harder to reconstruct from mean-pooled embeddings because averaging discards the positional information <ref:2603.06950#pg3>.

Ines: That leads to a trade-off where shorter sequences carry less genomic content, which seems intuitively safer for privacy, but they also found that shorter sequences are substantially easier to invert with mean-pooled embeddings <ref:2603.06950#pg3>.

Conclusion: Ines: Looking at the paper "How Private Are DNA Embeddings? Inverting Foundation Model Representations of Genomic Sequences," the main thing is that per-token embeddings offer virtually no meaningful privacy protection when you're sharing them.

Marcus: That's right, and mean pooled embeddings only provide partial resilience, and that level of protection varies a lot depending on which model you use and how long the sequence is.

Yuki: So for someone who just listens to the show, it means that sharing these embeddings in an Embeddings-as-a-Service framework isn't inherently safe without some kind of specific defense in place.

Ines: The authors suggest that this finding motivates the development of embedding-level privacy defenses for when we want to safely adopt these EaaS paradigms in genomics.

Marcus: They also point out that they used a single reference genome for training, which is a limitation they acknowledge, and they didn't actually test defenses like differential privacy or embedding perturbation yet.

Yuki: I think the most important thing to take away is that we need to be very careful about how we deploy these genomic foundation models in collaborative settings because the vulnerability extends beyond just the reference genome to real patient data, as shown by their cross-dataset validation <ref:2603.06950#pg3>.

More episodes

← Home