How Private Are DNA Embeddings? Inverting Foundation Model Representations of Genomic Sequences

arXiv:2603.06950 · q-bio.GN, cs.AI, cs.LG · Submitted 2026-03-06 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.

Ines: We're looking at the paper: "How Private Are DNA Embeddings? Inverting Foundation Model Representations of Genomic Sequences".

Marcus: Here's the gist of the paper. DNA foundation models can be susceptible to model inversion attacks when their embeddings are shared via Embeddings-as-a-Service (EaaS) frameworks, necessitating urgent privacy-aware design prior to widespread deployment The study evaluates the resilience of DNA foundation models to model inversion attacks where adversaries attempt to reconstruct sensitive training data from model outputs Our results show that per-token embeddings allow nearperfect sequence reconstruction across all models Model Inversion Attack Setup Model inversion attacks aim to recover original input sequences from their embedded representations by training an inversion model as a decoder network The study focuses on training-based attacks where the inversion model is optimized to minimize the reconstruction loss L = R(x, I(M(x))) The evaluation involves three DNA foundation models: DNABERT-2, Evo 2, and Nucleotide Transformer v2 (NTv2) Embedding Strategy Comparison The research compares two embedding sharing strategies: per-token embeddings and mean-pooled sequence embeddings Per-token embeddings provide virtually no privacy protection, allowing near-perfect reconstruction using a simple MLP across all three foundation models Mean pooling offers partial but insufficient protection, as meaningful partial reconstruction remains possible for shorter sequences and specific models Tokenization and Resilience The tokenisation strategy of the foundation model is a suspected determinant of privacy, with DNABERT-2’s Byte Pair Encoding (BPE) coinciding with substantially stronger resilience to inversion attacks compared to NTv2’s fixed 6-mer and Evo 2’s single-nucleotide tokenisers This difference is hypothesized to stem from variable-length BPE tokens introducing alignment ambiguity, which hinders inversion Sequence Length Trade-offs The privacy risk of sequence length presents a trade-off where longer sequences are more sensitive but harder to reconstruct from mean-pooled embeddings because averaging discards positional information Shorter sequences carry less genomic content but are substantially easier to invert, implying no single safe sequence length exists Correlation as a Diagnostic Tool The correlation between pairwise embedding distances and sequence similarities serves as a reliable predictor of reconstruction quality across models and sequence lengths A stronger correlation indicates that the embedding space preserves sequence-level structure, facilitating reconstruction This metric can be used as a lightweight diagnostic for assessing privacy risk without a full attack evaluation Generalizability to Real Data The vulnerability of DNA foundation model embeddings extends to individual level genomic data, as reconstruction performance on sequences derived from the 1000 Genomes Project closely mirrors results obtained on hg38 This confirms that the attack scenario generalises to private genomes The study concludes that per-token embeddings offer no meaningful privacy protection, while mean pooling provides partial resilience that varies substantially across models and sequence lengths The findings motivate the development of embedding-level privacy defences for the safe adoption of EaaS paradigms in genomics Future work should explore defenses such as differential privacy or embedding perturbation to mitigate identified vulnerabilities Key Findings Summary 1.

Ines: Per-token embeddings allow near-perfect reconstruction using a simple MLP, demonstrating that the vulnerability is inherent to the embeddings rather than dependent on large attack models 2.

Marcus: Mean pooling provides partial but insufficient protection, with meaningful partial reconstruction possible for shorter sequences and specific models 3.

Ines: BPE tokenisation in DNABERT-2 coincides with considerably greater reconstruction difficulty, suggesting tokenisation design warrants further investigation as a factor in privacy-aware model design 4.

Paper summary: Marcus: Moving into the details of this paper on "How Private Are DNA Embeddings? Inverting Foundation Model Representations of Genomic Sequences," they focus on comparing two specific sharing strategies for those embeddings, <ref:2603.06950#pg2>.

Ines: They are comparing per-token embeddings, where each token from the foundation model's tokeniser gets its own vector, against mean-pooled sequence embeddings, which is just averaging all those position-specific vectors together to get one fixed-size representation.

Yuki: I wonder how that averaging affects the information we keep about the sequence structure itself versus just a general summary.

Marcus: Well, they found that per-token embeddings preserve positional information and the full per-token structure of what the foundation model outputs, as shown in equation three for sequences of length l, where n is the number of tokens produced by the tokeniser <ref:2603.06950#pg3>.

Ines: That means that when you use those per-token embeddings, you have all that positional information locked in, which makes it super vulnerable to reconstruction.

Yuki: So the mean pooling is supposed to be the defense because it discards some of that specific positional detail, making it less reconstructible.

Marcus: They test this across three models: DNABERT-two Evo two and Nucleotide Transformer v2, showing how different architectures handle these two embedding strategies differently in terms of privacy <ref:2603.06950#pg2>.

Ines: A really interesting result they highlight is the tokenisation strategy itself being a big factor; for instance, DNABERT-two uses Byte Pair Encoding or BPE, and they found that this method coincides with substantially stronger resilience to inversion attacks compared to Nucleotide Transformer v2's fixed six-mer tokenisers <ref:2603.06950#pg3>.

Yuki: That suggests the way a model decides how to break down a sequence into tokens is actually a key part of its privacy profile, which is something I haven't thought about before.

Marcus: And they also looked at the trade-offs with sequence length; they found that longer sequences are more sensitive because there's more data, but they’re harder to reconstruct from mean-pooled embeddings because averaging discards the positional information <ref:2603.06950#pg3>.

Ines: That leads to a trade-off where shorter sequences carry less genomic content, which seems intuitively safer for privacy, but they also found that shorter sequences are substantially easier to invert with mean-pooled embeddings <ref:2603.06950#pg3>.

Conclusion: Ines: Looking at the paper "How Private Are DNA Embeddings? Inverting Foundation Model Representations of Genomic Sequences," the main thing is that per-token embeddings offer virtually no meaningful privacy protection when you're sharing them.

Marcus: That's right, and mean pooled embeddings only provide partial resilience, and that level of protection varies a lot depending on which model you use and how long the sequence is.

Yuki: So for someone who just listens to the show, it means that sharing these embeddings in an Embeddings-as-a-Service framework isn't inherently safe without some kind of specific defense in place.

Ines: The authors suggest that this finding motivates the development of embedding-level privacy defenses for when we want to safely adopt these EaaS paradigms in genomics.

Marcus: They also point out that they used a single reference genome for training, which is a limitation they acknowledge, and they didn't actually test defenses like differential privacy or embedding perturbation yet.

Yuki: I think the most important thing to take away is that we need to be very careful about how we deploy these genomic foundation models in collaborative settings because the vulnerability extends beyond just the reference genome to real patient data, as shown by their cross-dataset validation <ref:2603.06950#pg3>.

University of Tuebingen

q-bio.GN, cs.AI, cs.LG

Submitted: 2026-03-06

Updated: 2026-11-15

Code: https://github.com/not-a-feature/DNA-Embedding-Inversion

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 79/100

The gist: DNA foundation models can be susceptible to model inversion attacks when their embeddings are shared via Embeddings-as-a-Service (EaaS) frameworks…

Key concepts

Model Inversion Attack
This is an attempt by an adversary to reverse-engineer the original sensitive input data (like a DNA sequence) from the model's output or its learned numerical representations (embeddings). The attack works by training a separate 'inversion model' designed to predict the original input based on the embedding.
Per-token Embeddings
This strategy involves storing and sharing embeddings for every single token in a sequence separately. The study found this method provides virtually no privacy protection because a simple model can reconstruct the entire sequence with near-perfect accuracy, making it highly vulnerable.
Mean-Pooled Sequence Embeddings
Instead of sharing individual tokens, this method averages all the token embeddings in a sequence to create one single vector representing the whole sequence. This offers partial resilience against inversion attacks, but meaningful reconstruction is still possible for shorter sequences and specific models.
Tokenization Strategy
This refers to how a foundation model breaks down an input sequence (like DNA) into smaller pieces (tokens). The paper suggests that variable-length tokenizers, such as BPE in DNABERT-2, introduce alignment ambiguity that makes the sequence much harder to reconstruct than fixed-length tokenization.

Terminology

Summary

DNA foundation models can be susceptible to model inversion attacks when their embeddings are shared via Embeddings-as-a-Service (EaaS) frameworks, necessitating urgent privacy-aware design prior to widespread deployment The study evaluates the resilience of DNA foundation models to model inversion attacks where adversaries attempt to reconstruct sensitive training data from model outputs Our results show that per-token embeddings allow nearperfect sequence reconstruction across all models

Model Inversion Attack Setup

Model inversion attacks aim to recover original input sequences from their embedded representations by training an inversion model as a decoder network The study focuses on training-based attacks where the inversion model is optimized to minimize the reconstruction loss L = R(x, I(M(x))) The evaluation involves three DNA foundation models: DNABERT-2, Evo 2, and Nucleotide Transformer v2 (NTv2)

Embedding Strategy Comparison

The research compares two embedding sharing strategies: per-token embeddings and mean-pooled sequence embeddings Per-token embeddings provide virtually no privacy protection, allowing near-perfect reconstruction using a simple MLP across all three foundation models Mean pooling offers partial but insufficient protection, as meaningful partial reconstruction remains possible for shorter sequences and specific models

Tokenization and Resilience

The tokenisation strategy of the foundation model is a suspected determinant of privacy, with DNABERT-2’s Byte Pair Encoding (BPE) coinciding with substantially stronger resilience to inversion attacks compared to NTv2’s fixed 6-mer and Evo 2’s single-nucleotide tokenisers This difference is hypothesized to stem from variable-length BPE tokens introducing alignment ambiguity, which hinders inversion

Sequence Length Trade-offs

The privacy risk of sequence length presents a trade-off where longer sequences are more sensitive but harder to reconstruct from mean-pooled embeddings because averaging discards positional information Shorter sequences carry less genomic content but are substantially easier to invert, implying no single safe sequence length exists

Correlation as a Diagnostic Tool

The correlation between pairwise embedding distances and sequence similarities serves as a reliable predictor of reconstruction quality across models and sequence lengths A stronger correlation indicates that the embedding space preserves sequence-level structure, facilitating reconstruction This metric can be used as a lightweight diagnostic for assessing privacy risk without a full attack evaluation

Generalizability to Real Data

The vulnerability of DNA foundation model embeddings extends to individual level genomic data, as reconstruction performance on sequences derived from the 1000 Genomes Project closely mirrors results obtained on hg38 This confirms that the attack scenario generalises to private genomes

Conclusion and Future Directions

The study concludes that per-token embeddings offer no meaningful privacy protection, while mean pooling provides partial resilience that varies substantially across models and sequence lengths The findings motivate the development of embedding-level privacy defences for the safe adoption of EaaS paradigms in genomics Future work should explore defenses such as differential privacy or embedding perturbation to mitigate identified vulnerabilities

Key Findings Summary

  1. Per-token embeddings allow near-perfect reconstruction using a simple MLP, demonstrating that the vulnerability is inherent to the embeddings rather than dependent on large attack models

  2. Mean pooling provides partial but insufficient protection, with meaningful partial reconstruction possible for shorter sequences and specific models

  3. BPE tokenisation in DNABERT-2 coincides with considerably greater reconstruction difficulty, suggesting tokenisation design warrants further investigation as a factor in privacy-aware model design

  4. A privacy trade-off exists where sharing shorter sequences intuitively exposes less patient information but increases vulnerability to inversion attacks

  5. Embedding similarity predicts attack success, and this correlation could be used as a lightweight diagnostic for assessing the privacy risk of a given embedding scheme

  6. The results emphasize the necessity for rigorous privacy evaluation when deploying genomic foundation models in collaborative settings

Model Specific Vulnerabilities

Evo 2 is the most vulnerable model for shorter sequences, exhibiting a distinctive non-monotonic pattern where very short sequences yield lower reconstruction quality than slightly longer ones DNABERT-2 is the most resilient model, with Levenshtein similarities ranging from 0.46 (l = 10) to 0.47 (l = 100), comparable to the Nearest Neighbour baseline NTv2 shows sustained reconstruction quality across sequence lengths, achieving a Levenshtein similarity of 0. p. 4

Where the paper says it

NTv2 shows the strongest correlation at longer sequence lengths (up to 0. 231 at l = 100), consistent with its sustained reconstruction quality.

90 ± 0.11 at l = 10 and 0.57 ± 0.06 at l = 100

Reconstruction Method Performance

For mean-pooled embeddings, the encoder-only Transformer consistently outperforms all other inversion methods across all sequence lengths The Nearest Neighbour baseline provides a Levenshtein similarity of approximately 0.45–0.54 and accuracy of 0. p. 2

Where the paper says it

The success of this attack would compromise the privacy guarantees that the embedding transformation was intended to provide. Metrics To evaluate the reconstruction quality of the model inversion attack, we employ two sequence comparison metrics: nucleotide accuracy and Levenshtein distance.

28–0.37 for longer sequences (l ≥ 50)

Tokenization Effect Detail

DNABERT-2’s BPE tokeniser generates variable-length tokens that depend on sequence content, making reconstruction inherently more difficult as the model must resolve both token boundaries and nucleotide identities This variability means that the inversion model must resolve both token boundaries and nucleotide identities, significantly increasing the reconstruction difficulty compared to fixed-length tokenisation schemes

Cross-Dataset Validation

The reconstruction performance on real patient sequences closely mirrors the results obtained on hg38, indicating that the vulnerability of DNA foundation model embeddings is not an artefact of the reference genome but extends to realistic, individually derived genomic data This confirms that the attack scenario generalises to individual level genomic data

Inversion Model Architectures

The study utilized four types of inversion models for the attack: Encoder-only Transformer, Decoder-only Transformer, ResNet, and Nearest Neighbour Lookup All parametric models were intentionally compact to demonstrate that even small attack models can achieve meaningful reconstruction The Nearest Neighbour lookup stores all training embeddings and their corresponding sequences, returning the sequence whose embedding has the smallest Euclidean distance to the query embedding

Metrics Used for Evaluation

To evaluate reconstruction quality, two sequence comparison metrics were employed: nucleotide accuracy and Levenshtein distance Nucleotide Accuracy measures the proportion of positions where two DNA sequences share identical nucleotides Levenshtein distance quantifies the minimum number of single-nucleotide edits required to transform one sequence into another

Embedding Structure Visualization

UMAP projections visualize the structure of learned embeddings across different sequence lengths for DNABERT-2 and NTv2, showing how embedding space evolves with length The Euclidean vs Sequence Similarity Correlation analysis reveals that the correlation increases with sequence length for DNABERT-2, reaching 0.231 at l = 90

Token Count Analysis

Figure 4 illustrates how the number of tokens produced by each tokeniser scales with sequence length, showing that Evo 2 maintains a strict 1:1 ratio while DNABERT-2’s BPE tokeniser exhibits sub-linear, content-dependent growth This variability means that the inversion model must resolve both token boundaries and nucleotide identities

Ablation Experiment Results

When all inversion models were forced to use a fixed single-nucleotide tokeniser, reconstruction performance degraded slightly for models that natively use subword tokenisers like DNABERT-2 and NTv2 This degradation is hypothesized to occur because the inversion model must additionally learn to translate the embedding representations of variable-length subword tokens into single-nucleotide predictions

Inversion Model Architectures Summary

Table II summarizes the inversion model architectures, showing that all models use compact architectures (e.g., dmodel = 128, 8 attention heads, 6 layers) to demonstrate that even small attack models can achieve meaningful reconstruction The Decoder and Encoder-only Transformer use similar parameter counts of approximately 12M

Final Conclusion on Privacy

The study reveals that per-token embeddings offer virtually no privacy protection, while mean pooled embeddings offer partial resilience that varies substantially across models and sequence lengths The findings emphasize the necessity for rigorous privacy evaluation when deploying DNA foundation models in collaborative settings The study motivates the development of embedding-level privacy defences for the safe adoption of EaaS paradigms in genomics

Data Preparation Details

Embeddings are extracted from specific layers depending on the model, such as the last hidden-layer output of the transformer encoder for DNABERT-2 For mean-pooled settings, per-token embeddings are averaged across all token positions to produce a single fixed-size vector per sequence All embeddings are stored in HDF5 format with SHA-256 checksums for integrity verification

Sequence Complexity Analysis

Higher-entropy sequences (more uniform nucleotide distributions) tend to be harder to reconstruct, while highly repetitive sequences with lower informational complexity are more amenable to inversion These trends are consistent across all three foundation models and suggest that the inherent complexity of the target sequence is a limiting factor for reconstruction quality

Limitation Acknowledgment

The study acknowledges limitations, including using only a single reference genome for training and not evaluating defenses such as differential privacy or embedding perturbation Future work should explore these areas and extend the evaluation to additional foundation models and genomic contexts

Detailed Reconstruction Metrics

Table III provides detailed reconstruction results for DNABERT-2 embeddings, showing that the Decoder Levenshtein distance ranges from 0.53 ± 0.13 at l = 10 to 0.56 ± 0.07 at l = 100 The Encoder Levenshtein distance shows a high level of similarity, reaching up to 0. p. 5

Where the paper says it

0 Levenshtein Similarity DNABERT-2 Evo 2 NTv2 Random Baseline 20 40 60 80 100 Sequence Length 0. 0 Nucleotide Accuracy DNABERT-2 Evo 2 NTv2 Random Baseline Figure 3: Mean-pooled reconstruction performance across sequence lengths for the encoder-only architecture: (a) Levenshtein similarity and (b) nucleotide accuracy.

99 ± 0.04 at l = 25

Cross-Dataset Performance

Reconstruction performance on real patient sequences closely mirrors the results obtained on hg38, indicating that the vulnerability of DNA foundation model embeddings is not an artefact of the reference genome but extends to realistic, individually derived genomic data This confirms that the attack scenario generalises to individual level genomic data The reconstruction performance on 1000 Genomes Project data is consistent with hg38 results

Final Summary Statement

Our evaluation of DNABERT-2, Evo 2, and NTv2 reveals that per-token embeddings provide no meaningful privacy protection, while mean pooled embeddings offer partial resilience that varies substantially across models and sequence lengths Three key findings emerge The study motivates the development of embedding-level privacy defences for the safe adoption of EaaS paradigms in genomics The study has some limitations, including using only a single reference genome for training and not evaluating defenses such as differential privacy or embedding perturbation

Author Contributions

S.O. and J.K. conceived the project idea and the experimental setup, while J. p. 6

Where the paper says it

AUTHOR CONTRIBUTIONS STATEMENT S. conceived the project idea and the experimental setup.

K. conducted the experiments S. p. 6
Where the paper says it

conceived the project idea and the experimental setup. conducted the experiments.

O., N., and P supervised the experiments and provided additional ideas All authors analysed the results, wrote and reviewed the manuscript

Acknowledgments

This work was supported by the Carl Zeiss Stiftung Research Project “Certification and Foundations of Safe Machine Learning Systems in Healthcare”, by the German Research Foundation (DFG) under Germany’s Excellence Strategy—EXC number 2064/1—Project number 390727645, and by the German Federal Ministry of Research, Technology and Space (BMFTR) within the PrivateAIM project The authors thank the International Max Planck Research School for Intelligent Systems (IMPRS-IS)

References Cited

Adila, D., Shin, C., Cai, L., et al. (2024). Zero-shot robustification of zero-shot models Awais, M. p. 6

Where the paper says it

REFERENCES Adila, D. Zero-shot robustification of zero-shot models.

, Naseer, M. p. 6
Where the paper says it

Zero-shot robustification of zero-shot models. , Naseer, M.

, Khan, S., et al. (2025). Foundation models defining a new era in vision: A survey and outlook Balestriero, R. p. 6
Where the paper says it

, Naseer, M. Foundation models defining a new era in vision: A survey and outlook.

, Ibrahim, M. p. 6
Where the paper says it

Balestriero, R. , Ibrahim, M.

, Sobal, V. p. 6
Where the paper says it

, Ibrahim, M. , Sobal, V.

, et al. (2023). A cookbook of self-supervised learning Berger, B. p. 6
Where the paper says it

, Sobal, V. A cookbook of self-supervised learning.

, Waterman, M. p. 6
Where the paper says it

arXiv preprint arXiv:2304. , Waterman, M.

S., & Yu, Y. W. (2020). Levenshtein distance, sequence comparison and biological database search Bollas, A. p. 6
Where the paper says it

, Waterman, M. Levenshtein distance, sequence comparison and biological database search.

E., Rajkovic, A. p. 6
Where the paper says it

Theory, 67(6), 3287–3294. , Rajkovic, A.

, Ceyhan, D. p. 6
Where the paper says it

, Rajkovic, A. , Ceyhan, D.

, et al. (2024). Snvstory: Inferring genetic ancestry from genome sequencing data Bommasani, R. p. 6
Where the paper says it

, Ceyhan, D. Snvstory: Inferring genetic ancestry from genome sequencing data.

(2021). On the opportunities and risks of foundation models Bonomi, L. p. 6
Where the paper says it

Bommasani, R. On the opportunities and risks of foundation models.

, Huang, Y. p. 6
Where the paper says it

arXiv preprint arXiv:2108. , Huang, Y.

, & Ohno-Machado, L. p. 6
Where the paper says it

, Huang, Y. , & Ohno-Machado, L.

(2020). Privacy challenges and research opportunities for genomic data sharing Bostrom, K. p. 6
Where the paper says it

, & Ohno-Machado, L. Privacy challenges and research opportunities for genomic data sharing.

, & Durrett, G. p. 6
Where the paper says it

Bostrom, K. , & Durrett, G.

(2020). Byte pair encoding is suboptimal for language model pretraining Brixi, G. p. 6
Where the paper says it

, & Durrett, G. Byte pair encoding is suboptimal for language model pretraining.

, Durrant, M. p. 6
Where the paper says it

: EMNLP 2020, 4617–4624. , Durrant, M.

G., Ku, J., et al. (2025). Genome modeling and design across all domains of life with evo 2 Dalla-Torre, H. p. 6
Where the paper says it

, Durrant, M. Genome modeling and design across all domains of life with evo 2.

, Gonzalez, L. p. 6
Where the paper says it

Dalla-Torre, H. , Gonzalez, L.

, Mendoza-Revilla, J. p. 6
Where the paper says it

, Gonzalez, L. , Mendoza-Revilla, J.

, et al. (2025). Nucleotide transformer: Building and evaluating robust foundation models for human genomics Fairley, S. p. 6
Where the paper says it

, Mendoza-Revilla, J. Nucleotide transformer: Building and evaluating robust foundation models for human genomics.

, Lowy-Gallego, E. p. 6
Where the paper says it

Fairley, S. , Lowy-Gallego, E.

, Perry, E. p. 6
Where the paper says it

, Lowy-Gallego, E. , Perry, E.

, et al. (2019). The international genome sample resource (igsr) collection of open human genomic variation resources Feng, H. p. 6
Where the paper says it

, Perry, E. The international genome sample resource (igsr) collection of open human genomic variation resources.

, Wu, L., Zhao, B., et al. (2025). Benchmarking dna foundation models for genomic and genetic tasks Fredrikson, M. p. 6
Where the paper says it

, 48(D1), D941–D947. Benchmarking dna foundation models for genomic and genetic tasks.

, Jha, S., & Ristenpart, T. p. 6
Where the paper says it

Fredrikson, M. , & Ristenpart, T.

(2015). Model inversion attacks that exploit confidence information and basic countermeasures Guo, F. p. 6
Where the paper says it

, & Ristenpart, T. Model inversion attacks that exploit confidence information and basic countermeasures.

, Guan, R., Li, Y., et al. (2025). Foundation models in bioinformatics Levenshtein, V. p. 0
Where the paper says it

pfeifer}@uni-tuebingen. de Abstract—DNA foundation models have become transformative tools in bioinformatics and healthcare applications.

I., et al. (1966). Binary codes capable of correcting deletions, insertions, and reversals Lippert, C. p. 6
Where the paper says it

Levenshtein, V. Binary codes capable of correcting deletions, insertions, and reversals.

, Sabatini, R. p. 6
Where the paper says it

Lippert, C. , Sabatini, R.

, Maher, M. p. 6
Where the paper says it

, Sabatini, R. , Maher, M.

C., et al. (2017). Identification of individuals by trait prediction using wholegenome sequencing data Marin, F. p. 6
Where the paper says it

, Maher, M. Identification of individuals by trait prediction using wholegenome sequencing data.

I., Teufel, F. p. 6
Where the paper says it

USA, 114(38), 10166–10171. , Teufel, F.

, Horlacher, M. p. 6
Where the paper says it

, Teufel, F. , Horlacher, M.

, et al. (2024). Bend: Benchmarking dna language models on biologically meaningful tasks Naveed, M. p. 6
Where the paper says it

, Horlacher, M. Bend: Benchmarking dna language models on biologically meaningful tasks.

, Ayday, E. p. 6
Where the paper says it

Bend: Benchmarking dna language models on biologically meaningful tasks. , Ayday, E.

, Clayton, E. p. 6
Where the paper says it

, Ayday, E. , Clayton, E.

W., et al. (2015). Privacy in the genomic era Nguyen, N. p. 6
Where the paper says it

, Clayton, E. Privacy in the genomic era.

-B., Chandrasegaran, K. p. 6
Where the paper says it

, 48(1), 1–44. , Chandrasegaran, K.

, Abdollahzadeh, M. p. 6
Where the paper says it

, Chandrasegaran, K. , Abdollahzadeh, M.

, et al. (2023). Re-thinking model inversion attacks against deep neural networks Nikolaou, G. p. 6
Where the paper says it

, Abdollahzadeh, M. Re-thinking model inversion attacks against deep neural networks.

, Mencattini, T. p. 6
Where the paper says it

Nikolaou, G. , Mencattini, T.

, Crisostomi, D. p. 6
Where the paper says it

, Mencattini, T. , Crisostomi, D.

, et al. (2025). Language models are injective and hence invertible Ouaari, S. p. 6
Where the paper says it

, Crisostomi, D. Language models are injective and hence invertible.

, Ünal, A. p. 6
Where the paper says it

arXiv preprint arXiv:2510. , Ünal, A.

B., Akgün, M. p. 6
Where the paper says it

, Ünal, A. , Akgün, M.

, et al. (2025). Robust representation learning for privacy-preserving machine learning: A multi-objective autoencoder approach Radford, A. p. 6
Where the paper says it

, Akgün, M. Robust representation learning for privacy-preserving machine

, Wu, J., Child, R. p. 7
Where the paper says it

Radford, A. , Child, R.

, et al. (2019). Language models are unsupervised multitask learners Su, J. p. 7
Where the paper says it

, Child, R. Language models are unsupervised multitask learners.

, Ahmed, M. p. 7
Where the paper says it

, 24(14), 4167–4182. , Ahmed, M.

, Lu, Y., et al. (2024). Roformer: Enhanced transformer with rotary position embedding Workshop, B. p. 7
Where the paper says it

, Ahmed, M. Roformer: Enhanced transformer with rotary position embedding.

, Scao, T. L., Fan, A., et al. (2022). Bloom: A 176b-parameter open-access multilingual language model Wu, R. p. 7
Where the paper says it

Workshop, B. Bloom: A 176b-parameter open-access multilingual language model.

, Chen, X., Guo, C., et al. (2023). Learning to invert: Simple adaptive attacks for gradient inversion in federated learning Zhang, Y. p. 7
Where the paper says it

arXiv preprint arXiv:2211. Learning to invert: Simple adaptive attacks for gradient inversion in federated learning.

, Jia, R., Pei, H., et al. (2020). The secret revealer: Generative model-inversion attacks against deep neural networks Zhou, C. p. 7
Where the paper says it

, 2293–2303. The secret revealer: Generative model-inversion attacks against deep neural networks.

, Li, Q., Li, C., et al. (2025). A comprehensive survey on pretrained foundation models: A history from bert to chatgpt Zhou, S. p. 7
Where the paper says it

(CVPR), 253–261. A comprehensive survey on pretrained foundation models: A history from bert to chatgpt.

, Zhu, T., Ye, D., et al. (2023). Boosting model inversion attacks with adversarial examples Zhou, Z. p. 7
Where the paper says it

, 16(12), 9851– 9915. Boosting model inversion attacks with adversarial examples.

, Ji, Y., Li, W., et al. (2023).

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Sequence length management for privacy risk assessment can be improved by incorporating sequence complexity analysis using Shannon entropy and 4-mer repetitiveness as proxies for sequence information content to determine which lengths present the highest reconstruction difficulty.

  2. Tokenization strategy selection for foundation models should be guided by its impact on inversion difficulty; specifically, variable-length tokenization like DNABERT-2's BPE is hypothesized to increase reconstruction difficulty because the inversion model must resolve both token boundaries and nucleotide identities.

  3. Embedding similarity correlation with sequence similarity can serve as a lightweight diagnostic for assessing the privacy risk of a given embedding scheme without requiring a full attack evaluation, as the correlation between pairwise embedding distances and sequence similarities serves as a reliable predictor of reconstruction quality across models and sequence lengths.

  4. Deployment strategies should adopt an understanding that per-token embeddings offer virtually no privacy protection, meaning sharing these representations is functionally equivalent to sharing the raw sequences themselves, necessitating alternative privacy-preserving mechanisms if data sharing is required.

  5. For mean pooled embeddings, the system can leverage insights from collision analysis to ensure that embedding functions remain effectively injective over the dataset, as the absence of significant embedding collisions would suggest that the function is approximately injective, making reconstruction potentially possible.

Abstract

Genomic foundation models are increasingly used to share dense representations rather than raw DNA sequences. This is particularly pursued for mean-pooled representations that are later used for downstream prediction tasks. Whether this practice protects the privacy of the underlying sequence is largely unknown. We evaluated model-inversions against zero-shot embeddings from DNABERT-2, Evo 2, and Nucleotide Transformer v2 (NTv2). Substantial reconstruction remains possible: a single multi-length Length-Conditioned Query Decoder reconstructed NTv2 sequences with mean Levenshtein similarity from 0.98 at 10 nucleotides to 0.68 at 100, and Evo 2 sequences from nearly 1.00 at 15 nucleotides to 0.67 at 100. DNABERT-2 showed higher resistance (0.56 to 0.51), although it remained above the random baseline at short lengths. Therefore, we further evaluated, whether a linkage attack similar to Homer's attack is possible based on the mean-pooled representations of human DNA sequences. At SNP-enriched length-25 loci and known coordinates, Evo 2 and NTv2 reached 99% identification at 32 loci and DNABERT-2 at 256 loci. With alignment-based coordinate recovery, the corresponding 99% budgets were 64 and 128 loci for Evo 2 and NTv2, while DNABERT-2 did not reach the reported identification thresholds. Random loci required larger budgets. These findings show that mean-pooled embeddings can leak sequence and identity, with risk depending on sequence length, representation, locus informativeness, and coordinate information.

Sources

Related papers