C3P: Contrastive promoter-protein pretraining yields representations capturing bacterial gene regulation

arXiv:2605.25242 · q-bio.GN · Submitted 2026-05-24 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.

Ines: I'm Ines.

Marcus: And I'm Marcus, here with Yuki.

Ines: We're looking at the paper: "C3P: Contrastive promoter-protein pretraining yields representations capturing bacterial gene regulation".

Marcus: Here's the gist of the paper. Despite the increasing scale of genome language models (gLMs), their ability to decode the function of regulatory sequences remains unclear<ref:2605.25242#pg2> Self-supervised contrastive approaches provide a promising alternative<ref:2605.25242#pg2>.

Ines: By learning to align promoters to their corresponding proteins, we leverage the rich representations of proteins learned by protein language models as supervisory signal for the learning of promoter representations<ref:2605.25242#pg4>.

Marcus: After training on 88 million bacterial promoter-protein pairs, we evaluate the predictive power of C3P-learned promoter representations for inference of curated regulatory annotations, finding multi-fold improvement over leading gLMs<ref:2605.25242#pg2>.

Ines: We also introduce zero-shot co-regulated gene retrieval, the ability to find co-regulated genes in a genome using no experimental data<ref:2605.25242#pg3>.

Paper summary: Ines: So we've covered the basic setup. To expand on what C3P actually does, it’s inspired by how language-image models like CLIP work. They are learning to align two different types of sequences: DNA promoter sequences and the protein sequences they control >

Marcus: The methodology involves tokenizing the promoter sequence into overlapping three-mers and encoding it with a promoter encoder, while using a pre-trained protein encoder, specifically ESM2, for the protein sequence >

Yuki: So you have these two encoders generating embeddings, which are then passed through learnable linear projection layers so they can share a common space where alignment happens >

Ines: The training uses symmetric InfoNCE loss. This loss function makes the model maximize the cosine similarity between correct promoter-protein pairs while minimizing similarity between incorrect pairings across a large batch of data >

Marcus: They trained this on eighty-eight million bacterial promoter-protein pairs and then tested its predictive power on inferring curated regulatory annotations for a held-out species > <ref:2605.25242#pg1,on 88 million bacterial promoter-protein pairs>

Yuki: And according to the paper, they found that these C3P learned representations showed multi-fold improvement over leading gLMs when predicting those annotations >

Ines: They also showed that this approach provides significant zero-shot performance gains compared to a randomly initialized baseline, unlike what they saw with the standard gLMs >

Marcus: The key thing here is the scaling analysis. They managed to achieve strong performance at a fraction of the training cost required by leading gLMs >

Yuki: So, while it’s better than existing methods on this specific task, you have to consider that they are still comparing this against models that already learn about protein structure well >

Ines: Right. The paper shows that the promoter embeddings learned by C3P actually outperformed the ESM2 protein embeddings on their own when tested >

Conclusion: Marcus: So wrapping up the "C3P: Contrastive promoter-protein pretraining yields representations capturing bacterial gene regulation" paper, it seems like this self-supervised contrastive alignment method is a promising path for learning about regulatory sequences in genomes >

Ines: It moves away from the struggle of standard language models by using protein knowledge as a guide to learn what promoters actually do >

Yuki: For the wider picture, if this method can indeed infer regulation from DNA alone, it opens up the possibility of building complete models of cells entirely from their genomic sequence, which is a huge concept for understanding life >

Marcus: That zero-shot co-regulated gene retrieval capability is what really stands out as something that could change how we approach regulatory genomics across different bacterial datasets >

Ines: It’s important to remember the paper's caveat. They noted that while this method improves prediction, it still doesn't fully decode the function of regulatory sequences in a definitive way yet >

Yuki: So, it’s an improvement over what we have now, but it’s not the final answer for understanding all bacterial regulation >

Marcus: Exactly. The success is in showing that this contrastive alignment approach works well with just sequence data and existing protein knowledge to recover regulatory signals better than current gLMs >

University of Toronto

q-bio.GN

Submitted: 2026-05-24

Updated: 2026-11-05

Comments: Accepted (Oral) at 40th Conference on Neural Information Processing Systems (NeurIPS 2026)

Code: https://github.com/dufaultc/contrastive-promoter-protein-pretraining

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 96/100

The gist: Despite the increasing scale of genome language models (gLMs), their ability to decode the function of regulatory sequences remains unclear<ref:2605.25242#pg2> Self-supervised contrastive approaches…

Key concepts

Genome Language Models (gLMs)
These are deep learning models trained on DNA sequences to learn general representations of genomes. The paper investigates whether these models naturally capture the function of regulatory sequences, finding they often struggle in this specific area.
Protein Language Models (pLMs)
These models analyze amino acid sequences from proteins to understand their structure and function. Their rich protein representations are used as a supervisory signal to train the promoter encoder in the C3P approach.
Contrastive Learning (InfoNCE Loss)
This training technique forces the model to learn an alignment between two different types of data—promoters and proteins. The model is trained to maximize similarity between correct pairs while minimizing similarity between incorrect pairings, effectively teaching it what a functional promoter sequence looks like.
Zero-shot Co-regulated Gene Retrieval
This is the ability to find genes that are regulated together in a genome without any prior experimental data for those specific genes. C3P training significantly improves this capability compared to standard gLMs.

Terminology

Summary

Despite the increasing scale of genome language models (gLMs), their ability to decode the function of regulatory sequences remains unclear<ref:2605. p. 0

Where the paper says it

C3P: Contrastive promoter-protein pretraining yields representations capturing bacterial gene regulation Cameron Dufault∗ Department of Computer Science University of Toronto Scott Xu Department of Computer Science University of Toronto Alan M. Moses Department of Cell and Systems Biology Department of Computer Science University of Toronto Abstract Despite the increasing scale of genome language models (gLMs), their ability to decode the function of regulatory sequences remains unclear.

25242#pg2> Self-supervised contrastive approaches provide a promising alternative<ref:2605. p. 0
Where the paper says it

gLM pretraining relies on sequence reconstruction, which may struggle due to the noisy, rapidly evolving nature of regulatory DNA. Self-supervised contrastive approaches provide a promising alternative.

25242#pg2>. By learning to align promoters to their corresponding proteins, we leverage the rich representations of proteins learned by protein language models as supervisory signal for the learning of promoter representations<ref:2605. p. 0
Where the paper says it

Inspired by language-image architectures like CLIP, we introduce contrastive promoter-protein pretraining (C3P). By learning to align promoters to their corresponding proteins, we leverage the rich representations of proteins learned by protein language models as supervisory signal for the learning of promoter representations.

25242#pg4>. After training on 88 million bacterial promoter-protein pairs, we evaluate the predictive power of C3P-learned promoter representations for inference of curated regulatory annotations, finding multi-fold improvement over leading gLMs<ref:2605. p. 0
Where the paper says it

By learning to align promoters to their corresponding proteins, we leverage the rich representations of proteins learned by protein language models as supervisory signal for the learning of promoter representations. After training on 88 million bacterial promoter-protein pairs, we evaluate the predictive power of C3P-learned promoter representations for inference of curated regulatory annotations, finding multi-fold improvement over leading gLMs.

25242#pg2>. We also introduce zero-shot co-regulated gene retrieval, the ability to find co-regulated genes in a genome using no experimental data<ref:2605. p. 0
Where the paper says it

After training on 88 million bacterial promoter-protein pairs, we evaluate the predictive power of C3P-learned promoter representations for inference of curated regulatory annotations, finding multi-fold improvement over leading gLMs. We also introduce zero-shot co-regulated gene retrieval, the ability to find co-regulated genes in a genome using no experimental data.

25242#pg3>. We find that compared to a randomly initialized baseline, C3P training consistently provides significant zero-shot performance gains, unlike gLMs<ref:2605. p. 0
Where the paper says it

We also introduce zero-shot co-regulated gene retrieval, the ability to find co-regulated genes in a genome using no experimental data. We find that compared to a randomly initialized baseline, C3P training consistently provides significant zero-shot performance gains, unlike gLMs.

p. 0
Where the paper says it

We also introduce zero-shot co-regulated gene retrieval, the ability to find co-regulated genes in a genome using no experimental data. We find that compared to a randomly initialized baseline, C3P training consistently provides significant zero-shot performance gains, unlike gLMs.

25242#pg3>.

Introduction and Motivation

The biological effects of genes are determined by both the gene product (protein, tRNA, etc.) and when and where the product is expressed<ref:2605. p. 0

Where the paper says it

1 Introduction The biological effects of genes are determined by both the gene product (protein, tRNA, etc. ) and when and where the product is expressed.

25242#pg2>. Deep learning has revolutionized the tools for computational prediction of structure and function of proteins directly from their amino acid sequences [15]<ref:2605. p. 0
Where the paper says it

) and when and where the product is expressed. Deep learning has revolutionized the tools for computational prediction of structure and function of proteins directly from their amino acid sequences [15], but predicting the regulation of gene expression directly from DNA sequences remains a long-standing challenge.

25242#pg3>. Predicting the regulation of gene expression directly from DNA sequences remains a long-standing challenge<ref:2605. p. 0
Where the paper says it

) and when and where the product is expressed. Deep learning has revolutionized the tools for computational prediction of structure and function of proteins directly from their amino acid sequences [15], but predicting the regulation of gene expression directly from DNA sequences remains a long-standing challenge.

25242#pg4>. Supervised deep learning approaches have emerged for inferring regulation from DNA [2pg3], but require training on experimental data which is unavailable for the vast majority of organisms, in particular the millions of bacterial species [2pg3]. p. 0
Where the paper says it

Without regulatory context, we cannot determine from the genome sequence alone how the expression of encoded genes is orchestrated to build the cell and adapt to the environment [34]. Supervised deep learning approaches have emerged for inferring regulation from DNA [2, 3, 21], but require training on experimental data which is unavailable for the vast majority of organisms, ∗dufaultc@cs.

With the vast quantity of unlabeled DNA sequencing data now available, there is clear opportunity for unsupervised approaches to help us learn regulatory function<ref:2605. p. 1
Where the paper says it

Despite the relative simplicity of their genomes, where the regulatory sequences are primarily contained in the regions between proteins (hereafter referred to as promoters), for most bacterial species we know little about their gene regulation [34, 4]. With the vast quantity of unlabeled DNA sequencing data now available, there is clear opportunity for unsupervised approaches to help us learn regulatory function.

25242#pg3>. Existing unsupervised approaches to learning from the genome have centered on language models [7]<ref:2605. p. 1
Where the paper says it

With the vast quantity of unlabeled DNA sequencing data now available, there is clear opportunity for unsupervised approaches to help us learn regulatory function. Existing unsupervised approaches to learning from the genome have centered on language models [7].

25242#pg4>. However, while protein language models (pLMs), which operate on sequences of amino acids, succeed in learning representations capturing the structure and function of proteins [2pg3], multiple analyses have called into question whether existing genome language models (gLMs), which operate on DNA sequences, learn about gene regulation from their pretraining [45, 49]<ref:2605. p. 1
Where the paper says it

Existing unsupervised approaches to learning from the genome have centered on language models [7]. However, while protein language models (pLMs), which operate on sequences of amino acids, succeed in learning representations capturing the structure and function of proteins [20, 37], multiple analyses have called into question whether existing genome language models (gLMs), which operate on DNA sequences, learn about gene regulation from their pretraining [45, 49].

25242#pg4>. Specific demonstrations that such models learn to capture bacterial regulatory sequence function have been limited<ref:2605. p. 1
Where the paper says it

However, while protein language models (pLMs), which operate on sequences of amino acids, succeed in learning representations capturing the structure and function of proteins [20, 37], multiple analyses have called into question whether existing genome language models (gLMs), which operate on DNA sequences, learn about gene regulation from their pretraining [45, 49]. Specific demonstrations that such models learn to capture bacterial regulatory sequence function have been limited.

25242#pg4>.

How the training signal is built

The C3P approach is inspired by language-image architectures like CLIP [35]<ref:2605.25242#pg3>. The objective is to align the learned representations of promoters with the representations of their corresponding proteins<ref:2605. p. 3

Where the paper says it

1 Objective and Architecture An overview of the C3P architecture and training task is shown in Figure 1. Inspired by languageimage models such as CLIP [35], the objective is to align the learned representations of promoters with the representations of their corresponding proteins.

25242#pg5>. Given a promoter-protein pair, the promoter sequence is tokenized into overlapping 3-mers and encoded by the promoter encoder, a randomly initialized transformer encoder [48]<ref:2605. p. 3
Where the paper says it

Inspired by languageimage models such as CLIP [35], the objective is to align the learned representations of promoters with the representations of their corresponding proteins. Given a promoter-protein pair, the promoter sequence is tokenized into overlapping 3-mers and encoded by the promoter encoder, a randomly initialized transformer encoder [48].

25242#pg18>. The protein sequence is encoded by a frozen pretrained protein encoder, for which we utilize ESM2 (150M) [20], with embeddings averaged over the sequence length<ref:2605. p. 3
Where the paper says it

Given a promoter-protein pair, the promoter sequence is tokenized into overlapping 3-mers and encoded by the promoter encoder, a randomly initialized transformer encoder [48]. The protein sequence is encoded by a frozen pretrained protein encoder, for which we utilize ESM2 (150M) [20], with embeddings averaged over the sequence length.

25242#pg5>. The outputs of both encoders are then passed through learnable linear projection layers to have shared dimensionality (dprojection = 256)<ref:2605. p. 3
Where the paper says it

The protein sequence is encoded by a frozen pretrained protein encoder, for which we utilize ESM2 (150M) [20], with embeddings averaged over the sequence length. The outputs of both encoders are then passed through learnable linear projection layers to have shared dimensionality (dprojection = 256).

25242#pg5>. The model is trained with symmetric InfoNCE loss [30]<ref:2605. p. 3
Where the paper says it

The outputs of both encoders are then passed through learnable linear projection layers to have shared dimensionality (dprojection = 256). The model is trained with symmetric InfoNCE loss [30].

25242#pg5>. Given a batch of N promoter-protein pairs, the model learns to maximize the cosine similarity between the N correct promoter-protein pairs while minimizing the similarity between the N2 − N incorrect pairings<ref:2605. p. 3
Where the paper says it

Given a batch of N promoter-protein pairs, let z prom i and z prot i denote the L2-normalized, projected representations of the i-th promoter and protein, respectively. For each batch, the model learns to maximize the cosine similarity between the N correct promoter-protein pairs while minimizing the similarity between the N2 − N incorrect pairings.

25242#pg5>.

Where the gains come from

The multi-fold improvement over leading gLMs is demonstrated through several evaluations<ref:2605.25242#pg2>.

  1. Nearest neighbour prediction of curated regulatory annotations in a held-out species shows multi-fold improvement over leading gLMs at capturing regulatory function in bacterial promoter representations, despite having orders of magnitude fewer parameters<ref:2605. p. 1

    Where the paper says it

    After training on 88 million promoter-protein pairs from ∼23,000 bacterial species, C3P models show a multi-fold improvement over leading gLMs at capturing regulatory function in bacterial promoter representations, despite having orders of magnitude fewer parameters. We evaluate this through nearest neighbour prediction of curated regulatory annotations in a held-out species, as well as performance at zero-shot co-regulated gene retrieval, an evaluation framework we introduce for exploring the potential for solving a grand challenge of regulatory genomics: inferring regulation from the genome alone.

    25242#pg2>.

  2. Zero-shot co-regulated gene retrieval, an evaluation framework we introduce for exploring the potential for solving a grand challenge of regulatory genomics: inferring regulation from the genome alone<ref:2605. p. 1

    Where the paper says it

    After training on 88 million promoter-protein pairs from ∼23,000 bacterial species, C3P models show a multi-fold improvement over leading gLMs at capturing regulatory function in bacterial promoter representations, despite having orders of magnitude fewer parameters. We evaluate this through nearest neighbour prediction of curated regulatory annotations in a held-out species, as well as performance at zero-shot co-regulated gene retrieval, an evaluation framework we introduce for exploring the potential for solving a grand challenge of regulatory genomics: inferring regulation from the genome alone.

    25242#pg2>. We find that compared to a randomly initialized baseline, C3P training consistently provides significant zero-shot performance gains, unlike gLMs<ref:2605.25242#pg2>.

  3. C3P models show substantial improvement over their random baselines at top-1 full and any match accuracy on all 25 iModulonDB datasets<ref:2605. p. 7

    Where the paper says it

    C3P demonstrates multi-fold improvement over its random baselines across nearly all datasets, while other approaches broadly show minor improvement or in some cases worsened performance (Figure 5, Appendix F). 4 C3P demonstrates favorable scaling for regulatory inference compared to gLMs Noting that Evo2 showed evidence of improvement in only 3 (top-1 full match) and 2 (top-1 any match) more datasets than PromoGen2, despite having ∼45x more parameters, we next performed an analysis of the scaling of C3P and compared with the empirical trend of our gLM baselines.

    25242#pg7>.

Scaling and Generalization

The scaling analysis reveals the potential for further improvement as well as the efficiency of C3P, which achieved strong performance at a fraction of the training cost of leading gLMs<ref:2605.25242#pg2>. The success of C3P at zero-shot co-regulated gene retrieval suggests the possibility of inferring the regulatory networks of entirely unstudied species, an important step towards the ability to build complete models of cells entirely from the genome [34]<ref:2605. p. 9

Where the paper says it

Furthermore, we confirmed that C3P consistently leads to significant performance improvement over a randomly initialized baseline, unlike gLMs. The success of C3P at zero-shot co-regulated gene retrieval suggests the possibility of inferring the regulatory networks of entirely unstudied species, an important step towards the ability to build complete models of cells entirely from the genome [34].

p. 9
Where the paper says it

Furthermore, we confirmed that C3P consistently leads to significant performance improvement over a randomly initialized baseline, unlike gLMs. The success of C3P at zero-shot co-regulated gene retrieval suggests the possibility of inferring the regulatory networks of entirely unstudied species, an important step towards the ability to build complete models of cells entirely from the genome [34].

25242#pg2>. Furthermore, C3P promoter embeddings also outperformed ESM2 (150M) protein embeddings<ref:2605. p. 8
Where the paper says it

We believe that this is due to regulatory sequences being better suited for joint-embedding based self-supervised learning objectives than reconstruction-based objectives [1], as well as our use of a pretrained pLM. C3P promoter embeddings also outperformed ESM2 (150M) protein embeddings.

25242#pg7>.

Conclusion

Genome language models are known to struggle to decode the function of regulatory sequences[45, 49]. Inspired by the success of multi-modal contrastive alignment in the language-image domain [35], we introduced C3P, a novel self-supervised approach for learning representations of regulatory sequences<ref:2605. p. 9

Where the paper says it

6 Conclusion Genome language models are known to struggle to decode the function of regulatory sequences [45, 49]. Inspired by the success of multi-modal contrastive alignment in the language-image domain [35], we introduced C3P, a novel self-supervised approach for learning representations of regulatory sequences.

25242#pg2>. Using C3P, we trained a promoter encoder on 88 million bacterial promoter-protein pairs and evaluated its performance on nearest-neighbour prediction of gold-standard regulatory annotations from a held-out species, as well as zero-shot co-regulated gene retrieval across 25 diverse bacterial datasets<ref:2605. p. 9
Where the paper says it

Inspired by the success of multi-modal contrastive alignment in the language-image domain [35], we introduced C3P, a novel self-supervised approach for learning representations of regulatory sequences. Using C3P, we trained a promoter encoder on 88 million bacterial promoter-protein pairs and evaluated its performance on nearest-neighbour prediction of gold-standard regulatory annotations from a held-out species, as well as zero-shot co-regulated gene retrieval across 25 diverse bacterial datasets.

25242#pg2>. We demonstrated that C3P representations are substantially more predictive of gene regulation than leading gLMs, despite requiring orders of magnitude less compute<ref:2605. p. 9
Where the paper says it

Using C3P, we trained a promoter encoder on 88 million bacterial promoter-protein pairs and evaluated its performance on nearest-neighbour prediction of gold-standard regulatory annotations from a held-out species, as well as zero-shot co-regulated gene retrieval across 25 diverse bacterial datasets. We demonstrated that C3P representations are substantially more predictive of gene regulation than leading gLMs, despite requiring orders of magnitude less compute.

25242#pg2>. Furthermore, we confirmed that C3P consistently leads to significant performance improvement over a randomly initialized baseline, unlike gLMs<ref:2605. p. 9
Where the paper says it

We demonstrated that C3P representations are substantially more predictive of gene regulation than leading gLMs, despite requiring orders of magnitude less compute. Furthermore, we confirmed that C3P consistently leads to significant performance improvement over a randomly initialized baseline, unlike gLMs.

25242#pg2>. The success of C3P at zero-shot co-regulated gene retrieval suggests the possibility of inferring the regulatory networks of entirely unstudied species, an important step towards the ability to build complete models of cells entirely from the genome [34]<ref:2605.25242#pg2>.

Acknowledgments

We thank David Knowles, Alex Lu, Philip Fradkin, Andrew Duncan, Ami Sangster, and Rain Jin for feedback on the manuscript<ref:2605.25242#pg2>. CD and AMM acknowledge support from the Canada Research Chairs program<ref:2605. p. 9

Where the paper says it

7 Acknowledgments We thank David Knowles, Alex Lu, Philip Fradkin, Andrew Duncan, Ami Sangster, and Rain Jin for feedback on the manuscript. CD and AMM acknowledge support from the Canada Research Chairs program.

25242#pg2>. Computing resources used in preparing this research were provided, in part by the Province of Ontario, the Government of Canada through CIFAR, and companies sponsoring the Vector Institute<ref:2605. p. 9
Where the paper says it

CD and AMM acknowledge support from the Canada Research Chairs program. Computing resources used in preparing this research were provided, in part, by the Province of Ontario, the Government of Canada through CIFAR, and companies sponsoring the Vector Institute.

25242#pg2>.

Data and Code Availability

All data used in this work was sourced from publicly available databases<ref:2605.25242#pg2>. Codes for downloading data, extracting promoter protein pairs, and training C3P models are available at https://github. p. 9

Where the paper says it

8 Data and Code Availability All data used in this work was sourced from publicly available databases. Codes for downloading data, extracting promoter protein pairs, and training C3P models are available at https://github.

com/dufaultc/contrastive-promoter-protein-pretraining<ref:2605. p. 9
Where the paper says it

Codes for downloading data, extracting promoter protein pairs, and training C3P models are available at https://github. com/dufaultc/contrastive-promoter-protein-pretraining.

25242#pg2>. Trained models available at https://huggingface. p. 9
Where the paper says it

com/dufaultc/contrastive-promoter-protein-pretraining. Trained models available at https://huggingface.

co/dufaultc/contrastive-promoter-protein-pretraining<ref:2605. p. 9
Where the paper says it

Codes for downloading data, extracting promoter protein pairs, and training C3P models are available at https://github. com/dufaultc/contrastive-promoter-protein-pretraining.

25242#pg2>.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Contrastive promoter-protein pretraining (C3P) can be used to learn representations of bacterial regulatory sequences by learning to align promoters to their corresponding proteins, which allows for capturing functional protein features in promoter representations, as similar protein embeddings signals similar function.

  2. The C3P model can perform zero-shot co-regulated gene retrieval, which is the ability to find co-regulated genes in a genome using no experimental data, suggesting the possibility of inferring the regulatory networks of entirely unstudied species.

  3. C3P models show a multi-fold improvement over leading genome language models at capturing regulatory function, and they consistently provide significant improvement over randomly initialized baselines at zero-shot co-regulated gene retrieval, unlike gLMs.

  4. The learned promoter representations from C3P can be used for nearest neighbour prediction of curated regulatory annotations with a multi-fold improvement over leading gLMs, demonstrating the predictive power of C3P-learned promoter representations for inference of curated regulatory annotations.

Abstract

Despite the increasing scale of genome language models (gLMs), their ability to decode the function of regulatory sequences remains unclear. gLM pretraining relies on sequence reconstruction, which may struggle due to the noisy, rapidly evolving nature of regulatory DNA. Self-supervised contrastive approaches provide a promising alternative. Inspired by language-image architectures like CLIP, we introduce contrastive promoter-protein pretraining (C3P). By learning to align promoters to their corresponding proteins, we leverage the rich representations of proteins learned by protein language models as supervisory signal for the learning of promoter representations. After training on 88 million bacterial promoter-protein pairs, we evaluate the predictive power of C3P-learned promoter representations for inference of curated regulatory annotations, finding multi-fold improvement over leading gLMs. We also introduce zero-shot co-regulated gene retrieval, the ability to find co-regulated genes in a genome using no experimental data. We find that compared to a randomly initialized baseline, C3P training consistently provides significant zero-shot performance gains, unlike gLMs. Scaling analysis reveals the potential for further improvement as well as the efficiency of C3P, which achieved strong performance at a fraction of the training cost of leading gLMs. In addition to demonstrating that C3P training is effective for learning representations of bacterial regulatory sequences, our strong zero-shot co-regulated gene retrieval performance suggests the possibility of decoding gene regulation for millions of bacteria from their genomes alone.

Sources

Related papers