From Scarce Functional Labels to Label-Aware Generation in Homologous Protein Families

arXiv:2507.15651 · q-bio.QM, cond-mat.dis-nn · Submitted 2025-07-21 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.

Ines: We're looking at the paper: "From Scarce Functional Labels to Label-Aware Generation in Homologous Protein Families".

Marcus: Here's the gist of the paper. Accurately annotating and controlling protein function from sequence data remains a major challenge in protein engineering, especially when functional labels are scarce within large homologous families Here, we study a two-stage light-supervision strategy for fine-grained functional annotation and label-aware sequence generation.

Ines: Problem Context The ability to generate such functionally specific artificial protein sequences is of great interest for protein design, potentially in combination with subsequent refinement by directed evolution.

Marcus: The first point already presents three major difficulties, which directly impact our ability to succeed in the second First, the limited number of labeled sequences makes it difficult for traditional supervised methods to generalize in the high-dimensional space of homologous proteins, whose size is typically exponential in the protein length.

Ines: Second, because homologous protein sequences share a highly similar three-dimensional structure, predicted folding offers limited additional information.

Paper summary: Ines: So, looking at the paper "From Scarce Functional Labels to Label-Aware Generation in Homologous Protein Families," they're focusing on overcoming the difficulty of annotating and controlling protein function when labels are scarce in large homologous families.

Marcus: The core claim is that a two-stage light-supervision strategy can successfully achieve fine-grained functional annotation and label-aware sequence generation even under those difficult conditions.

Ines: They investigate sequence representations like RBMs, one-hot encodings, and ESM2 embeddings to see which ones best predict those subtle intra-family specificities from limited supervision.

Marcus: The key result is that the family-specific RBM representation often performs better than the more general ESM2 embeddings in this context, especially when you look at how well it supports the label-specific generation pipeline.

Yuki: What this means for us in population genetics is that it suggests we can use these models to make predictions about protein function within a species, even if our direct experimental labels are sparse or phylogenetic correlations muddy the waters.

Ines: They emphasize that the quality of the annotation depends heavily on using phylogeny-aware evaluation when assessing how well these methods work across different homologous families.

Marcus: Basically, they show that high-capacity pre-trained representations don't automatically win in this fine-grained setting; you need to adapt or use a specific model structure like the family RBM for the best results.

Conclusion: Ines: So we’re wrapping up this look at "From Scarce Functional Labels to Label-Aware Generation in Homologous Protein Families." Essentially, the researchers found a way to make functional predictions when you only have a tiny bit of labeled data.

Marcus: They tackled the problem of getting specific function labels for proteins that are part of huge families, where you just don't have many examples. They used this two-stage method: first, they generate lots of extra training data by annotating everything they already have, and then they use that to create new sequences belonging to each function class.

Yuki: And the main finding is how much those few labels matter when you’re trying to design new proteins. They showed that using a specific family representation called an RBM worked better than the general protein language models like ESM2 in predicting these fine details.

Ines: That’s interesting because it suggests that for really small amounts of data, adapting a model specifically to the family structure gives you much more reliable results than just relying on a massive, general pre-trained model. It shows us that we need to be careful about how we evaluate these methods across different types of protein families.

Marcus: Exactly. And when they used only about one hundred annotated sequences, they found that those same one hundred sequences were enough for the conditional generation to actually work well. It’s a trade-off between how much you want to annotate and how reliable your generated data is going to be.

Yuki: So what does this mean for us looking at evolution? It means we can start making more informed guesses about protein function within a lineage, even when the evolutionary relationships are complicated or the labels are sparse. We can use these models as a tool for exploring sequence space that respects the known biological family structure.

Ines: Right. But they also made it clear that this success depends entirely on how good those initial annotations are, and they stressed you need to be careful when you test if a method actually works across different protein families. We’ll take another look at how these specific RBM embeddings compare in the next segment.

Ecole Normale Sup´erieure · Sorbonne Universit´e · Institut Universitaire de France

q-bio.QM, cond-mat.dis-nn

Submitted: 2025-07-21

Updated: 2026-06-22

Comments: 13 pages, 4 figures + SI

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 88/100

The gist: Accurately annotating and controlling protein function from sequence data remains a major challenge in protein engineering, especially when functional labels are scarce within large homologous

Key concepts

Homologous Families: These are groups of proteins that share a highly similar three-dimensional structure due to evolutionary relatedness. A key challenge is that the sheer number of possible sequences in these families is massive, making it hard to find functional labels for specific members.
Restricted Boltzmann Machine (RBM): An RBM is a type of machine learning model used here to create an embedding for protein sequences. It learns a representation that captures the underlying structure and relationships within the family, even when only a few labels are available. The paper found this specific embedding is most effective for predicting intra-family function.
Label-Aware Sequence Generation: This is the process of creating entirely new protein sequences that are designed to possess a specific, desired function. The method uses the inferred functional labels from the RBM to guide a generative model, allowing researchers to explore sequence space and design proteins with targeted properties.
Light-Supervision Strategy: This is a two-step approach where limited experimental data (light supervision) guides the training of a powerful generative model. The first step uses this limited data to build better internal representations (the RBM), which then enables the second, more complex step of generating high-quality, label-specific sequences.

Terminology

AUC — 40× in this paper · explained in 2 episodes
Embeddings — 25× in this paper · explained in 2 episodes
Generative Model — 11× in this paper · explained in 2 episodes
Fine-Tuning — 10× in this paper · explained in 5 episodes
Restricted Boltzmann Machine — 9× in this paper
Augmentation — 8× in this paper · explained in 2 episodes
Generator — 7× in this paper · explained in 2 episodes
Benchmark — 6× in this paper · explained in 2 episodes
Data Augmentation — 6× in this paper · explained in 10 episodes
Abundance — 5× in this paper · explained in 1 episode
Consistency — 5× in this paper · explained in 3 episodes
Latent Variables — 5× in this paper · explained in 2 episodes
Self-Consistency — 5× in this paper · explained in 3 episodes
Accuracy — 4× in this paper · explained in 1 episode
Generalization — 4× in this paper · explained in 14 episodes
Generative Models — 4× in this paper · explained in 4 episodes
Transformer — 4× in this paper · explained in 2 episodes
F1 Score — 3× in this paper · explained in 4 episodes
Generative Modeling — 3× in this paper · explained in 3 episodes
Hyperparameters — 3× in this paper · explained in 2 episodes
Response Regulator — 3× in this paper
Clustering — 2× in this paper · explained in 2 episodes
Computational Biology — 2× in this paper · explained in 2 episodes
Latent Representation — 2× in this paper · explained in 2 episodes
Ontology — 2× in this paper · explained in 2 episodes
Class Imbalance — 1× in this paper · explained in 3 episodes
Contrastive Learning — 1× in this paper · explained in 9 episodes
Core — 1× in this paper · explained in 2 episodes
Generalization Capabilities — 1× in this paper · explained in 2 episodes
Initial Data — 1× in this paper · explained in 2 episodes
Latent Representations — 1× in this paper · explained in 3 episodes
Latent Space — 1× in this paper · explained in 7 episodes
Localization — 1× in this paper · explained in 2 episodes
Overfitting — 1× in this paper · explained in 2 episodes
Phase Transition — 1× in this paper · explained in 3 episodes
Regularization — 1× in this paper · explained in 3 episodes
Restricted Boltzmann Machine (RBM) — 1× in this paper · explained in 4 episodes
Sampling Biases — 1× in this paper · explained in 2 episodes
Semi-supervised Learning — 1× in this paper · explained in 4 episodes
Transformer Architecture — 1× in this paper · explained in 4 episodes
Weight Decay — 1× in this paper · explained in 2 episodes

Summary

Problem Context

The ability to generate such functionally specific artificial protein sequences is of great interest for protein design, potentially in combination with subsequent refinement by directed evolution. The first point already presents three major difficulties, which directly impact our ability to succeed in the second First, the limited number of labeled sequences makes it difficult for traditional supervised methods to generalize in the high-dimensional space of homologous proteins, whose size is typically exponential in the protein length. Second, because homologous protein sequences share a highly similar three-dimensional structure, predicted folding offers limited additional information. Third, homologous protein families show complex phylogenetic correlations between sequences that, for example, make it difficult to split the dataset into meaningful test and training subsets, or to dissect functional similarity from evolutionary relatedness. In this setting, we face a weak labeling signal, phylogenetically correlated sequences, and hardly defined functional distinctions.

Sequence Representation Comparison

The authors compare several sequence representations for predicting intra-family specificity labels from limited supervision These representations include one-hot encodings, Restricted Boltzmann Machines (RBMs), and ESM2-based protein language model embeddings. By using train/test splits that explicitly reduce phylogenetic leakage, they show that ESM2-based representations do not systematically outperform family-specific RBM embeddings or even simple one-hot baselines in this regime.

Training Signal Construction

The pipeline involves two main stages: (i) annotating all the sequences in the input dataset (data augmentation), and (ii) generating new, artificial sequences that belong to each of the functionality classes (labelspecific generation). The data augmentation and label-specific generation pipeline is implemented as follows

(a)

The predictor models are trained using a logistic regression applied to a given embedding of the data. The RBM representation should be interpreted as a transductive, family-specific embedding, as the model is trained on the full unlabeled MSA including sequences that later appear in the test set without their labels.

Where the Gains Come From

The results show that scarce annotations can support label-aware protein design when they are accurately propagated, while also highlighting the importance of phylogeny-aware evaluation for assessing functional annotation methods within homologous families. In the context of conditional generation, a fully annotated training set yields a conditional generative model that accurately reproduces the label-specific distributions of the natural data. Furthermore, when only 103 annotated sequences are used, approximately 103 annotated sequences provide a favorable trade-off between the experimental cost of annotation and the reliability of conditional generation.

Model Performance Metrics

The performance of different methods is quantified using various metrics across several protein families. For instance, in the RR family with t1=0.4 and 100 training sequences, the RBM achieved an ROC AUC of ∼ 0.96 when compared to other methods. In contrast, when using t1=0.7 and 100 training sequences, the RBM achieved an ROC AUC of ∼ 0.834. The symmetric Kullback–Leibler (KL) divergence analysis confirms that the RBM embedding is more effective than both the one-hot representation and the pLM embedding at providing label-specific information to the generative model.

Conclusion on Representation Efficacy

The study concludes that in this fine-grained intra-family setting, tested ESM2-based representations do not systematically outperform simpler baselines. Specifically, the family-specific RBM representation performs best in several cases. The improvement in self-consistency when labels are inferred from the RBM embedding suggests an architectural alignment between the model used to infer the labels and the model used for conditional sequence generation. This establishes a practical light-supervision framework for connecting sparse functional measurements to targeted exploration of protein sequence space. The results show that fine-grained intra-family specificity prediction remains demanding, and that high-capacity pretrained representations do not automatically outperform family-adapted or simple sequence-based baselines. The annotation quality requirements on which such strategies depend are explicitly highlighted.

Supplementary Data and Methods

The paper details the construction of training and test sets using the cobalt algorithm to address phylogenetic correlations. The RBM is implemented with a bipartite graph structure where visible units are vectors of categorical variables representing aligned sequences. The label-aided RBM integrates label information into the energy function by adding an additional term. The fine-tuning of ESM2-650M is performed when the number of annotated training sequences is ≥ 1000. The symmetric KL divergence quantifies the agreement between natural and conditionally generated data distributions. The generation error analysis shows that self-consistency is substantially higher when labels are inferred from the RBM embedding, reaching AUC ≃ 0.96. The results hold true for other protein families with very different types of annotations even when t1=0.7. Detailed results are given in Tables S2a-S3c. The code that allows to reproduce the results of this paper is available at the following Github repository. The datasets used in this work are publicly available through a Zenodo repository. The code that allows to reproduce the results of this paper is available at the following Github repository. The datasets used in this work are publicly available through a Zenodo repository. The code that allows to reproduce the results of this paper is available at the following Github repository. The datasets used in this work are publicly available through a Zenodo repository. The code that allows to reproduce the results of this paper is available at the following Github repository. The datasets used in this work are publicly available through a Zenodo repository. The code that allows to reproduce the results of this paper is available at the following Github repository. The datasets used in this work are publicly available through a Zenodo repository. The code that allows to reproduce the results of this paper is available at the following Github repository. The datasets used in this work are publicly available through a Zenodo repository. The code that allows to reproduce the results of this paper is available at the following Github repository. The datasets used in this work are publicly available through a Zenodo repository. The code that allows to reproduce the results of this paper is available at the following Github repository. The datasets used in this work are publicly available through a Zenodo repository.

Improvements for AI systems

Label-aware sequence generation from scarce data: The improved system can generate artificial homologs conditioned on a chosen specificity, as shown in Sec II C, which is crucial for protein design by allowing targeted exploration of sequence space with minimal functional labels.
Phylogenetic leakage mitigation in representation learning: The system employs cobalt train/test splitting to address phylogenetic correlations, ensuring that models are evaluated on data from sequences distant from the training set to obtain an unbiased measure of generalization ability.
Transfer learning for sub-function recognition: By fine-tuning pre-trained protein language models like ESM2 using a classification head, the system can potentially align the model's latent space with distinct specificities, although this requires a moderate number of annotated sequences (at least 1000) to be effective.
Robustness against annotation noise: The label-aware RBM architecture is shown to be more consistent in downstream generation when labels are inferred from the RBM embedding compared to methods using one-hot or foundation embeddings, suggesting better architectural alignment between label inference and sequence generation.
Quantification of annotation quality requirements: The study provides a controlled assessment showing that approximately 103 annotated sequences provide a favorable trade-off between experimental cost and the reliability of conditional generation, establishing a practical threshold for functional annotation propagation.

Abstract

Accurately annotating and controlling protein function from sequence data remains a major challenge in protein engineering, especially when functional labels are scarce within large homologous families. Here, we study a two-stage light-supervision strategy for fine-grained functional annotation and label-aware sequence generation. First, we compare several sequence representations, including one-hot encodings, Restricted Boltzmann Machines (RBMs), and ESM2-based protein language model embeddings, for predicting intra-family specificity labels from limited supervision. By using train/test splits that explicitly reduce phylogenetic leakage, we show that ESM2-based representations do not systematically outperform family-specific RBM embeddings or even simple one-hot baselines in this regime. Second, we use the inferred annotations to train an annotation-aware RBM capable of generating artificial homologs conditioned on prescribed labels. Across several protein families, we quantify how the number and quality of available labels determine the reliability of conditional generation. Our results show that scarce annotations can support label-aware protein design when they are accurately propagated, while also highlighting the importance of phylogeny-aware evaluation for assessing functional annotation methods within homologous families.