Do Vision CIL Methods Transfer to Protein? A Diagnostic Benchmark of Frozen Protein Encoders

arXiv:2610.18879 · cs.LG, cs.AI, q-bio.QM · Submitted 2026-10-01 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "Do Vision CIL Methods Transfer to Protein? A Diagnostic Benchmark of Frozen Protein Encoders".

Jane: The gist The study conducts a controlled benchmark of nine representative class-incremental learning methods across five frozen protein encoders and two enzyme function datasets to determine if methods developed in…

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So, to wrap up the paper "Do Vision CIL Methods Transfer to Protein? A Diagnostic Benchmark of Frozen Protein Encoders," the core finding is that existing class-incremental learning methods don't transfer well between different frozen protein encoders.

Jane: It’s a diagnostic benchmark because they systematically tested nine different CIL methods against five sequence, structure, and sequence-structure encoders across two enzyme datasets at varying scales.

Lu: The paper shows that the failure isn't in the incremental learning strategy itself across phases; it’s fundamentally tied to the representation space created by the frozen protein encoder (<ref:2610.18879#pg3>).

Meng: So, for someone looking at this from a deployment standpoint, it means you can't just swap in a new CIL algorithm and expect it to work without understanding how that specific encoder shapes the data space first.

Lalam: The authors suggest that future protein CIL needs to be encoder-aware, meaning the method has to adjust its statistical assumptions and capacity based on the geometry of the representation it receives (<ref:2610.18879#pg3>).

Tom: That’s it—the message is that we need methods that adapt their statistical requirements to the specific geometry they operate on, rather than assuming a universal one holds true for all protein representations.

Jane: It really points toward treating the representation as a central component of continual learning design, instead of just a passive feature extractor (<ref:2610.18879#pg3>).

Conclusion: Tom: So we're wrapping up this look at "Do Vision CIL Methods Transfer to Protein? A Diagnostic Benchmark of Frozen Protein Encoders."

Jane: It sounds like they tested if class incremental learning methods that work in vision can just be plugged into protein science without breaking.

Tom: Right. The authors set up a controlled test with nine different learning methods and five frozen protein encoders across two enzyme datasets at different sizes.

Lu: What’s important is that they kept the encoders frozen, so the representation space itself didn't change during the learning process for any method.

Meng: I’m curious, what was the main takeaway from those tests about transferability? Did anything actually work consistently?

Jane: Well, they found no reliable transfer across different protein encoders. No matter which method you used, it didn't reliably move from one encoder to another.

Tom: That instability continued even when they tried different ways of structuring the learning phases or changing the way they built the classifier head.

Lalam: The diagnostics pointed to three main issues causing this failure: weak stability signals, big differences in embedding geometry, and covariance assumptions that don't fit well with both data size and encoder type.

Meng: So it means a method isn't just failing because of the learning schedule; it’s failing because the underlying protein representation is too different for that method’s math.

Jane: Exactly. The paper suggests we need methods that are aware of the geometry they are operating in, not methods that assume everything is uniform across all representations.

Tom: It shifts the focus from just tweaking learning algorithms to actually designing those learning algorithms to match the protein's specific structure.

Lu: It’s a clear call for encoder-aware design, where you have to adapt your statistical assumptions based on what that frozen representation geometry actually looks like.

Jane: So, reliable protein AI won't be about finding one perfect learning trick; it’ll be about building systems that understand the specific shape of the data they are seeing.

Tom: And that leads us right into how we can start making these methods actually aware of those geometries in our next discussion on encoder-aware models.

Ziyuan Gao

University of Cambridge

cs.LG, cs.AI, q-bio.QM

Submitted: 2026-10-01

Updated: 2026-10-01

Comments: Accepted at NeurIPS 2026 Workshop on Continual Learning for Foundation Models and Agents (CL4FM Agents)

Code: https://github.com/zygao930/ProteinCIL

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 81/100

The gist: The gist The study conducts a controlled benchmark of nine representative class-incremental learning methods across five frozen protein encoders and two enzyme function datasets to determine if

Key concepts

Class-Incremental Learning (CIL): A learning framework where a model learns new classes sequentially without forgetting previously learned ones. It is used here to see if methods from vision transfer to the domain of protein function annotation.
Frozen Protein Encoders: These are pre-trained models that process protein sequences or structures but whose internal parameters are kept fixed during the learning process. The study tested nine CIL methods against five different types of these frozen encoders to see if they could adapt.
Embedding Geometry: This refers to the shape and structure of the data points (protein representations) in a high-dimensional space. The paper found that different protein encoders create vastly different geometries, which causes CIL methods built on fixed geometric assumptions to fail.
Covariance Capacity: This is a statistical assumption about how related classes are distributed within the representation space. The study showed that the required capacity for this assumption changes significantly based on both the size of the dataset and which protein encoder is being used.

Terminology

Benchmark — 1× in this paper · explained in 2 episodes

Summary

The gist The study conducts a controlled benchmark of nine representative class-incremental learning methods across five frozen protein encoders and two enzyme function datasets to determine if methods developed in vision transfer reliably to frozen protein representations.

Benchmark Setup

The researchers established a controlled benchmark involving nine CIL methods, five frozen protein encoders, two enzyme function datasets, and four phase construction protocols The evaluation protocol involves mapping every protein through a frozen backbone encoder whose parameters are never updated The five frozen protein encoders span sequence language models, a structurebased graph network, and a sequence–structure fusion model These encoders differ substantially in embedding dimension and geometry, as illustrated in Fig. 1. The method-specific hyperparameters are fixed across all encoder-dataset configurations, directly testing the plug-and-play transfer setting rather than encoder-specific tuning.

Key Findings on Transferability

The primary finding is that no CIL method transfers reliably across encoders This instability persists across alternative phase constructions, indicating that the dominant variation is encoder-dependent rather than phase-dependent Diagnostic analyses trace this instability to three recurring causes: weak gradient-based stability signals when only the classifier head is trainable, large differences in encoder-induced geometry, and covariance capacity requirements that change with both dataset scale and encoder. This suggests that the representation itself should be treated as a central component of continual-learning design rather than a passive feature extractor

Diagnostic Analysis of Failures

The instability is rooted in mismatches between the fixed assumptions of existing CIL methods and the representations they operate on The causes identified are detailed as follows:

** Weak gradient-based stability signals when only the classifier head is trainable, inter-phase conflict cosines are near zero and Fisher information is highly concentrated in a small fraction of head parameters This leads to regularization methods like EWC collapsing toward the naive baseline and RER toward plain Replay. 3.3 Diagnostic Analysis**

** Variation in embedding geometry, within-class covariance condition number spans roughly five orders of magnitude across encoders, while intrinsic dimensionality and class separability also vary substantially with both encoder and dataset These differences show that the representations induce markedly different class geometries, making a single fixed geometric assumption unlikely to transfer uniformly across encoders. 3.3 Diagnostic Analysis**

** Covariance capacity requirements that change with both dataset scale and representation geometry This is exemplified by the failure of prototype methods, where a single shared covariance can be too coarse when different regions of the representation space exhibit substantially different local structure. 6**

Covariance Capacity and Geometry

Table 4 demonstrates that failures under a single shared covariance are structured rather than uniformly distributed across classes Errors concentrate particularly on transferases and hydrolases, two large and functionally distinct EC families that are repeatedly mapped into one another under the shared metric The severity of this confusion varies sharply across encoders, suggesting that the appropriate covariance capacity depends jointly on dataset scale and encoder-induced geometry For instance, full shared covariance is best across all five encoders on Enzyme Reaction, while per-class covariance is best for four of five encoders on EC Number

Conclusion and Design Principle

The results suggest that the central difficulty is a mismatch between the assumptions built into existing CIL methods and the representation on which they operate Reliable protein CIL will require encoder-aware methods that adapt their statistical assumptions and capacity to the geometry of the representation on which they operate The benchmark provides a design criterion: reliable protein CIL should respond to encoder geometry instead of assuming it away. Future methods should adapt their statistical capacity to the geometry of the representation they receive rather than being fixed in advance

Robustness Across Protocols

The robustness check across phase construction protocols showed that while method rankings vary substantially across encoders, the performance patterns remain stable under alternative phase constructions For example, RanPAC remains the strongest method on EC Number, while FeCAM reaches 44–50% accuracy on EC Number across all protocols This suggests that the encoder-dependent variability observed in the main experiments is not specific to a particular phase schedule The failures are therefore representation-dependent rather than driven by phase construction. 3.2 Robustness Across Phase Construction Protocols

Encoder-Specific Behavior

Per-encoder breakdowns show substantial differences between encoders, with some methods performing well on one encoder and poorly on another For example, RanPAC is strong on most sequence and sequence–structure encoders but deteriorates sharply on GearNet-Edge FeCAM preserves old classes extremely well yet loses plasticity as its shared-covariance assumption becomes too restrictive This supports the main finding that phase construction has a smaller effect than the choice of frozen protein representation. 3.3 Diagnostic Analysis

Dataset Specificity

The cross-family confusion patterns consistently involve transferases and hydrolases across both datasets, indicating these families remain difficult to separate under a single shared covariance However, the magnitude of these confusions varies substantially across encoders and datasets This variation further supports the conclusion that the effectiveness of a shared covariance assumption depends strongly on the geometry induced by the underlying protein representation 6

Experimental Details

The study utilized five frozen encoders with different geometric properties, including ESM2 models, GearNet-Edge, and SaProt The datasets used were Enzyme Reaction (348 classes) and EC Number (4788 classes). All reported results are averaged over three random seeds unless otherwise specified The evaluation is limited to enzyme function classification under the EC hierarchy, and whether the same failure modes extend to other protein tasks remains open. 5

Data and Encoder Details

Table 6 summarizes the two datasets and the preprocessing steps used in our benchmark Table 7 lists the five frozen encoders and their extraction settings, detailing input dimension, pooling strategy, and normalization methods. The baseline CIL methods include Naive fine-tuning and Joint training as lower and offline reference points All hyperparameters are fixed across encoder-dataset configurations, with no setting-specific tuning 3

Data and Encoder Details

Table 7 lists the five frozen encoders and their extraction settings, detailing input dimension, pooling strategy, and normalization methods.

Improvements for AI systems

Bold header: Encoder-aware CIL design based on representation geometry

Develop continual learning methods where statistical capacity adapts to representation geometry rather than being fixed in advance, specifically by adapting covariance capacity based on whether the encoder is a sequence-based model or a structure-based graph network. This allows the system to choose between full shared covariance and per-class covariance dynamically, as shown by how the appropriate covariance capacity depends jointly on dataset scale and encoder geometry.

Bold header: Geometric diagnostic framework for representation stability

Implement a diagnostic layer that measures key geometric properties—such as embedding dimension, intrinsic rank r95, effective rank, within-class covariance condition number, and between/within class separability—for frozen protein representations. This allows researchers to identify the root causes of failure by correlating instability with specific geometric variations across different encoders.

Bold header: Representation-dependent forgetting mitigation strategies

Design incremental learning strategies that are tailored to the observed representation-dependent failure modes. For instance, if a representation induces markedly different class geometries, methods should be modified to address the resulting inter-phase conflict cosines near zero or manage the resulting high variance in embedding space structure.

Abstract

Protein function annotation operates over an expanding label space, making class-incremental learning (CIL) a natural framework for incorporating newly discovered functions without full retraining. Yet existing CIL methods have been developed largely in vision and have not been systematically evaluated over frozen protein representations. We conduct a controlled benchmark of nine representative CIL methods across five sequence-, structure-, and sequence--structure-based protein encoders and two enzyme function datasets at different scales, while keeping the encoders frozen and evaluation settings consistent across representations. We find that no method transfers reliably across encoders, and that these failures persist across alternative phase constructions. Diagnostic analyses reveal that this instability arises from weak gradient-based stability signals, substantial heterogeneity in embedding geometry, and covariance assumptions whose appropriate capacity changes with both dataset scale and encoder. These findings suggest that the representation itself should be treated as a central component of continual-learning design rather than a passive feature extractor. More generally, reliable protein CIL will require encoder-aware methods that adapt their statistical assumptions and capacity to the geometry of the representation on which they operate. Code is available at https://github.com/zygao930/ProteinCIL.