Ablating Design Choices for Protein Ensemble Generators Across Structural Regimes

arXiv:2611.05055 · q-bio.BM, cs.LG · Submitted 2026-11-03 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.

Ines: We're looking at the paper: "Ablating Design Choices for Protein Ensemble Generators Across Structural Regimes".

Marcus: Here's the gist of the paper. Generative models of protein conformational ensembles inherit much of their design from structure prediction for folded proteins, but ordered and disordered proteins occupy distinct structural regimes that require different design choices <ref:2611.05055#pg2>.

Ines: Using a minimal Cα flow-matching model, the study investigates how design choices hold up across these structural regimes by evaluating variants against experimental SAXS and NMR chemical-shift data partitioned into ordered, disordered, multi-domain and transitional proteins <ref:2611.05055#pg2>.

Marcus: Evaluation Metrics and Partitioning The evaluation focuses on agreement with experimental data using PeptoneBench, which scores generated ensembles against SAXS (constraining global dimensions and shape) and CS (constraining local backbone geometry) <ref:2611.05055#pg3>.

Ines: The benchmark partitions proteins into four cells by G-score, defining multi-domain proteins as those containing at least one block of each kind, ordered proteins as those with mean G 0.7 <ref:2611.05055#pg3>.

Paper summary: Marcus: So to wrap up "Ablating Design Choices for Protein Ensemble Generators Across Structural Regimes," the paper shows that protein ensemble generators need to be aware of whether they are dealing with an ordered or a disordered protein because the best design choices depend on the structural regime.

Ines: They tested several architectural components and training sources, and they found that mixture of experts routing, static pair representation, and a disorder probe don't actually change performance much across these regimes.

Yuki: And they also saw that smaller models struggle with ordered proteins but not disordered ones, which gives us a clue about what kind of structural information the model is learning for each type.

Marcus: They concluded that the training data source is the key factor here, especially since improving it seems to be the only way to get better results for disordered proteins across both local and global scales.

Ines: It means that if we want better protein ensemble generators, we need training data that actually captures both local details and global shapes for all these different structural regimes.

Yuki: So the implication is that future progress depends on training sources being accurate enough to handle the diversity of protein structures they encounter in nature.

Conclusion: Ines: So we're wrapping up this look at "Ablating Design Choices for Protein Ensemble Generators Across Structural Regimes." Basically, they're showing that how you build these models matters depending on whether the protein is ordered or disordered.

Marcus: Right. They tested all these different design choices—like mixture of experts routing and a disorder probe—and found that they didn't really make a big difference in performance across those two types of proteins.

Yuki: That suggests the architecture itself isn't the biggest hurdle here, is it? It points more toward what we feed into it.

Ines: Exactly. The authors are pointing to the training data as the main constraint at this scale, because weakening a design choice barely hurts performance anyway. They found that using different training sources trades local accuracy for global accuracy in disordered proteins.

Marcus: So what they’re suggesting is that if you want better models for disordered proteins, you need training data that is accurate on both the local level and the global shape level.

Yuki: That makes sense from a population perspective. The sample we use has to represent all those structural possibilities accurately to support any big claim about evolution across different regimes.

Ines: And they did run some comparisons, like Kabsch alignment, and it showed that even when they reweighted the ensembles towards what the measurement actually looks like, the model still struggled with some of those ordered proteins.

Marcus: That's a key detail. It means that having conformers is one thing, but making sure those conformers are weighted correctly for a specific experiment is another whole layer of difficulty.

Yuki: And it’s interesting because the scaling exponents they got from their Flory analysis matched what we expect for ordered versus disordered chains, which confirms the model understands the basic physics of each state.

Ines: So, it looks like we need to focus on making those training sources more versatile across all structural regimes before we can really say we've cracked this problem.

Eoin Quinn, Oliver Bent

InstaDeep

q-bio.BM, cs.LG

Submitted: 2026-11-03

Updated: 2026-11-03

Comments: 13 pages, 2 figures

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 88/100

The gist: Generative models of protein conformational ensembles inherit much of their design from structure prediction for folded proteins…

Key concepts

Structural Regimes
Proteins are categorized into distinct structural regimes: ordered (well-folded), disordered (flexible), multi-domain, and transitional. These categories require different design strategies for generating accurate conformational ensembles because their underlying physical constraints are fundamentally different.
PeptoneBench Evaluation
This benchmark measures generated protein ensembles against experimental data using two metrics: SAXS, which constrains the overall shape and size of the protein, and CS (chemical shift), which constrains local backbone geometry. This allows researchers to assess how well the model captures both global and local structural features.
Training Data Trade-offs
The study investigates using different training data sources, such as coarse-grained simulations versus fragment-assembled ensembles. Using a simulation source improves chemical shift agreement but hurts SAXS accuracy for disordered proteins, showing that the choice of training data significantly impacts performance across structural regimes.

Terminology

Summary

Generative models of protein conformational ensembles inherit much of their design from structure prediction for folded proteins, but ordered and disordered proteins occupy distinct structural regimes that require different design choices <ref:2611. p. 0

Where the paper says it

com Abstract Generative models of protein conformational ensembles inherit much of their design from structure prediction for folded proteins. Ordered and disordered proteins, however, occupy distinct structural regimes, and we investigate how design choices hold up across them.

05055#pg2>. Using a minimal Cα flow-matching model, the study investigates how design choices hold up across these structural regimes by evaluating variants against experimental SAXS and NMR chemical-shift data partitioned into ordered, disordered, multi-domain and transitional proteins <ref:2611. p. 0
Where the paper says it

Ordered and disordered proteins, however, occupy distinct structural regimes, and we investigate how design choices hold up across them. Using a minimal Cα flow-matching model, we ablate choices of architecture, sequence conditioning, rigid-body alignment and training source, and evaluate each variant against experimental SAXS and NMR chemical-shift data partitioned into ordered, disordered, multi-domain and transitional proteins.

05055#pg2>.

Evaluation Metrics and Partitioning

The evaluation focuses on agreement with experimental data using PeptoneBench, which scores generated ensembles against SAXS (constraining global dimensions and shape) and CS (constraining local backbone geometry) <ref:2611.05055#pg3>. The benchmark partitions proteins into four cells by G-score, defining multi-domain proteins as those containing at least one block of each kind, ordered proteins as those with mean G 0. p. 1

Where the paper says it

7 (flexible), and assign all proteins as follows (cell counts and the scored cohort in Appendix E): • multi-domain: contains at least one block of each kind. • ordered / disordered: mean G < 0.

7 <ref:2611. p. 0
Where the paper says it

Instead the probes available for disordered proteins measure ensemble-averaged observables, principally small-angle X-ray scattering (SAXS) [11] and NMR chemical shifts (CS) [12]. 1 arXiv:2611.

05055#pg3>.

Ablation of Architectural Components

The ablation study tested several design choices on the base model, revealing that mixture-of-experts routing, a static pair representation, and a disorder probe have no detectable effect on performance <ref:2611.05055#pg2>. Furthermore, replacing coarse-grained simulations of disordered proteins with fragment-assembled ensembles improves agreement with chemical shifts at a cost to SAXS, while an equal mixture keeps most of the gain at a smaller cost, pointing to the training data rather than the architecture as the main constraint <ref:2611. p. 0

Where the paper says it

Replacing coarse-grained simulations of disordered proteins with fragment-assembled ensembles improves agreement with chemical shifts at a cost to SAXS, while an equal mixture of the two keeps most of the gain at a smaller cost. This points to the training data, rather than the architecture, as the main constraint.

05055#pg2>.

Impact of Specific Design Choices Across Regimes

The study found that a smaller protein language model performs worse on ordered proteins but not on disordered ones <ref:2611.05055#pg4>. Kabsch alignment in the loss degrades agreement with SAXS in every cell, from +26% on ordered proteins to +99% on disordered ones, and this degradation persists after reweighting, suggesting ensembles contain too few conformers compatible with the measurement <ref:2611. p. 4

Where the paper says it

It is worse even on ordered proteins, where the gradient stabilisation that motivates it should help most, and the largest effect, on disordered proteins, is consistent with the lever-arm mechanism of Sec. The degradation persists after reweighting (Appendix G), suggesting that the ensembles contain too few conformers compatible with the measurement.

05055#pg5>.

Conclusion on Training Data Importance

The ablations suggest that at this scale, the model is limited by its training data rather than its architecture, as adding architectural components has no detectable effect and even deliberately weakening the model costs little <ref:2611.05055#pg7>. The source of the disordered training data trades local against global accuracy, and an equal mixture of the two sources softens this trade-off <ref:2611. p. 5

Where the paper says it

Of the two premised on a stable native fold, the size of the protein language model matters only for folded proteins, an effect the chemical-shift AUC barely registers, while alignment in the loss degrades agreement with SAXS in every regime. The source of the disordered training data trades local against global accuracy, a trade-off that an equal mixture of the two sources softens.

05055#pg7>. Only the training source improves either observable, suggesting future progress requires training sources accurate at both local and global scales across structural regimes <ref:2611. p. 5
Where the paper says it

The source of the disordered training data trades local against global accuracy, a trade-off that an equal mixture of the two sources softens. Only the training source improves either observable, so further progress will need training sources that are accurate at both local and global scales and across structural regimes.

05055#pg7>.

Limitations of the Study

Limitations include using coarse-grained simulation data and predicted structures as approximations, and the ordered data being restricted to confidently predicted structures with pLDDT ≥ 90 <ref:2611.05055#pg2>. The conclusions are drawn at a single scale, with ordered, disordered and multi-domain data in fixed equal proportion <ref:2611. p. 5

Where the paper says it

The ordered data are also restricted to confidently predicted structures (pLDDT ≥ 90), which leaves partially ordered regions underrepresented. Our conclusions about architecture are drawn at a single scale, with ordered, disordered and multi-domain data in fixed equal proportion.

05055#pg7>. Furthermore, the evaluation rests on two observables, and the small non-ordered SAXS cells leave several comparisons unresolved <ref:2611. p. 5
Where the paper says it

Finally, the evaluation rests on two observables. In particular, SAXS on ordered proteins tests compactness rather than fold correctness, and with three seeds per variant, the small non-ordered SAXS cells (34–42 proteins) leave several comparisons unresolved.

05055#pg7>.

Statistical Comparison of Variants

The statistical comparison involves combining a paired comparison over proteins with a Welch comparison over training seeds to determine the effect of variants on performance <ref:2611.05055#pg13>. The geometric mean of per-protein ratios is reported as a percentage change, and intervals containing zero but extending beyond ±5% are considered unresolved <ref:2611. p. 2

Where the paper says it

We compare each variant with the base model protein by protein. For each protein we take the ratio of the variant’s RMSE to the base model’s, each averaged geometrically over training seeds, and summarise a cell by the geometric mean of these ratios, reported as a percentage change.

05055#pg13>.

G Scores After Reweighting

After reweighting the ensembles towards the measurement, the base model becomes competitive with baselines on SAXS in AUC and in every cell except the transitional one. Kabsch remains worse in every SAXS cell, suggesting that its ensembles contain too few conformers compatible with the measurement for reweighting to recover. p. 4

Where the paper says it

It is worse even on ordered proteins, where the gradient stabilisation that motivates it should help most, and the largest effect, on disordered proteins, is consistent with the lever-arm mechanism of Sec. The degradation persists after reweighting (Appendix G), suggesting that the ensembles contain too few conformers compatible with the measurement.

The SAXS degradation of ESM2-8M on ordered proteins disappears after reweighting, consistent with suitable conformers being present but under-weighted. p. 4
Where the paper says it

It is worse even on ordered proteins, where the gradient stabilisation that motivates it should help most, and the largest effect, on disordered proteins, is consistent with the lever-arm mechanism of Sec. The degradation persists after reweighting (Appendix G), suggesting that the ensembles contain too few conformers compatible with the measurement.

Flory Scaling Exponents

The Flory scaling exponents follow the expected ordering from 0.34 for ordered to 0. p. 0

Where the paper says it

com Abstract Generative models of protein conformational ensembles inherit much of their design from structure prediction for folded proteins. Ordered and disordered proteins, however, occupy distinct structural regimes, and we investigate how design choices hold up across them.

54 for disordered proteins across all three baselines <ref:2611. p. 3
Where the paper says it

The base model is a reasonable reference (Table 1). It is competitive with the three baselines on SAXS, mainly through disordered proteins, but trails IDPFold2 on chemical shifts in every cell, and 4

05055#pg13>. This consistency check confirms whether a model reproduces the chain statistics characteristic of each regime <ref:2611. p. 2
Where the paper says it

We compute Rg per conformer from the Cα coordinates, weighted by residue mass and averaged over the ensemble, and fit ν as the least-squares slope of log Rg against log N across the proteins of a cell, with a standard error from a bootstrap over proteins. Unlike the benchmark scores, ν does not depend on a forward model or experimental data, and so provides a consistency check on whether a model reproduces the chain statistics characteristic of each regime (Sec.

05055#pg4>.

Training Source Trade-offs

Replacing CALVADOS ensembles with fragment-assembled ones improves chemical shifts by 11–16% outside the ordered cell, but worsens SAXS on disordered proteins by 62% and raises their Flory exponent to 0.61 <ref:2611. p. 0

Where the paper says it

Instead the probes available for disordered proteins measure ensemble-averaged observables, principally small-angle X-ray scattering (SAXS) [11] and NMR chemical shifts (CS) [12]. 1 arXiv:2611.

05055#pg5>. The Fragment 50% variant keeps most of the chemical-shift gain at a fraction of the SAXS cost, with the SAXS AUC only slightly worse <ref:2611. p. 4
Where the paper says it

Replacing the CALVADOS ensembles with fragment-assembled ones improves chemical shifts by 11–16% outside the ordered cell, bringing the chemical-shift AUC level with IDPFold2’s, but worsens SAXS on disordered proteins by 62% and raises their Flory exponent to 0. Fragment 50% keeps most of the chemical-shift gain (−9% against −11% in AUC) at a fraction of the SAXS cost, with the SAXS AUC only slightly worse.

05055#pg5>. The training source is identified as a key factor in performance across structural regimes <ref:2611. p. 0
Where the paper says it

Ordered and disordered proteins, however, occupy distinct structural regimes, and we investigate how design choices hold up across them. Using a minimal Cα flow-matching model, we ablate choices of architecture, sequence conditioning, rigid-body alignment and training source, and evaluate each variant against experimental SAXS and NMR chemical-shift data partitioned into ordered, disordered, multi-domain and transitional proteins.

05055#pg7>.

Model Size Effects

Reducing the size of the protein language model degrades performance on ordered and multi-domain proteins but not on disordered ones <ref:2611.05055#pg4>. With ESM2-8M in place of ESM2-650M, ordered proteins are worse on both observables, while disordered proteins show no detectable change <ref:2611. p. 4

Where the paper says it

Reducing the size of the protein language model degrades performance on ordered and multi-domain proteins but not on disordered ones (Figure 1). With ESM2-8M in place of ESM2-650M, ordered proteins are worse on both observables (SAXS +24%, chemical shifts +12%), as are multi-domain proteins on SAXS (+26%), while disordered proteins show no detectable change, in line with the coevolutionary argument of Sec.

05055#pg4>. The small model is a third the size of the base model and shows no detectable effect <ref:2611. p. 5
Where the paper says it

Even deliberately weakening the model changes little. The small model is a third the size of the base model and shows no detectable effect, while removing RoPE costs only a few percent, mostly on SAXS (AUC +8%).

05055#pg7>.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Training data selection can be optimized by developing training sources that are accurate at both local and global scales across structural regimes. This addresses a key finding that the source of the disordered training data trades local against global accuracy, a trade-off that an equal mixture of the two sources softens.

  2. The model architecture should be adapted based on the target structural regime. For example, replacing ESM2 with ESM2-8M can improve performance on ordered and multi-domain proteins while showing no detectable change on disordered ones, as stated in Reducing the size of the protein language model degrades performance on ordered and multi-domain proteins but not on disordered ones.

  3. The choice of architectural components should be regime dependent to maximize utility. For instance, Loss-side Kabsch alignment degrades SAXS in every cell, from +26% on ordered proteins to +99% on disordered ones, suggesting that this component is detrimental when the ensemble lacks conformers compatible with the measurement in specific regimes.

  4. The model size should be scaled appropriately for the target protein type. Reducing the size of the protein language model degrades performance on ordered and multi-domain proteins but not on disordered ones, indicating that larger models are beneficial for capturing features relevant to folded structures without negatively impacting disordered regions.

Abstract

Generative models of protein conformational ensembles inherit much of their design from structure prediction for folded proteins. Ordered and disordered proteins, however, occupy distinct structural regimes, and we investigate how design choices hold up across them. Using a minimal C α flow-matching model, we ablate choices of architecture, sequence conditioning, rigid-body alignment and training source, and evaluate each variant against experimental SAXS and NMR chemical-shift data partitioned into ordered, disordered, multi-domain and transitional proteins. Mixture-of-experts routing, a static pair representation and a disorder probe have no detectable effect, a smaller protein language model performs worse on ordered proteins but not on disordered ones, and alignment in the loss degrades agreement with SAXS. Replacing coarse-grained simulations of disordered proteins with fragment-assembled ensembles improves agreement with chemical shifts at a cost to SAXS, while an equal mixture of the two keeps most of the gain at a smaller cost. This points to the training data, rather than the architecture, as the main constraint.

Sources

Related papers