Ablating Design Choices for Protein Ensemble Generators Across Structural Regimes
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.
Ines: We're looking at the paper: "Ablating Design Choices for Protein Ensemble Generators Across Structural Regimes".
Marcus: Here's the gist of the paper. Generative models of protein conformational ensembles inherit much of their design from structure prediction for folded proteins, but ordered and disordered proteins occupy distinct structural regimes that require different design choices <ref:2611.05055#pg2>.
Ines: Using a minimal Cα flow-matching model, the study investigates how design choices hold up across these structural regimes by evaluating variants against experimental SAXS and NMR chemical-shift data partitioned into ordered, disordered, multi-domain and transitional proteins <ref:2611.05055#pg2>.
Marcus: Evaluation Metrics and Partitioning The evaluation focuses on agreement with experimental data using PeptoneBench, which scores generated ensembles against SAXS (constraining global dimensions and shape) and CS (constraining local backbone geometry) <ref:2611.05055#pg3>.
Ines: The benchmark partitions proteins into four cells by G-score, defining multi-domain proteins as those containing at least one block of each kind, ordered proteins as those with mean G 0.7 <ref:2611.05055#pg3>.
Paper summary: Marcus: So to wrap up "Ablating Design Choices for Protein Ensemble Generators Across Structural Regimes," the paper shows that protein ensemble generators need to be aware of whether they are dealing with an ordered or a disordered protein because the best design choices depend on the structural regime.
Ines: They tested several architectural components and training sources, and they found that mixture of experts routing, static pair representation, and a disorder probe don't actually change performance much across these regimes.
Yuki: And they also saw that smaller models struggle with ordered proteins but not disordered ones, which gives us a clue about what kind of structural information the model is learning for each type.
Marcus: They concluded that the training data source is the key factor here, especially since improving it seems to be the only way to get better results for disordered proteins across both local and global scales.
Ines: It means that if we want better protein ensemble generators, we need training data that actually captures both local details and global shapes for all these different structural regimes.
Yuki: So the implication is that future progress depends on training sources being accurate enough to handle the diversity of protein structures they encounter in nature.
Conclusion: Ines: So we're wrapping up this look at "Ablating Design Choices for Protein Ensemble Generators Across Structural Regimes." Basically, they're showing that how you build these models matters depending on whether the protein is ordered or disordered.
Marcus: Right. They tested all these different design choices—like mixture of experts routing and a disorder probe—and found that they didn't really make a big difference in performance across those two types of proteins.
Yuki: That suggests the architecture itself isn't the biggest hurdle here, is it? It points more toward what we feed into it.
Ines: Exactly. The authors are pointing to the training data as the main constraint at this scale, because weakening a design choice barely hurts performance anyway. They found that using different training sources trades local accuracy for global accuracy in disordered proteins.
Marcus: So what they’re suggesting is that if you want better models for disordered proteins, you need training data that is accurate on both the local level and the global shape level.
Yuki: That makes sense from a population perspective. The sample we use has to represent all those structural possibilities accurately to support any big claim about evolution across different regimes.
Ines: And they did run some comparisons, like Kabsch alignment, and it showed that even when they reweighted the ensembles towards what the measurement actually looks like, the model still struggled with some of those ordered proteins.
Marcus: That's a key detail. It means that having conformers is one thing, but making sure those conformers are weighted correctly for a specific experiment is another whole layer of difficulty.
Yuki: And it’s interesting because the scaling exponents they got from their Flory analysis matched what we expect for ordered versus disordered chains, which confirms the model understands the basic physics of each state.
Ines: So, it looks like we need to focus on making those training sources more versatile across all structural regimes before we can really say we've cracked this problem.
Eoin Quinn, Oliver Bent
InstaDeep
q-bio.BM, cs.LG
Submitted: 2026-11-03
Updated: 2026-11-03
Comments: 13 pages, 2 figures
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 88/100
The gist: Generative models of protein conformational ensembles inherit much of their design from structure prediction for folded proteins…
Key concepts
- Structural Regimes
- Proteins are categorized into distinct structural regimes: ordered (well-folded), disordered (flexible), multi-domain, and transitional. These categories require different design strategies for generating accurate conformational ensembles because their underlying physical constraints are fundamentally different.
- PeptoneBench Evaluation
- This benchmark measures generated protein ensembles against experimental data using two metrics: SAXS, which constrains the overall shape and size of the protein, and CS (chemical shift), which constrains local backbone geometry. This allows researchers to assess how well the model captures both global and local structural features.
- Training Data Trade-offs
- The study investigates using different training data sources, such as coarse-grained simulations versus fragment-assembled ensembles. Using a simulation source improves chemical shift agreement but hurts SAXS accuracy for disordered proteins, showing that the choice of training data significantly impacts performance across structural regimes.
Terminology
Summary
Generative models of protein conformational ensembles inherit much of their design from structure prediction for folded proteins, but ordered and disordered proteins occupy distinct structural regimes that require different design choices <ref:2611. p. 0 com
Abstract
Generative models of protein conformational ensembles inherit much of their
design from structure prediction for folded proteins. Ordered and disordered
proteins, however, occupy distinct structural regimes, and we investigate how
design choices hold up across them. Ordered and disordered
proteins, however, occupy distinct structural regimes, and we investigate how
design choices hold up across them. Using a minimal Cα flow-matching model,
we ablate choices of architecture, sequence conditioning, rigid-body alignment
and training source, and evaluate each variant against experimental SAXS and
NMR chemical-shift data partitioned into ordered, disordered, multi-domain and
transitional proteins.Where the paper says it
Where the paper says it
Evaluation Metrics and Partitioning
The evaluation focuses on agreement with experimental data using PeptoneBench, which scores generated ensembles against SAXS (constraining global dimensions and shape) and CS (constraining local backbone geometry) <ref:2611.05055#pg3>. The benchmark partitions proteins into four cells by G-score, defining multi-domain proteins as those containing at least one block of each kind, ordered proteins as those with mean G 0. p. 1 7 (flexible), and assign all proteins as follows (cell counts and the scored cohort in
Appendix E):
• multi-domain: contains at least one block of each kind. • ordered / disordered: mean G < 0. Instead the probes available for disordered proteins measure ensemble-averaged observables,
principally small-angle X-ray scattering (SAXS) [11] and NMR chemical shifts (CS) [12]. 1
arXiv:2611.Where the paper says it
Where the paper says it
Ablation of Architectural Components
The ablation study tested several design choices on the base model, revealing that mixture-of-experts routing, a static pair representation, and a disorder probe have no detectable effect on performance <ref:2611.05055#pg2>. Furthermore, replacing coarse-grained simulations of disordered proteins with fragment-assembled ensembles improves agreement with chemical shifts at a cost to SAXS, while an equal mixture keeps most of the gain at a smaller cost, pointing to the training data rather than the architecture as the main constraint <ref:2611. p. 0 Replacing coarse-grained simulations
of disordered proteins with fragment-assembled ensembles improves agreement
with chemical shifts at a cost to SAXS, while an equal mixture of the two keeps
most of the gain at a smaller cost. This points to the training data, rather than the
architecture, as the main constraint.Where the paper says it
Impact of Specific Design Choices Across Regimes
The study found that a smaller protein language model performs worse on ordered proteins but not on disordered ones <ref:2611.05055#pg4>. Kabsch alignment in the loss degrades agreement with SAXS in every cell, from +26% on ordered proteins to +99% on disordered ones, and this degradation persists after reweighting, suggesting ensembles contain too few conformers compatible with the measurement <ref:2611. p. 4 It is worse even on ordered proteins, where
the gradient stabilisation that motivates it should help most, and the largest effect, on disordered
proteins, is consistent with the lever-arm mechanism of Sec. The degradation persists after
reweighting (Appendix G), suggesting that the ensembles contain too few conformers compatible
with the measurement.Where the paper says it
Conclusion on Training Data Importance
The ablations suggest that at this scale, the model is limited by its training data rather than its architecture, as adding architectural components has no detectable effect and even deliberately weakening the model costs little <ref:2611.05055#pg7>. The source of the disordered training data trades local against global accuracy, and an equal mixture of the two sources softens this trade-off <ref:2611. p. 5 Of the two premised on a stable native fold,
the size of the protein language model matters only for folded proteins, an effect the chemical-shift
AUC barely registers, while alignment in the loss degrades agreement with SAXS in every regime. The source of the disordered training data trades local against global accuracy, a trade-off that an
equal mixture of the two sources softens. The source of the disordered training data trades local against global accuracy, a trade-off that an
equal mixture of the two sources softens. Only the training source improves either observable, so
further progress will need training sources that are accurate at both local and global scales and across
structural regimes.Where the paper says it
Where the paper says it
Limitations of the Study
Limitations include using coarse-grained simulation data and predicted structures as approximations, and the ordered data being restricted to confidently predicted structures with pLDDT ≥ 90 <ref:2611.05055#pg2>. The conclusions are drawn at a single scale, with ordered, disordered and multi-domain data in fixed equal proportion <ref:2611. p. 5 The ordered data are also restricted to confidently
predicted structures (pLDDT ≥ 90), which leaves partially ordered regions underrepresented. Our
conclusions about architecture are drawn at a single scale, with ordered, disordered and multi-domain
data in fixed equal proportion. Finally, the evaluation rests on two observables. In particular, SAXS
on ordered proteins tests compactness rather than fold correctness, and with three seeds per variant,
the small non-ordered SAXS cells (34–42 proteins) leave several comparisons unresolved.Where the paper says it
Where the paper says it
Statistical Comparison of Variants
The statistical comparison involves combining a paired comparison over proteins with a Welch comparison over training seeds to determine the effect of variants on performance <ref:2611.05055#pg13>. The geometric mean of per-protein ratios is reported as a percentage change, and intervals containing zero but extending beyond ±5% are considered unresolved <ref:2611. p. 2 We compare each variant with the base model protein by protein. For each protein we take the ratio
of the variant’s RMSE to the base model’s, each averaged geometrically over training seeds, and
summarise a cell by the geometric mean of these ratios, reported as a percentage change.Where the paper says it
G Scores After Reweighting
After reweighting the ensembles towards the measurement, the base model becomes competitive with baselines on SAXS in AUC and in every cell except the transitional one. Kabsch remains worse in every SAXS cell, suggesting that its ensembles contain too few conformers compatible with the measurement for reweighting to recover. p. 4 It is worse even on ordered proteins, where
the gradient stabilisation that motivates it should help most, and the largest effect, on disordered
proteins, is consistent with the lever-arm mechanism of Sec. The degradation persists after
reweighting (Appendix G), suggesting that the ensembles contain too few conformers compatible
with the measurement. It is worse even on ordered proteins, where
the gradient stabilisation that motivates it should help most, and the largest effect, on disordered
proteins, is consistent with the lever-arm mechanism of Sec. The degradation persists after
reweighting (Appendix G), suggesting that the ensembles contain too few conformers compatible
with the measurement.Where the paper says it
Where the paper says it
Flory Scaling Exponents
The Flory scaling exponents follow the expected ordering from 0.34 for ordered to 0. p. 0 com
Abstract
Generative models of protein conformational ensembles inherit much of their
design from structure prediction for folded proteins. Ordered and disordered
proteins, however, occupy distinct structural regimes, and we investigate how
design choices hold up across them. The base model is a reasonable reference (Table 1). It is competitive with the three baselines on
SAXS, mainly through disordered proteins, but trails IDPFold2 on chemical shifts in every cell, and
4 We compute Rg per conformer from the Cα coordinates, weighted by residue mass and averaged
over the ensemble, and fit ν as the least-squares slope of log Rg against log N across the proteins of
a cell, with a standard error from a bootstrap over proteins. Unlike the benchmark scores, ν does not
depend on a forward model or experimental data, and so provides a consistency check on whether a
model reproduces the chain statistics characteristic of each regime (Sec.Where the paper says it
Where the paper says it
Where the paper says it
Training Source Trade-offs
Replacing CALVADOS ensembles with fragment-assembled ones improves chemical shifts by 11–16% outside the ordered cell, but worsens SAXS on disordered proteins by 62% and raises their Flory exponent to 0.61 <ref:2611. p. 0 Instead the probes available for disordered proteins measure ensemble-averaged observables,
principally small-angle X-ray scattering (SAXS) [11] and NMR chemical shifts (CS) [12]. 1
arXiv:2611. Replacing the CALVADOS ensembles with
fragment-assembled ones improves chemical shifts by 11–16% outside the ordered cell, bringing
the chemical-shift AUC level with IDPFold2’s, but worsens SAXS on disordered proteins by 62%
and raises their Flory exponent to 0. Fragment 50% keeps most of the chemical-shift gain (−9%
against −11% in AUC) at a fraction of the SAXS cost, with the SAXS AUC only slightly worse. Ordered and disordered
proteins, however, occupy distinct structural regimes, and we investigate how
design choices hold up across them. Using a minimal Cα flow-matching model,
we ablate choices of architecture, sequence conditioning, rigid-body alignment
and training source, and evaluate each variant against experimental SAXS and
NMR chemical-shift data partitioned into ordered, disordered, multi-domain and
transitional proteins.Where the paper says it
Where the paper says it
Where the paper says it
Model Size Effects
Reducing the size of the protein language model degrades performance on ordered and multi-domain proteins but not on disordered ones <ref:2611.05055#pg4>. With ESM2-8M in place of ESM2-650M, ordered proteins are worse on both observables, while disordered proteins show no detectable change <ref:2611. p. 4 Reducing the size of the protein language model degrades performance on ordered and multi-domain
proteins but not on disordered ones (Figure 1). With ESM2-8M in place of ESM2-650M, ordered
proteins are worse on both observables (SAXS +24%, chemical shifts +12%), as are multi-domain
proteins on SAXS (+26%), while disordered proteins show no detectable change, in line with the
coevolutionary argument of Sec. Even deliberately weakening the model changes little. The small model is a third the size of the base
model and shows no detectable effect, while removing RoPE costs only a few percent, mostly on
SAXS (AUC +8%).Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Training data selection can be optimized by developing training sources that are accurate at both local and global scales across structural regimes. This addresses a key finding that
the source of the disordered training data trades local against global accuracy, a trade-off that an equal mixture of the two sources softens.
-
The model architecture should be adapted based on the target structural regime. For example, replacing ESM2 with ESM2-8M can improve performance on ordered and multi-domain proteins while showing
no detectable change
on disordered ones, as stated inReducing the size of the protein language model degrades performance on ordered and multi-domain proteins but not on disordered ones.
-
The choice of architectural components should be regime dependent to maximize utility. For instance,
Loss-side Kabsch alignment degrades SAXS in every cell, from +26% on ordered proteins to +99% on disordered ones,
suggesting that this component is detrimental when the ensemble lacks conformers compatible with the measurement in specific regimes. -
The model size should be scaled appropriately for the target protein type.
Reducing the size of the protein language model degrades performance on ordered and multi-domain proteins but not on disordered ones,
indicating that larger models are beneficial for capturing features relevant to folded structures without negatively impacting disordered regions.
Abstract
Generative models of protein conformational ensembles inherit much of their design from structure prediction for folded proteins. Ordered and disordered proteins, however, occupy distinct structural regimes, and we investigate how design choices hold up across them. Using a minimal C α flow-matching model, we ablate choices of architecture, sequence conditioning, rigid-body alignment and training source, and evaluate each variant against experimental SAXS and NMR chemical-shift data partitioned into ordered, disordered, multi-domain and transitional proteins. Mixture-of-experts routing, a static pair representation and a disorder probe have no detectable effect, a smaller protein language model performs worse on ordered proteins but not on disordered ones, and alignment in the loss degrades agreement with SAXS. Replacing coarse-grained simulations of disordered proteins with fragment-assembled ensembles improves agreement with chemical shifts at a cost to SAXS, while an equal mixture of the two keeps most of the gain at a smaller cost. This points to the training data, rather than the architecture, as the main constraint.
Sources
Related papers
- Atomic Trajectory Modeling with State Space Models for Biomolecular Dynamics
- Agent-guided epitope excision enables economical de novo binder design
- CupOFMoCA: Coupled Objective-Guided Discrete Flows for Molecular Conjugate Assembly
- BioBlobs: Unsupervised Discovery of Functional Substructures
- CPTCs Drive Somatic-Visceral Communication via the Wnt Axis in Somatic Mechanotherapy: A Single-Cell Deep Learning Study
- Speak to a Protein: An Interactive Multimodal Co-Scientist