Generative modeling of conditional probability distributions on the level-sets of collective variables

arXiv:2512.17374 · stat.ML, math.OC · Submitted 2025-12-19 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Generative modeling of conditional probability distributions on the level-sets of collective variables".

Paper summary: Tom: So, looking at the whole paper, the main thing is that they've developed a general and practical approach for conditional generative modeling on manifolds defined by collective variables.

Jane: It’s about taking a probability distribution and learning how to generate samples conditioned on being on a specific level-set of some map.

Lu: The authors successfully showed how to learn these models simultaneously for different level-sets of the same collective variable, which is a key feature.

Meng: It’s interesting because it tackles the challenge of learning constrained distributions where we usually only have data from the full space.

Tom: The paper proposes this general and efficient learning approach, and they demonstrate its effectiveness with concrete numerical examples on systems like the Müller Brown potential.

Jane: The authors also introduced a data enrichment strategy by using enhanced sampling techniques, which helps improve learning in those low-probability regions where things get sparse under the full distribution.

Lu: The main conclusion is that exploiting biased data through this reweighting strategy leads to an objective function that enhances the loss for values z in those low-density regions under mu e.

Tom: It means you can use techniques from enhanced sampling, like adaptive biasing force simulations, to train a generative model that respects specific constraints much better than if you just trained it on the raw data.

Jane: This framework gives us a powerful tool for conditional generative modeling on manifolds defined by collective variables, which has potential applications in molecular modeling and beyond.

Lu: Future work they suggest involves applying this method to more complex molecular systems and combining it with methods for identifying good collective variable maps and propagating effective dynamics in the CV space.

Meng: I wonder if the main limitation is that we still need to figure out how to best identify those good collective variable maps before we can use this efficiently at scale.

Tom: Exactly, so the practical challenge moves from just training a model to figuring out which constraints, which level-sets, are worth modeling first and how to map them onto our existing data structure.

Conclusion: Tom: So, we're wrapping up this paper, "Generative modeling of conditional probability distributions on the level-sets of collective variables."

Jane: Basically, they’ve figured out a way to create generative models that work for specific shapes or constraints defined by some set of variables.

Lu: They’re looking at how you can learn these models simultaneously for many different level-sets at once, which is pretty general.

Meng: It sounds like a way to handle those weird, constrained distributions where you don't have enough data for every single possibility.

Tom: Exactly, it's about learning the distribution on these specific surfaces rather than just the whole space.

Jane: They use a flow-matching setup, that ODE thing they learned earlier, but they adapt it to these level-sets using a parameter called z.

Lu: The real clever part is how they make it better for low-probability areas by using data from enhanced sampling simulations, like adaptive biasing force ones.

Meng: So instead of just training on the raw data where things are sparse, they reweight the distribution to make those rare events easier to learn.

Tom: That leads them to a loss function that actually focuses more on those hard-to-learn regions under the target distribution.

Jane: The validation on systems like the Müller Brown potential showed that this approach works really well, giving consistent samples along the right contours.

Lu: They even have a post-processing step to make sure the generated data perfectly matches those level-set constraints, which is important for accuracy.

Meng: From a deployment standpoint, if you can train a model that respects these constraints and learns from biased data, it could be useful in modeling things with strict physical boundaries.

Tom: It opens up conditional generative modeling on manifolds defined by collective variables as a practical thing.

Jane: It’s a solid framework for molecular modeling and beyond, offering flexibility in how you define those constraints.

Lu: The next step is figuring out how to use this when we have really complex systems where defining those good collective variable maps isn't straightforward yet.

ENPC Institut Polytechnique de Paris · Zuse Institute Berlin

stat.ML, math.OC

Submitted: 2025-12-19

Updated: 2026-11-18

Comments: To appear at Commun. Appl. Math. Comput. Sci (CAMCoS). See https://msp.org/soon/coming.php?jpath=camcos

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 79/100

The gist: Generative modeling of conditional probability distributions on the level-sets of collective variables proposes a general and efficient learning approach that can learn generative models on different…

Key concepts

Level-set
A level-set is a set of points where a smooth mathematical function, the collective variable map $\xi$, has the same constant value, $z$. In this context, it defines a specific surface or manifold in the system's space that corresponds to a particular state of the collective variables.
Flow-matching generative modeling
This is a learning framework where an ordinary differential equation (ODE) is trained to transform an initial random distribution into a desired target distribution. The authors adapt this by making the ODE depend on the specific value of the collective variable $z$, allowing the model to generate data constrained to different level-sets.
Adaptive Biasing Force (ABF)
ABF is an enhanced sampling technique used in simulations to overcome slow dynamics or low-probability regions. The paper uses data generated by ABF simulations, which are reweighted using a specific probability distribution ($\mu_{RW}$), to train the generative model. This helps the model learn accurately where data is scarce.
Conditional Probability Distribution
This refers to the probability of observing a certain outcome given that another variable (the collective variable $z$) has taken on a specific value. The goal is to learn how the system behaves specifically when its collective variables are fixed at a particular level-set.

Terminology

Summary

Generative modeling of conditional probability distributions on the level-sets of collective variables proposes a general and efficient learning approach that can learn generative models on different level-sets of a collective variable simultaneously. p. 0

Where the paper says it

Generative modeling of conditional probability distributions on the level-sets of collective variables Fatima-Zahrae Akhyar∗ Wei Zhang† Gabriel Stoltz‡ Christof Sch¨utte§ Abstract Given a probability distribution µ in R d represented by data, we study in this paper the generative modeling of the corresponding conditional probability distributions on the level-sets of a collective variable ξ : R d → R k , where 1 ≤ k < d. We propose a general and efficient learning approach that can learn generative models on different level-sets of ξ simultaneously.

Background and Mathematical Formulation

The paper introduces the study of conditional probability distributions on the level-sets of a smooth map ξ, where the distribution µ is given by a positive probability density ρ with respect to the Lebesgue measure. The authors define the level-set Σz as the set of points x such that ξ(x) = z. p. 5

Where the paper says it

3 Projection step Once the model is learned using the approach described in the previous section, it can be used to generate new data points on different level-sets of ξ by integrating the ODE (10) with different values z ∈ R k . However, due to numerical errors, the generated data may not lie exactly on the target level-set.

They derive formulas relating the conditional distribution µz on Σz to the pushforward measure µe on Rk via a normalizing factor Q(z). p. 2
Where the paper says it

2]) µz(dx) = 1 Q(z) ρ(x) det ∇ξ(x) ⊤∇ξ(x) − 1 2 σΣz (dx), x ∈ Σz , (3) and µe(dz) = Q(z) dz , z ∈ R k , (4) where σΣz denotes the surface measure of Σz and Q(z) = Z Σz ρ(x) det ∇ξ(x) ⊤∇ξ(x) − 1 2 σΣz (dx), z ∈ R k , (5) is a normalizing factor in (3). For processes whose invariant distribution is the Boltzmann–Gibbs distribution µ(dx) = 1 Z e −βV (x)dx, the free energy associated to the map ξ is defined as F(z) = −β −1 ln Q(z), z ∈ R k , (6) and, from (4), we have µe(dz) = e−βF (z) dz , z ∈ R k .

For the Boltzmann–Gibbs distribution, they define the free energy F(z) as F(z) = −β−1ln Q(z), which implies that µe(dz) = e−βF (z) dz. p. 2
Where the paper says it

2]) µz(dx) = 1 Q(z) ρ(x) det ∇ξ(x) ⊤∇ξ(x) − 1 2 σΣz (dx), x ∈ Σz , (3) and µe(dz) = Q(z) dz , z ∈ R k , (4) where σΣz denotes the surface measure of Σz and Q(z) = Z Σz ρ(x) det ∇ξ(x) ⊤∇ξ(x) − 1 2 σΣz (dx), z ∈ R k , (5) is a normalizing factor in (3). For processes whose invariant distribution is the Boltzmann–Gibbs distribution µ(dx) = 1 Z e −βV (x)dx, the free energy associated to the map ξ is defined as F(z) = −β −1 ln Q(z), z ∈ R k , (6) and, from (4), we have µe(dz) = e−βF (z) dz , z ∈ R k .

Standard Learning Methodology

The standard approach utilizes a flow-matching generative modeling framework where an ordinary differential equation (ODE) dXt/dt = v(Xt, t), X0 ∼ νprior, t ∈ [0, 1] is learned to transform a prior distribution νprior to the target distribution ν. To adapt this for level-sets, they propose learning an ODE dXt/dt = v(z)(Xt, t), X0 ∼ νprior, t ∈ [0, 1], parameterized by z ∈ Rk. p. 3

Where the paper says it

Adapting this framework to our setting, we propose to learn the ODE dXt dt = v (z) (Xt, t), X0 ∼ νprior , t ∈ [0, 1], (10) parametrized by a variable z ∈ R k , such that, for a fixed z ∈ R k , the system (10) transforms the prior distribution from t = 0 to the conditional distribution µz at t = 1. To achieve this goal, we consider the following objective for learning the vector field in (10): L(v) = Et∼U([0,1])EX0∼νprior,X1∼µ v (ξ(X1))(1 − t)X0 + tX1, t − (X1 − X0) 2 , (11) where µ is the probability distribution introduced in the previous section.

The objective function L(v) is derived to minimize the standard flow-matching objective simultaneously for different target distributions µz. p. 3
Where the paper says it

1 Standard learning approach Our goal is to sample the conditional probability distributions µz for different values z ∈ R k . For this purpose, we adopt the flow-matching generative modeling framework [32, 31, 1], which employs the ordinary differential equation (ODE) in R d dXt dt = v(Xt, t), X0 ∼ νprior , t ∈ [0, 1] (8) to transform a prior distribution νprior to a target distribution ν that is represented by data.

In practice, this is replaced by the empirical objective L(N)(v) which involves samples from the prior and data points. p. 4
Where the paper says it

Therefore, it allows us to learn the ODE (10) for generative modeling on level-sets Σz corresponding to different values z at the same time. In practice, (11) is replaced by the empirical objective L (N) (v) = 1 N X N n=1 v (ξ(X(n) ))(1 − tn)X (n) 0 + tnX(n) , t − (X(n) − X (n) 0 ) 2 , (13) where t1, t2, .

Enhanced Learning Strategy

To improve learning in regions of low probability density under µe, an enhanced learning strategy is proposed that leverages data obtained from enhanced sampling techniques such as adaptive biasing force (ABF) simulations. This involves reweighting the probability distribution to µRW(dx) = Z¯−1Z−1e−βV(x − A(ξ(x))) dx, which ensures that the conditional probability distribution of µRW on the level-set Σz coincides with µz. p. 4

Where the paper says it

In particular, let us consider the reweighted (RW) probability distribution µRW(dx) = Z¯−1Z −1 e −β V (x)−A(ξ(x)) dx , x ∈ R d , (15) where A : R k → R and Z¯ = Z −1 R Rd e −β V (x)−A(ξ(x)) dx is a normalizing factor. Then, for each z ∈ R k , it is straightforward to verify that the conditional probability distribution of µRW on the level-set Σz coincides with µz given in (3).

Minimizing the objective LRW(v) leads to an objective that enhances the contribution of loss for values z in low-density regions under µe. p. 4
Where the paper says it

Likewise, for a proper CV map ξ, the pushforward distribution µe defined in (4) is highly non-uniform in R k as well. The generative model that is learned by minimizing the objective (11) (or its empirical counterpart (13)) using training data sampled from (14) may be inferior in the low-density regions under µe, because of the relatively small contribution of the loss in such regions to the aggregated objective (11) (see (12)).

Validation and Results

The proposed approach is validated on a hierarchy of systems, including a 2D synthetic dataset and the Müller–Brown potential system. In the Müller–Brown case, training the ODE model using ABF-biased trajectory data produces spatially consistent samples distributed along the correct CV contours. p. 9

Where the paper says it

These samples are visualized in Figures 7a and 8a. Comparing these two figures reveals that the model trained using the ABF-biased trajectory data produces spatially consistent samples distributed along the correct CV contours (Figure 8a), whereas the model trained 10

Furthermore, a post-processing step involving projecting generated data onto the target level-set can be used to ensure exact satisfaction of constraints. p. 5
Where the paper says it

However, due to numerical errors, the generated data may not lie exactly on the target level-set. Here, we propose a post-processing step that can be used optionally (depending on the specific applications) to project the generated data onto the target level-set.

The results demonstrate that the model trained on the ABF-biased trajectory data achieves a lower deviation across the CV range compared to models trained on unbiased trajectory data. p. 8
Where the paper says it

The results in Figure 2 demonstrate that the trained model performs consistently well across all evaluation metrics. Namely, the generated CV values closely match the intended targets, the deviation remains low across the entire range of values, and the model accurately learns the conditional distribution.

Conclusion and Future Directions

The study concludes that the framework provides a general and practical approach for conditional generative modeling on manifolds defined by CVs, successfully exploiting biased data to improve learning in scarcely sampled regions. Future work is suggested to apply the method to more complex molecular systems and combine it with methods for identifying good CV maps and propagating effective dynamics in the CV space. The paper offers a general and practical approach for conditional generative modeling on manifolds defined by CVs, offering both methodological flexibility and the potential for applications in molecular modeling and beyond. p. 1

Where the paper says it

The approach is validated on a hierarchy of systems of increasing complexity, from synthetic low-dimensional examples to high-dimensional molecular datasets. The main contributions of this paper are summarized below: (i) A general and efficient generative modeling framework capable of learning conditional distributions on families of manifolds implicitly defined by CVs; (ii) A data enrichment strategy that utilizes enhanced sampling data to improve accuracy in low-density regions; and (iii) Comprehensive numerical validations that demonstrate the flexibility, robustness, and practical applicability of the method.

Acknowledgements

Fatima-Zahrae Akhyar acknowledges Zuse institute Berlin for hosting her internship during which this research was conducted. Wei Zhang has been funded through DFG’s “Eigene Stelle” program (project 524086759) and has been supported by the Young Investigator Program of the DFGgrant CRC 1114 “Scaling Cascades in Complex Systems” Project Number 235221301. Gabriel Stoltz was funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (project EMC2, grant agreement No 810367) and was supported by Hi! p. 0

Where the paper says it

Generative modeling of conditional probability distributions on the level-sets of collective variables Fatima-Zahrae Akhyar∗ Wei Zhang† Gabriel Stoltz‡ Christof Sch¨utte§ Abstract Given a probability distribution µ in R d represented by data, we study in this paper the generative modeling of the corresponding conditional probability distributions on the level-sets of a collective variable ξ : R d → R k , where 1 ≤ k < d. We propose a general and efficient learning approach that can learn generative models on different level-sets of ξ simultaneously.

PARIS and ANR/France 2030 program (ANR-23-IACL-0005). p. 0
Where the paper says it

A notable approach is the effective dynamics constructed using conditional expectations, where the coefficients (i. drift and diffusion) in the surrogate model are computed by taking ∗ENPC, Institut Polytechnique de Paris, Marne-la-Vall´ee, France; Email: fatimazahrae.

Christof Sch¨utte has been supported through DFG-grant CRC 1114 (Project A04 “Efficient calculation of slow and stationary scales in molecular dynamics”, Project A05 “Probing scales in equilibrated systems by optimal nonequilibrium forcing”, Project B03 “Multilevel coarse graining of multiscale problems”, and Project C03 “Multiscale modelling and simulation for spatiotemporal master equation”). The references listed are [1] through [52].

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Learn generative models on families of manifolds implicitly defined by CVs using flow-matching neural ODEs, which does not require explicit geometric information of underlying manifolds and can generate samples across different level-sets of a CV map ξ.

  2. Implement a data enrichment strategy that leverages data obtained from enhanced sampling techniques such as adaptive biasing force (ABF) simulations to improve accuracy in low-density regions.

  3. Generate new data points on different level-sets of ξ by integrating the learned ODE with different values z in R k, and optionally employ a post-processing step to project generated data onto the target level-set using methods like projecting the generated state x along the ODE flow dYt dt = -∇G(Yt).

  4. Train models on enhanced sampling biased trajectory data to produce spatially consistent samples distributed along correct CV contours, which is demonstrated for complex systems like the Müller–Brown potential.

  5. Improve generation quality by using trigonometric functions as inputs to the vector field when conditioning on dihedral angles, as this helps reduce mean deviations in regions close to boundaries of the CV space.

Abstract

Given a probability distribution μ in R d represented by data, we study in this paper the generative modeling of the corresponding conditional probability distributions on the level-sets of a collective variable ξ: R d to R k, where 1 at most k<d. We propose a general and efficient learning approach that can learn generative models on different level-sets of ξ simultaneously. To improve the learning quality on level-sets in low-probability regions, we also propose a data enrichment strategy by utilizing data from enhanced sampling techniques. We demonstrate the effectiveness of our proposed learning approach through concrete numerical examples. The proposed approach is potentially useful for the generative modeling of molecular systems in biophysics.

Related papers