Conservative neural posterior estimation via distributionally robust training

arXiv:2605.28516 · stat.ML, cs.LG · Submitted 2026-05-27 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Conservative neural posterior estimation via distributionally robust training".

Jane: Here's the gist of the paper. Simulation-based inference with neural posterior estimation (NPE) often yields overconfident and unreliable posteriors under limited simulation budgets.

Tom: The paper proposes DRO-NPE, a distributionally robust approach that replaces the standard NPE objective with a worst-case loss over a Wasserstein ambiguity set to control overfitting and reduce posterior overconfidence.

Jane: Pruning redundant reasoning tokens The central practical limitation of NPE is the simulation budget required to train the conditional density estimator.

Tom: Prior work has sought to reduce simulation cost using sequential training schemes, multi-fidelity simulations, or by exploiting heterogeneous simulation costs.

Paper summary: Jane: So, to wrap up what we’ve heard about "Conservative neural posterior estimation via distributionally robust training," the authors are showing that you can use this distributionally robust approach to tame the overconfidence issues in neural posterior estimation when simulation budgets are small.

Tom: Exactly. The core idea is replacing the standard NPE objective with a worst-case loss over a Wasserstein ambiguity set, which essentially gives us an upper bound on the true population risk (<ref:2605.28516#pg1>).

Lu: What this means in practice is that you get statistical guarantees on excess risk and miscalibration because they’ve shown how the KL-based metrics for miscoverage and miscalibration relate to each other, upper-bounding both (<ref:2605.28516#pg3>).

Meng: For someone deploying a model, this means you can trust that the uncertainty statements are not just based on what the model thinks is most likely with limited data, but on a more conservative measure of risk across all possible distributions.

Jane: The paper demonstrates that DRO-NPE improves coverage and calibration across various tasks while narrowing the gap between empirical and population NPE loss, which leads to more reliable inference in those low simulation regimes (<ref:2605.28516#pg1>).

Tom: So, the title points to a specific mechanism—using distributionally robust training—to achieve that conservative yet efficient posterior estimation. It’s a way to ensure that when you use these neural networks for inference, you aren't blindly trusting their high confidence scores when the data is scarce.

Conclusion: Tom: So, we've been looking at how neural networks guess the true parameters from limited simulation data, and this paper introduces something called DRO-NPE, Distributionally Robust Neural Posterior Estimation.

Jane: That sounds like a mouthful for a title. Basically, they’re trying to make those neural network guesses more reliable when you don't have a ton of simulations to train on.

Tom: Right. The authors are using this distributionally robust approach to replace the standard way they measure risk. Instead of just minimizing the expected loss from their limited data, they look at the worst-case loss over a specific set of possible distributions called an ambiguity set.

Lu: What that means is they are essentially building a safety net for their uncertainty estimates. They aren't just trusting one single way the model might guess wrong; they're hedging against all plausible ways it could fail in the real world.

Meng: From my side, what I like is that instead of just saying "this model has this confidence," they are giving us a bound on how much we can actually trust that confidence when the simulation budget is low. That’s something I deal with every day when models start breaking at scale.

Jane: So, it seems the main goal here isn't to make a single perfect prediction, but to get an estimate of the uncertainty that doesn't wildly overestimate how sure the model is.

Tom: Exactly. They show that this method actually makes coverage and calibration better across all these different simulation tasks we tested, like Lotka–Volterra and Inverse Kinematics. The results are quite solid when you look at those specific benchmarks, showing conservative posters for some problems and near-perfect coverage for others.

Lu: The math is clever because they link the uncertainty quantification metrics—like KL-based miscoverage—to each other, creating a chain where one bounds the other. That’s how they get the statistical guarantees on excess risk and miscalibration you see in their results.

Jane: It sounds like this method provides a more honest picture of what we don't know when we’re working with sparse simulation data. But what does it actually change for someone just trying to understand these models?

Tom: Well, it means that for those of us using simulation-based inference, we might start deploying these neural posterior estimators with a bit more confidence in the actual risk assessment they provide, even when we're running on a tight budget.

Lu: It’s a way to get something conservative but still efficient. There is still a trade-off because it’s more computationally intensive than just standard neural posterior estimation, but the statistical bounds justify that extra work.

Jane: So the final word is that this method gives us a tool to infer parameters reliably in those low-simulation scenarios we all struggle with. Next up, we're going to look at how they actually tune the main setting for this robustness—that epsilon value.

William Laplante, Yuga Hikida, Charita Dellaporta, François-Xavier Briol, Ayush Bharti

Department of Statistical Science, University College London, UK · Department of Computer Science, Aalto University, Finland · The Alan Turing Institute

stat.ML, cs.LG

Submitted: 2026-05-27

Updated: 2026-11-04

Code: https://github.com/yugahikida/dro-npe

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 85/100

The gist: Simulation-based inference with neural posterior estimation (NPE) often yields overconfident and unreliable posteriors under limited simulation budgets.

Key concepts

Neural Posterior Estimation (NPE)
This is the standard method where a neural network is trained to estimate the true probability distribution from simulation data. It minimizes the expected KL divergence between the true posterior and what the neural network predicts. The problem arises when simulation budgets are small, making this estimation unreliable.
Distributionally Robust Optimization (DRO)
DRO is a technique used to handle uncertainty in data distributions by optimizing against the worst-case scenario within a defined set of possible distributions. In this paper, it replaces the standard NPE loss with a worst-case value over a Wasserstein ball, ensuring robustness against distribution shifts.

Terminology

Summary

Simulation-based inference with neural posterior estimation (NPE) often yields overconfident and unreliable posteriors under limited simulation budgets. p. 0

Where the paper says it

hikida@aalto. fi Abstract Simulation-based inference with neural posterior estimation (NPE) often yields overconfident and unreliable posteriors under limited simulation budgets.

The paper proposes DRO-NPE, a distributionally robust approach that replaces the standard NPE objective with a worst-case loss over a Wasserstein ambiguity set to control overfitting and reduce posterior overconfidence. p. 0
Where the paper says it

To address this, we propose DRO-NPE, a distributionally robust approach that replaces the standard NPE objective with a worst-case loss over a Wasserstein ambiguity set. We introduce KL-based metrics for miscoverage and miscalibration, and use these to show that the DRO-NPE objective controls overfitting and reduces posterior overconfidence.

Pruning redundant reasoning tokens

The central practical limitation of NPE is the simulation budget required to train the conditional density estimator. Prior work has sought to reduce simulation cost using sequential training schemes, multi-fidelity simulations, or by exploiting heterogeneous simulation costs. p. 0

Where the paper says it

As many scientific simulators are expensive to run, generating a sufficiently large training set is often the main computational bottleneck. Prior work has sought to reduce simulation cost using sequential training schemes (Papamakarios and Murray, 2016; Lueckmann et al.

However, these strategies do not directly address the challenge of ensuring reliable uncertainty estimates in low-simulation regimes. p. 0
Where the paper says it

, 2025), or by exploiting heterogeneous simulation costs (Bharti et al. However, these strategies do not directly address the challenge of ensuring reliable uncertainty estimates in low-simulation regimes.

Under finite simulations, NPE can yield overconfident posterior approximations whose credible regions contain the true parameter less often than their nominal coverage suggests. p. 1
Where the paper says it

(2022) to warn of the potential for a “trust crisis”. Under finite simulations, NPE can yield overconfident posterior approximations whose credible regions contain the true parameter less often than their nominal coverage suggests.

How the training signal is built

The standard NPE objective is obtained by minimizing the expected KL divergence between the true posterior and the neural estimator over all possible estimators. When simulation budgets are limited, this empirical risk can be a poor proxy for population risk, leading to overfitting and poorly calibrated posteriors. p. 1

Where the paper says it

Standard NPE minimises an empirical risk over a finite set of simulated parameter–data pairs. When the simulation budget is limited, the empirical risk can be a poor proxy for the population risk, leading to overfitting and poorly calibrated posteriors (Hermans et al.

To address this, DRO-NPE minimizes an upper bound on the population NPE loss by replacing the empirical risk with its worst-case value over an ambiguity set defined by a Wasserstein ball around the empirical measure. p. 4
Where the paper says it

We formalise DRO-NPE as a Wasserstein distributionally robust objective. Rather than minimising the empirical NPE risk directly, we replace it by its worst-case value over an ambiguity set A(Pn) of distributions centred at Pn (Blanchet et al.

Where the gains come from

The authors introduce Kullback–Leibler (KL) divergence-based uncertainty-quantification metrics, specifically KL-based miscoverage and miscalibration, to generalize standard notions of coverage and calibration errors. They propose a Wasserstein DRO objective that upper-bounds these metrics and the population NPE loss, which controls both posterior-fitting and uncertainty-quantification error under suitable conditions. p. 1

Where the paper says it

, 2022; Talts et al. Second, we propose a Wasserstein DRO objective that upper-bounds these metrics and the population NPE loss, so that, under suitable conditions, minimising it controls both posterior-fitting and uncertainty-quantification error (Theorem 4.

The method is tractable, parallelisable, and readily integrates with standard normalising flows. p. 0
Where the paper says it

We introduce KL-based metrics for miscoverage and miscalibration, and use these to show that the DRO-NPE objective controls overfitting and reduces posterior overconfidence. Our method is tractable, parallelisable, and readily integrates with standard normalising flows.

Performance across benchmarks

Across benchmark SBI tasks, DRO-NPE consistently improves coverage and calibration while narrowing the gap between empirical and population NPE loss. In experiments on Lotka–Volterra, SLCP, Two Moons, and Inverse Kinematics, DRO-NPE yields conservative posteriors for SLCP and near-perfect coverage for Two Moons and Inverse Kinematics. p. 8

Where the paper says it

3 for details. DRO-NPE yields conservative posteriors for SLCP and near-perfect coverage for Two Moons and Inverse Kinematics.

The application to cosmology using the CAMELS suite shows that DRO-NPE performs best overall, with the closest coverage to the diagonal, the lowest NLPD, and the lowest klq cal. p. 9
Where the paper says it

5 kl q cal Figure 4: Cosmology results: expected coverage, NLPD, and klq cal for DRO-NPE ( ), CR-NPE ( ), Bal-NPE ( ), and standard NPE ( ). Finally, we evaluate DRO-NPE on the CAMELS suite (Villaescusa-Navarro et al.

Analysis of design choices

The selection of hyperparameter epsilon is performed using a validation metric called KL-based miscalibration (klq cal), which is motivated by its link to entropy maximisation. Experiments show that selecting epsilon via klq cal yields coverage curves closest to the diagonal while maintaining competitive NLPD. p. 9

Where the paper says it

NLPD is a natural validation metric because it equals the NPE loss and upper-bounds klq cal, while absolute miscoverage directly measures deviation from nominal coverage. As shown in Figure 3(b), selecting ε via klq cal yields a coverage curve closest to the diagonal.

The computational cost of DRO-NPE is higher than standard NPE, but only by a moderate constant factor due to higher-order automatic differentiation. p. 9
Where the paper says it

DRO-NPE is more expensive than standard NPE, but only by a moderate constant factor. The main overhead comes from higher-order automatic differentiation through ∇z log qϕ(θ | x), so the computational cost of this regularisation term grows with the joint parameter–data dimension dZ.

Conclusion

DRO-NPE provides a conservative yet efficient posterior inference in challenging low-simulation settings. It yields an objective that upper bounds the population NPE loss and KL-based metrics for uncertainty-quantification error, admits a tractable regularised objective, and enjoys statistical guarantees on excess risk, miscoverage, and miscalibration. p. 1

Where the paper says it

Second, we propose a Wasserstein DRO objective that upper-bounds these metrics and the population NPE loss, so that, under suitable conditions, minimising it controls both posterior-fitting and uncertainty-quantification error (Theorem 4. We also derive a tractable regularised objective that upper-bounds the DRO objective (Proposition 4.

Although it inherits limitations from DRO—being more expensive than standard NPE—it remains compatible with standard normalising-flow architectures commonly used in SBI. p. 1
Where the paper says it

We also derive a tractable regularised objective that upper-bounds the DRO objective (Proposition 4. 2), and verify its assumptions for standard normalising-flow architectures used in NPE (Lemma 4.

It could also be extended to other conditional density estimation problems and combined with complementary post-hoc calibration methods. p. 1
Where the paper says it

, 2024; Cabezas et al. , 2025) post-hoc calibrate credible sets with coverage guarantees, but do not directly address poor conditional density estimation.

Acknowledgments

The authors thank Niall Jeffrey for help with using the CAMELS dataset. WL was supported by UCL’s Center for Doctoral Training in Data-Intensive Science and by the Alan Turing Institute. p. 0

Where the paper says it

Conservative neural posterior estimation via distributionally robust training William Laplante1,3,∗ Yuga Hikida2,∗ Charita Dellaporta1 François-Xavier Briol1 Ayush Bharti2 1Department of Statistical Science, University College London, UK 2Department of Computer Science, Aalto University, Finland 3The Alan Turing Institute, UK Corresponding authors: william. hikida@aalto.

YH and AB were supported by the Research Council of Finland grant no. p. 0
Where the paper says it

Conservative neural posterior estimation via distributionally robust training William Laplante1,3,∗ Yuga Hikida2,∗ Charita Dellaporta1 François-Xavier Briol1 Ayush Bharti2 1Department of Statistical Science, University College London, UK 2Department of Computer Science, Aalto University, Finland 3The Alan Turing Institute, UK Corresponding authors: william. hikida@aalto.

362534. CD and FXB were supported by the EPSRC grant [EP/Y022300/EP/Y02]. p. 4
Where the paper says it

By contrast, Wasserstein balls are based on mass transport (Kuhn et al. , 2019), and can contain distributions supported away from the observed simulations, including continuous distributions.

The authors acknowledge the computational resources provided by the Aalto Science-IT project. p. 0
Where the paper says it

Conservative neural posterior estimation via distributionally robust training William Laplante1,3,∗ Yuga Hikida2,∗ Charita Dellaporta1 François-Xavier Briol1 Ayush Bharti2 1Department of Statistical Science, University College London, UK 2Department of Computer Science, Aalto University, Finland 3The Alan Turing Institute, UK Corresponding authors: william. hikida@aalto.

References

Alquier, P. (2024). User-friendly introduction to PAC-Bayes bounds. p. 0

Where the paper says it

Across benchmark SBI tasks, DRO-NPE consistently improves coverage and calibration, while narrowing the gap between empirical and population NPE loss, leading to more reliable inference in low-simulation regimes. 1 Introduction Simulation-based inference (SBI; Cranmer et al.

Alsing, J., Wandelt, B., and Feeney, S. (2018). Massive optimal data compression and density estimation for scalable, likelihood-free inference in cosmology. p. 0
Where the paper says it

1 Introduction Simulation-based inference (SBI; Cranmer et al. , 2020) is a powerful framework for inferring parameters of scientific models whose likelihood functions are unavailable or computationally prohibitive to evaluate, but for which simulating data is straightforward.

Bharti, A., Dellaporta, C., Hikida, Y. p. 0
Where the paper says it

Conservative neural posterior estimation via distributionally robust training William Laplante1,3,∗ Yuga Hikida2,∗ Charita Dellaporta1 François-Xavier Briol1 Ayush Bharti2 1Department of Statistical Science, University College London, UK 2Department of Computer Science, Aalto University, Finland 3The Alan Turing Institute, UK Corresponding authors: william. hikida@aalto.

, and Briol, F.-X. (2026). Amortised and provably-robust simulation-based inference. p. 0
Where the paper says it

fi Abstract Simulation-based inference with neural posterior estimation (NPE) often yields overconfident and unreliable posteriors under limited simulation budgets. To address this, we propose DRO-NPE, a distributionally robust approach that replaces the standard NPE objective with a worst-case loss over a Wasserstein ambiguity set.

Bishop, C. M. (1994). Mixture density networks. p. 2
Where the paper says it

The goal of SBI is to approximate this posterior distribution Pθ|x0 given observed data xo. NPE is an amortised SBI method that approximates the posterior using flexible conditional density estimators such as mixture-density networks (Bishop, 1994) and normalising flows (Papamakarios et al.

Cranmer, K. p. 0
Where the paper says it

Across benchmark SBI tasks, DRO-NPE consistently improves coverage and calibration, while narrowing the gap between empirical and population NPE loss, leading to more reliable inference in low-simulation regimes. 1 Introduction Simulation-based inference (SBI; Cranmer et al.

, Brehmer, J. p. 0
Where the paper says it

, 2020) is a powerful framework for inferring parameters of scientific models whose likelihood functions are unavailable or computationally prohibitive to evaluate, but for which simulating data is straightforward. The use of flexible neural conditional density estimators has substantially expanded the applicability of SBI to challenging problems, especially in fields such as particle physics (Brehmer, 2021), cognitive neuroscience (Fengler et al.

, and Louppe, G. (2020). The frontier of simulation-based inference. p. 0
Where the paper says it

hikida@aalto. fi Abstract Simulation-based inference with neural posterior estimation (NPE) often yields overconfident and unreliable posteriors under limited simulation budgets.

Delaunoy, A. p. 1
Where the paper says it

By contrast, conservative posteriors are often safer as they widen uncertainty statements rather than overstating confidence. This has motivated several works on conservative uncertainty quantification in neural SBI: Delaunoy et al.

p. 1
Where the paper says it

By contrast, conservative posteriors are often safer as they widen uncertainty statements rather than overstating confidence. This has motivated several works on conservative uncertainty quantification in neural SBI: Delaunoy et al.

, Hermans, J. p. 1
Where the paper says it

simulations can exhibit poor uncertainty quantification, leading Hermans et al. (2022) to warn of the potential for a “trust crisis”.

, Rozet, F., Wehenkel, A., and Louppe, G. (2024). Low-budget simulation-based inference with Bayesian neural networks. p. 1
Where the paper says it

Instead of regularising standard NPE, Delaunoy et al. (2024) use Bayesian neural networks for conservative inference, at the cost of Bayesian inference over network weights.

p. 1
Where the paper says it

Instead of regularising standard NPE, Delaunoy et al. (2024) use Bayesian neural networks for conservative inference, at the cost of Bayesian inference over network weights.

Dey, B., Zhao, D., Andrews, B. H., Newman, J. A., Izbicki, R., and Lee, A. B. (2025). Towards instance-wise calibration: local amortized diagnostics and reshaping of conditional densities (LADaR). p. 6
Where the paper says it

Provided qϕmin (θi | xi) > 0, this implies ∇zqϕmin (θi | xi) = 0 ∀i = 1, . Thus, for large ε, the objective favours conditional densities that vary less sharply in z at the simulated pairs.

Dengler, M., Boelts, J., Steinbach, P., Moss, G., Moreau, T., Gloeckler, M., Rodrigues, P. L., Linhart, J. p. 2
Where the paper says it

Other diagnostics have also been studied (Zhao et al. , 2021; Linhart et al.

p. 2
Where the paper says it

Other diagnostics have also been studied (Zhao et al. , 2021; Linhart et al.

, Lappalainen, J. K., Miller, B. p. 1
Where the paper says it

, 2020) to inflate estimator uncertainty, and Delaunoy et al. (2023) extended this idea to contrastive NRE (Miller et al.

K., et al. (2025). Simulation-based inference: A practical guide. p. 0
Where the paper says it

hikida@aalto. fi Abstract Simulation-based inference with neural posterior estimation (NPE) often yields overconfident and unreliable posteriors under limited simulation budgets.

Delaunoy, A., Bonardeaux, M. d. l. B., Mishra-Sharma, S., and Louppe, G. (2024). Low-budget simulation-based inference with Bayesian neural networks. Dyer, J., Cannon, P., Farmer, J. D., and Schmon, S. M. (2024). Black-box Bayesian inference for agent-based models. p. 0
Where the paper says it

1 Introduction Simulation-based inference (SBI; Cranmer et al. , 2020) is a powerful framework for inferring parameters of scientific models whose likelihood functions are unavailable or computationally prohibitive to evaluate, but for which simulating data is straightforward.

Falkiewicz, M. p. 1
Where the paper says it

, 2022) and NPE. Falkiewicz et al.

, Takeishi, N., Shekhzadeh, I., Wehenkel, A., Delaunoy, A. p. 1
Where the paper says it

By contrast, conservative posteriors are often safer as they widen uncertainty statements rather than overstating confidence. This has motivated several works on conservative uncertainty quantification in neural SBI: Delaunoy et al.

, Louppe, G., and Kalousis, A. (2023). Calibrating neural simulation-based inference with differentiable coverage probability. p. 0
Where the paper says it

hikida@aalto. fi Abstract Simulation-based inference with neural posterior estimation (NPE) often yields overconfident and unreliable posteriors under limited simulation budgets.

Fengler, A. p. 0
Where the paper says it

, 2020) is a powerful framework for inferring parameters of scientific models whose likelihood functions are unavailable or computationally prohibitive to evaluate, but for which simulating data is straightforward. The use of flexible neural conditional density estimators has substantially expanded the applicability of SBI to challenging problems, especially in fields such as particle physics (Brehmer, 2021), cognitive neuroscience (Fengler et al.

, Govindarajan, L. N., Chen, T., and Frank, M. J. (2021). Likelihood approximation networks (LANs) for fast inference of simulation models in cognitive neuroscience. p. 0
Where the paper says it

, 2020) is a powerful framework for inferring parameters of scientific models whose likelihood functions are unavailable or computationally prohibitive to evaluate, but for which simulating data is straightforward. The use of flexible neural conditional density estimators has substantially expanded the applicability of SBI to challenging problems, especially in fields such as particle physics (Brehmer, 2021), cognitive neuroscience (Fengler et al.

Kingma, D. P. and Dhariwal, P. (2018). Glow: Generative flow with invertible 1x1 convolutions. p. 1
Where the paper says it

We also derive a tractable regularised objective that upper-bounds the DRO objective (Proposition 4. 2), and verify its assumptions for standard normalising-flow architectures used in NPE (Lemma 4.

Krouglova, A. p. 0
Where the paper says it

, 2019), multi-fidelity simulations (Hikida et al. , 2025; Krouglova et al.

N., Johnson, H. R., Confavreux, B., Deistler, M. p. 5
Where the paper says it

In practice, however, SBI is often performed on handcrafted or learned summary statistics rather than raw observations (Radev et al. , 2022; Deistler et al.

p. 5
Where the paper says it

In practice, however, SBI is often performed on handcrafted or learned summary statistics rather than raw observations (Radev et al. , 2022; Deistler et al.

, and Goncalves, P. J. (2026). Multifidelity simulation-based inference for computationally expensive simulators. p. 0
Where the paper says it

1 Introduction Simulation-based inference (SBI; Cranmer et al. , 2020) is a powerful framework for inferring parameters of scientific models whose likelihood functions are unavailable or computationally prohibitive to evaluate, but for which simulating data is straightforward.

Kuhn, D., Esfahani, P. p. 6
Where the paper says it

1 is a high-probability concentration radius ensuring P ∈ Ap(Pn; ε), but it depends on unknown constants and is not directly usable. We therefore follow common DRO practice and treat ε as a hyperparameter selected using a validation set (Mohajerin Esfahani and Kuhn, 2018).

M., Nguyen, V. A., and Shafieezadeh-Abadeh, S. (2019). Wasserstein distributionally robust optimization: Theory and applications in machine learning. p. 9
Where the paper says it

Finally, we evaluate DRO-NPE on the CAMELS suite (Villaescusa-Navarro et al. , 2021, 2023), a state-of-the-art cosmology dataset (dX = 39) for machine learning on simulated universes, used previously by Hikida et al.

Kühmichel, L. p. 3
Where the paper says it

The function S allows SBC to be applied in higher-dimensional settings. For example, S(θ; x) = ∥θ − θ0∥2 for a reference point θ0 is used in the BayesFlow library (Kühmichel et al.

, Huang, J. M., Pratz, V., Arruda, J., Olischläger, H., Habermann, D., Kucharsky, S., Elsemüller, L., Mishra, A., Bracher, N., Jedhoff, S., Schmitt, M., Bürkner P.-C. and Radev S. p. 0
Where the paper says it

(2024) for a recent review. In this work, we focus on the widely used neural posterior estimation (NPE) method (Papamakarios and Murray, 2016; Radev et al.

T. (2026). BayesFlow 2: Multi-backend amortized Bayesian inference in python. p. 1
Where the paper says it

Instead of regularising standard NPE, Delaunoy et al. (2024) use Bayesian neural networks for conservative inference, at the cost of Bayesian inference over network weights.

Lemos, P., Coogan, A., Hezaveh, Y., and Perreault-Levasseur, L. (2023). Sampling-based accuracy testing of posterior estimators for general inference. p. 9
Where the paper says it

We use 1,000 high-fidelity baryonic hydrodynamic simulations available to us, reserving 900 for training and 100 for testing. Conservative inference is particularly important in this setting as simulations are scarce, and the observables depend not only on Ωm and σ8, but also on nuisance astrophysical processes that can produce similar simulated data (Villaescusa-Navarro et al.

Linhart, J., Gramfort, A., and Rodrigues, P. L. C. (2022). Validation diagnostics for SBI algorithms based on normalizing flows. p. 0
Where the paper says it

Our method is tractable, parallelisable, and readily integrates with standard normalising flows. Across benchmark SBI tasks, DRO-NPE consistently improves coverage and calibration, while narrowing the gap between empirical and population NPE loss, leading to more reliable inference in low-simulation regimes.

Lotka, A. J. (1920). Analytical note on certain rhythmic relations in organic systems. p. 2
Where the paper says it

Assuming Pθ|x is absolutely continuous with respect to Qθ|x, the NPE objective is obtained by minimising the expected KL divergence between Pθ|x and Qθ|x over ϕ: Ex∼Px h KL Pθ|x ∥ Qθ|x i +K(P) = E(θ,x)∼P [− log qϕ(θ | x)] := LNPE(qϕ; P), (1) where z = (x, θ), and the additive constant K(P) does not depend on ϕ. Note that LNPE(qϕ; P) can be thought of as a population risk, as it is defined as an expectation under P, induced by the pointwise loss ℓNPE(qϕ; z) := − log qϕ(θ | x).

Lueckmann, J. p. 0
Where the paper says it

As many scientific simulators are expensive to run, generating a sufficiently large training set is often the main computational bottleneck. Prior work has sought to reduce simulation cost using sequential training schemes (Papamakarios and Murray, 2016; Lueckmann et al.

-M., Boelts, J., Greenberg, D. p. 0
Where the paper says it

Prior work has sought to reduce simulation cost using sequential training schemes (Papamakarios and Murray, 2016; Lueckmann et al. , 2017; Greenberg et al.

p. 0
Where the paper says it

Prior work has sought to reduce simulation cost using sequential training schemes (Papamakarios and Murray, 2016; Lueckmann et al. , 2017; Greenberg et al.

, Goncalves, P., and Macke, J. H. (2017). Flexible statistical inference for mechanistic models of neural dynamics. p. 0
Where the paper says it

, 2020) is a powerful framework for inferring parameters of scientific models whose likelihood functions are unavailable or computationally prohibitive to evaluate, but for which simulating data is straightforward. The use of flexible neural conditional density estimators has substantially expanded the applicability of SBI to challenging problems, especially in fields such as particle physics (Brehmer, 2021), cognitive neuroscience (Fengler et al.

Masserano, L. p. 1
Where the paper says it

(2024) use Bayesian neural networks for conservative inference, at the cost of Bayesian inference over network weights. Complementarily, conformal and Neyman-inversion methods (Masserano et al.

, Dorigo, T., Izbicki, R., Kuusela, M., and Lee, A. (2023). Simulator-based inference with WALDO: Confidence regions by leveraging prediction algorithms and posterior estimators for inverse problems. Miller, B. K., Weniger, C., and Forré, P. (2022). Contrastive neural ratio estimation. p. 1
Where the paper says it

This has motivated several works on conservative uncertainty quantification in neural SBI: Delaunoy et al. (2022) proposed regularised balanced neural ratio estimation (NRE; Hermans et al.

Modrák, M. p. 2
Where the paper says it

, 2018; Säilynoja et al. , 2022; Modrák et al.

, Moon, A. H., Kim, S., Bürkner P., Huurre N., Faltejsková K., Gelman A., and Vehtari A. (2023). Simulation-based calibration checking for Bayesian computation: The choice of test quantities shapes sensitivity. p. 7
Where the paper says it

(b) Negative log predictive density (NLPD) on held-out test data. (c) KL-based calibration metric klq cal used for selecting ε.

Rossi, B. and Sekhposyan, T. p. 3
Where the paper says it

, 2018; Gneiting et al. For the pushforward distributions (F Q S|x )#PS|x and (F Q S|x )#QS|x, where the latter is uniformly distributed under mild assumptions, this deviation is often measured via Kolmogorov–Smirnov distances or via Cramér–von Mises criteria (Rossi and Sekhposyan, 2019; Zhao et al.

(2019). Alternative tests for correct specification of conditional predictive densities. p. 6
Where the paper says it

Provided qϕmin (θi | xi) > 0, this implies ∇zqϕmin (θi | xi) = 0 ∀i = 1, . Thus, for large ε, the objective favours conditional densities that vary less sharply in z at the simulated pairs.

Säilynoja, T. p. 2
Where the paper says it

, 2022) and simulation-based calibration (Talts et al. , 2018; Säilynoja et al.

, Bürkner P.-C., and Vehtari A. (2022). Graphical test for discrete uniformity and its applications in goodness-of-fit evaluation and multiple sample comparison. p. 7
Where the paper says it

(2026): SLCP (dX = 8, dΘ = 5), Lotka–Volterra (dX = 20, dΘ = 4), Two Moons (dX = 2, dΘ = 2), and Inverse Kinematics (dX = 2, dΘ = 4). Evaluation is based on three metrics: (i) coverage plots, which show empirical versus nominal coverage, (ii) negative log predictive density (NLPD), which is the average negative log-density assigned by the learned posterior to the true parameter on held-out test points, and 8

Saoulis, A. p. 0
Where the paper says it

, 2026; Tatsuoka et al. , 2025; Saoulis et al.

A., Piras, D., Jeffrey, N. p. 0
Where the paper says it

, 2024) and cosmology (Alsing et al. , 2018; Jeffrey et al.

, Spurio-Mancini, A., Ferreira, A. M. G., and Joachimi, B. (2025). Transfer learning for multifidelity simulation-based inference in cosmology. p. 9
Where the paper says it

Finally, we evaluate DRO-NPE on the CAMELS suite (Villaescusa-Navarro et al. , 2021, 2023), a state-of-the-art cosmology dataset (dX = 39) for machine learning on simulated universes, used previously by Hikida et al.

Schmitt, M., Pratz, V., Köthe, U., Bürkner P.-C., and Radev S. p. 0
Where the paper says it

(2024) for a recent review. In this work, we focus on the widely used neural posterior estimation (NPE) method (Papamakarios and Murray, 2016; Radev et al.

T. (2024). Consistency models for scalable and fast simulation-based inference. p. 0
Where the paper says it

1 Introduction Simulation-based inference (SBI; Cranmer et al. , 2020) is a powerful framework for inferring parameters of scientific models whose likelihood functions are unavailable or computationally prohibitive to evaluate, but for which simulating data is straightforward.

Tejero-Cantero, A. p. 6
Where the paper says it

3 below, these assumptions hold for Masked Autoregressive Flows (MAFs) (Papamakarios et al. , 2017) and coupling flows, the default density estimators in the sbi (Tejero-Cantero et al.

, Boelts, J., Deistler, M., Lueckmann, J. p. 0
Where the paper says it

As many scientific simulators are expensive to run, generating a sufficiently large training set is often the main computational bottleneck. Prior work has sought to reduce simulation cost using sequential training schemes (Papamakarios and Murray, 2016; Lueckmann et al.

-M., Durkan, C., Gonçalves, P., Greenberg, D., and Macke, J. (2020). sbi: A toolkit for simulation-based inference. p. 0
Where the paper says it

Across benchmark SBI tasks, DRO-NPE consistently improves coverage and calibration, while narrowing the gap between empirical and population NPE loss, leading to more reliable inference in low-simulation regimes. 1 Introduction Simulation-based inference (SBI; Cranmer et al.

Utpala, S. and Rai, P. p. 4
Where the paper says it

This notion is motivated by KL-based miscalibration discrepancies previously used in probabilistic regression (Utpala and Rai, 2020; Dheur and Taieb, 2023). As with klα cov, klS cal is weaker than the full KL divergence between Pθ|x and Qθ|x by the data-processing inequality.

(2020). Quantile regularization: Towards implicit calibration of regression models. p. 4
Where the paper says it

This notion is motivated by KL-based miscalibration discrepancies previously used in probabilistic regression (Utpala and Rai, 2020; Dheur and Taieb, 2023). As with klα cov, klS cal is weaker than the full KL divergence between Pθ|x and Qθ|x by the data-processing inequality.

Vapnik, V. (1991). Principles of risk minimization for learning theory. p. 9
Where the paper says it

Finally, we evaluate DRO-NPE on the CAMELS suite (Villaescusa-Navarro et al. , 2021, 2023), a state-of-the-art cosmology dataset (dX = 39) for machine learning on simulated universes, used previously by Hikida et al.

Villaescusa-Navarro, F. p. 9
Where the paper says it

5 kl q cal Figure 4: Cosmology results: expected coverage, NLPD, and klq cal for DRO-NPE ( ), CR-NPE ( ), Bal-NPE ( ), and standard NPE ( ). Finally, we evaluate DRO-NPE on the CAMELS suite (Villaescusa-Navarro et al.

et al. (2023). The CAMELS Project: Cosmology and Astrophysics with Machine-learning Simulations. p. 9
Where the paper says it

Finally, we evaluate DRO-NPE on the CAMELS suite (Villaescusa-Navarro et al. , 2021, 2023), a state-of-the-art cosmology dataset (dX = 39) for machine learning on simulated universes, used previously by Hikida et al.

Wang, S., Wang, H., Li, X., and Honorio, J. (2025). Learning against distributional uncertainty: On the trade-off between robustness and specificity. p. 8
Where the paper says it

This is consistent with the fact that the baselines primarily encourage conservativeness, often at the expense of predictive performance. DRO-NPE also often improves on standard NPE, which we attribute to reduced generalisation error induced by distributional robustness.

Ward, D., Cannon, P., Beaumont, M., Fasiolo, M., and Schmon, S. M. (2022). Robust neural posterior estimation and statistical model criticism. p. 0
Where the paper says it

Conservative neural posterior estimation via distributionally robust training William Laplante1,3,∗ Yuga Hikida2,∗ Charita Dellaporta1 François-Xavier Briol1 Ayush Bharti2 1Department of Statistical Science, University College London, UK 2Department of Computer Science, Aalto University, Finland 3The Alan Turing Institute, UK Corresponding authors: william. hikida@aalto.

Zeng, Y. and Lam, H. (2022). Generalization bounds with minimal dependency on hypothesis class via distributionally robust optimization. p. 0
Where the paper says it

Conservative neural posterior estimation via distributionally robust training William Laplante1,3,∗ Yuga Hikida2,∗ Charita Dellaporta1 François-Xavier Briol1 Ayush Bharti2 1Department of Statistical Science, University College London, UK 2Department of Computer Science, Aalto University, Finland 3The Alan Turing Institute, UK Corresponding authors: william. hikida@aalto.

Zhao, D., Dalmasso, N., Izbicki, R., and Lee, A. B. (2021). Diagnostics for conditional density models and Bayesian inference algorithms. p. 0
Where the paper says it

, 2020) is a powerful framework for inferring parameters of scientific models whose likelihood functions are unavailable or computationally prohibitive to evaluate, but for which simulating data is straightforward. The use of flexible neural conditional density estimators has substantially expanded the applicability of SBI to challenging problems, especially in fields such as particle physics (Brehmer, 2021), cognitive neuroscience (Fengler et al.

The paper introduces distributionally robust optimisation (DRO) to minimize an upper bound on the population risk, accounting for distributional uncertainty. The strong dual formulation of DRO-NPE is exact but computationally challenging, while a regularised form provides a tractable objective that upper-bounds the population NPE risk. p. 5

Where the paper says it

Both provide upper bounds on the NPE population risk as in Theorem 4. 1; the strong dual formulation is exact but challenging to obtain numerically, while the regularised form is computationally cheaper but approximate.

This tractable objective is derived using a second-order Taylor argument and growth control of the loss function. p. 5
Where the paper says it

Focusing on the case p = 2, we therefore derive in Proposition 4. 2 a tractable finite-sample upper bound on the dual objective based on a second-order Taylor argument and growth control of the loss.

The theoretical results establish that KL-based miscoverage upper-bounds miscalibration, and miscalibration in turn upper-bounds miscoverage. The DRO objective is shown to upper-bound the population NPE risk by relying on Wasserstein concentration results which show that the true distribution P belongs to the ambiguity set A(Pn; ε) with high probability when epsilon is chosen appropriately. p. 4

Where the paper says it

, 2019), stating that for a suitable radius ε decreasing with n, the true distribution P belongs to the ambiguity set A(Pn) with high probability. This directly implies that the DRO-NPE objective upper-bounds the population NPE risk, and therefore also klα cov and klS cal; see Appendix B.

Furthermore, a tractable regularised objective is derived using Proposition 4. p. 1
Where the paper says it

Second, we propose a Wasserstein DRO objective that upper-bounds these metrics and the population NPE loss, so that, under suitable conditions, minimising it controls both posterior-fitting and uncertainty-quantification error (Theorem 4. We also derive a tractable regularised objective that upper-bounds the DRO objective (Proposition 4.

2, which yields an excess risk upper bound for NPE and for the regularised objective of DRO-NPE. p. 5
Where the paper says it

2 a tractable finite-sample upper bound on the dual objective based on a second-order Taylor argument and growth control of the loss. This yields a regularised objective well-suited for gradient-based optimisation.

Experimental details

The experiments evaluate DRO-NPE on common SBI tasks like SLCP, Lotka–Volterra, Two Moons, and Inverse Kinematics. The results show that DRO-NPE yields conservative posteriors for SLCP and near-perfect coverage for Two Moons and Inverse Kinematics. p. 8

Where the paper says it

3 for details. DRO-NPE yields conservative posteriors for SLCP and near-perfect coverage for Two Moons and Inverse Kinematics.

The selection of the regularization hyperparameter epsilon is performed by minimizing the validation estimate of klq cal, which generally yields coverage curves closest to the diagonal while maintaining competitive NLPD. p. 9
Where the paper says it

NLPD is a natural validation metric because it equals the NPE loss and upper-bounds klq cal, while absolute miscoverage directly measures deviation from nominal coverage. As shown in Figure 3(b), selecting ε via klq cal yields a coverage curve closest to the diagonal.

The code to reproduce experiments is available at https://github. p. 7
Where the paper says it

Finally, we apply DRO-NPE to a real-world cosmology problem in Section 5. The code to reproduce our experiments is available at https://github.

com/yugahikida/dro-npe. p. 7
Where the paper says it

The code to reproduce our experiments is available at https://github. com/yugahikida/dro-npe.

Additional results

The sensitivity analysis shows that increasing epsilon consistently reduces the generalisation gap across tasks and simulation budgets. The comparison with early stopping confirms that DRO-NPE remains more conservative in low-data regimes, whereas standard NPE improves both coverage and NLPD. p. 9

Where the paper says it

Figure 4 reports expected coverage, NLPD, and klq cal across five independent training runs for each method on the same fixed train–test split. Standard NPE ( ) is severely overconfident and has much worse NLPD than all conservative methods, while Bal-NPE ( ) improves NLPD but remains strongly undercovered.

A more conservative variant of DRO-NPE, obtained by modifying the selection criterion to minimize klq cal − γε˜, yields a more conservative posterior approximation. p. 9
Where the paper says it

This explains why selecting ε via absolute miscoverage can favour overly conservative posterior approximations. By contrast, klq cal is minimised at an intermediate value of ε, making it more informative for selecting ε in DRO-NPE.

The results suggest that DRO-NPE can provide conservative yet efficient posterior inference in challenging low-simulation settings. p. 4
Where the paper says it

Thus, if P were available, the NPE objective would provide a principled criterion for controlling these forms of posterior uncertainty. In finite-simulation settings, however, P is unavailable, so we instead construct a high-probability upper bound on the population risk.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Distributionally robust training of neural posterior estimation (DRO-NPE) improves uncertainty quantification by replacing standard NPE with a worst-case loss over a Wasserstein ambiguity set, which controls overfitting and reduces posterior overconfidence. This makes the method more reliable in low-simulation regimes where simulation budgets are limited.

  2. The introduction of KL-based metrics for miscoverage and miscalibration provides a principled way to quantify uncertainty errors, allowing for the selection of an appropriate regularization hyperparameter epsilon based on these metrics rather than relying on prior blindness.

  3. DRO-NPE can provide sharper bounds compared to standard NPE when the learned empirical minimiser has high functional variation relative to the oracle, meaning it can yield better risk guarantees by accounting for distributional uncertainty.

  4. The method admits a tractable regularised objective, such as the regularised form derived in Proposition 4.2, which is computationally cheaper than the strong dual formulation while still providing upper bounds on population NPE risk.

  5. For specific architectures like Masked Autoregressive Flows and coupling flows, DRO-NPE satisfies the necessary conditions for Proposition 4.2, ensuring that the training objective provides an upper bound on KL-based miscoverage and miscalibration rates relative to the oracle estimator.

  6. The method can be used to select a robust regularization hyperparameter epsilon by minimizing klq cal, which is shown in Section C.1 to yield coverage curves closest to the diagonal while maintaining competitive negative log predictive density values across various tasks and simulation budgets.

  7. By selecting epsilon using klq cal minus gamma epsilon˜, the system can be tuned to favor more conservative posteriors, allowing for a controlled trade-off between calibration and additional conservativeness in low-data regimes.

Abstract

Simulation-based inference with neural posterior estimation (NPE) often yields overconfident and unreliable posteriors under limited simulation budgets. To address this, we propose DRO-NPE, a distributionally robust approach that replaces the standard NPE objective with a worst-case loss over a Wasserstein ambiguity set. We introduce KL-based metrics for miscoverage and miscalibration, and use these to show that the DRO-NPE objective controls overfitting and reduces posterior overconfidence. Our method is tractable, parallelisable, and readily integrates with standard normalising flows. Across benchmark SBI tasks, DRO-NPE consistently improves coverage and calibration, while narrowing the gap between empirical and population NPE loss, leading to more reliable inference in low-simulation regimes.

Sources

Related papers