Conservative neural posterior estimation via distributionally robust training
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "Conservative neural posterior estimation via distributionally robust training".
Jane: Here's the gist of the paper. Simulation-based inference with neural posterior estimation (NPE) often yields overconfident and unreliable posteriors under limited simulation budgets.
Tom: The paper proposes DRO-NPE, a distributionally robust approach that replaces the standard NPE objective with a worst-case loss over a Wasserstein ambiguity set to control overfitting and reduce posterior overconfidence.
Jane: Pruning redundant reasoning tokens The central practical limitation of NPE is the simulation budget required to train the conditional density estimator.
Tom: Prior work has sought to reduce simulation cost using sequential training schemes, multi-fidelity simulations, or by exploiting heterogeneous simulation costs.
Paper summary: Jane: So, to wrap up what we’ve heard about "Conservative neural posterior estimation via distributionally robust training," the authors are showing that you can use this distributionally robust approach to tame the overconfidence issues in neural posterior estimation when simulation budgets are small.
Tom: Exactly. The core idea is replacing the standard NPE objective with a worst-case loss over a Wasserstein ambiguity set, which essentially gives us an upper bound on the true population risk (<ref:2605.28516#pg1>).
Lu: What this means in practice is that you get statistical guarantees on excess risk and miscalibration because they’ve shown how the KL-based metrics for miscoverage and miscalibration relate to each other, upper-bounding both (<ref:2605.28516#pg3>).
Meng: For someone deploying a model, this means you can trust that the uncertainty statements are not just based on what the model thinks is most likely with limited data, but on a more conservative measure of risk across all possible distributions.
Jane: The paper demonstrates that DRO-NPE improves coverage and calibration across various tasks while narrowing the gap between empirical and population NPE loss, which leads to more reliable inference in those low simulation regimes (<ref:2605.28516#pg1>).
Tom: So, the title points to a specific mechanism—using distributionally robust training—to achieve that conservative yet efficient posterior estimation. It’s a way to ensure that when you use these neural networks for inference, you aren't blindly trusting their high confidence scores when the data is scarce.
Conclusion: Tom: So, we've been looking at how neural networks guess the true parameters from limited simulation data, and this paper introduces something called DRO-NPE, Distributionally Robust Neural Posterior Estimation.
Jane: That sounds like a mouthful for a title. Basically, they’re trying to make those neural network guesses more reliable when you don't have a ton of simulations to train on.
Tom: Right. The authors are using this distributionally robust approach to replace the standard way they measure risk. Instead of just minimizing the expected loss from their limited data, they look at the worst-case loss over a specific set of possible distributions called an ambiguity set.
Lu: What that means is they are essentially building a safety net for their uncertainty estimates. They aren't just trusting one single way the model might guess wrong; they're hedging against all plausible ways it could fail in the real world.
Meng: From my side, what I like is that instead of just saying "this model has this confidence," they are giving us a bound on how much we can actually trust that confidence when the simulation budget is low. That’s something I deal with every day when models start breaking at scale.
Jane: So, it seems the main goal here isn't to make a single perfect prediction, but to get an estimate of the uncertainty that doesn't wildly overestimate how sure the model is.
Tom: Exactly. They show that this method actually makes coverage and calibration better across all these different simulation tasks we tested, like Lotka–Volterra and Inverse Kinematics. The results are quite solid when you look at those specific benchmarks, showing conservative posters for some problems and near-perfect coverage for others.
Lu: The math is clever because they link the uncertainty quantification metrics—like KL-based miscoverage—to each other, creating a chain where one bounds the other. That’s how they get the statistical guarantees on excess risk and miscalibration you see in their results.
Jane: It sounds like this method provides a more honest picture of what we don't know when we’re working with sparse simulation data. But what does it actually change for someone just trying to understand these models?
Tom: Well, it means that for those of us using simulation-based inference, we might start deploying these neural posterior estimators with a bit more confidence in the actual risk assessment they provide, even when we're running on a tight budget.
Lu: It’s a way to get something conservative but still efficient. There is still a trade-off because it’s more computationally intensive than just standard neural posterior estimation, but the statistical bounds justify that extra work.
Jane: So the final word is that this method gives us a tool to infer parameters reliably in those low-simulation scenarios we all struggle with. Next up, we're going to look at how they actually tune the main setting for this robustness—that epsilon value.
William Laplante, Yuga Hikida, Charita Dellaporta, François-Xavier Briol, Ayush Bharti
Department of Statistical Science, University College London, UK · Department of Computer Science, Aalto University, Finland · The Alan Turing Institute
stat.ML, cs.LG
Submitted: 2026-05-27
Updated: 2026-11-04
Code: https://github.com/yugahikida/dro-npe
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 85/100
The gist: Simulation-based inference with neural posterior estimation (NPE) often yields overconfident and unreliable posteriors under limited simulation budgets.
Key concepts
- Neural Posterior Estimation (NPE)
- This is the standard method where a neural network is trained to estimate the true probability distribution from simulation data. It minimizes the expected KL divergence between the true posterior and what the neural network predicts. The problem arises when simulation budgets are small, making this estimation unreliable.
- Distributionally Robust Optimization (DRO)
- DRO is a technique used to handle uncertainty in data distributions by optimizing against the worst-case scenario within a defined set of possible distributions. In this paper, it replaces the standard NPE loss with a worst-case value over a Wasserstein ball, ensuring robustness against distribution shifts.
Terminology
Summary
Simulation-based inference with neural posterior estimation (NPE) often yields overconfident and unreliable posteriors under limited simulation budgets. p. 0 hikida@aalto. fi
Abstract
Simulation-based inference with neural posterior estimation (NPE) often yields overconfident and unreliable posteriors under limited simulation budgets. To address this, we propose
DRO-NPE, a distributionally robust approach that replaces the standard NPE objective
with a worst-case loss over a Wasserstein ambiguity set. We introduce KL-based metrics for
miscoverage and miscalibration, and use these to show that the DRO-NPE objective controls
overfitting and reduces posterior overconfidence.Where the paper says it
Where the paper says it
Pruning redundant reasoning tokens
The central practical limitation of NPE is the simulation budget required to train the conditional density estimator. Prior work has sought to reduce simulation cost using sequential training schemes, multi-fidelity simulations, or by exploiting heterogeneous simulation costs. p. 0 As many scientific simulators are expensive to run, generating a sufficiently
large training set is often the main computational bottleneck. Prior work has sought to reduce
simulation cost using sequential training schemes (Papamakarios and Murray, 2016; Lueckmann
et al. , 2025), or by exploiting heterogeneous simulation
costs (Bharti et al. However, these strategies do not directly address the challenge of
ensuring reliable uncertainty estimates in low-simulation regimes. (2022) to warn of
the potential for a “trust crisis”. Under finite simulations, NPE can yield overconfident posterior approximations whose
credible regions contain the true parameter less often than their nominal coverage suggests.Where the paper says it
Where the paper says it
Where the paper says it
How the training signal is built
The standard NPE objective is obtained by minimizing the expected KL divergence between the true posterior and the neural estimator over all possible estimators. When simulation budgets are limited, this empirical risk can be a poor proxy for population risk, leading to overfitting and poorly calibrated posteriors. p. 1 Standard NPE minimises an empirical risk over a
finite set of simulated parameter–data pairs. When the simulation
budget is limited, the empirical risk can be a poor proxy for
the population risk, leading to overfitting and poorly calibrated
posteriors (Hermans et al. We formalise DRO-NPE as a Wasserstein distributionally robust
objective. Rather than minimising the empirical NPE risk directly, we replace it by its worst-case
value over an ambiguity set A(Pn) of distributions centred at Pn (Blanchet et al.Where the paper says it
Where the paper says it
Where the gains come from
The authors introduce Kullback–Leibler (KL) divergence-based uncertainty-quantification metrics, specifically KL-based miscoverage and miscalibration, to generalize standard notions of coverage and calibration errors. They propose a Wasserstein DRO objective that upper-bounds these metrics and the population NPE loss, which controls both posterior-fitting and uncertainty-quantification error under suitable conditions. p. 1 , 2022; Talts et al. Second, we propose a Wasserstein
DRO objective that upper-bounds these metrics and the population NPE loss, so that, under
suitable conditions, minimising it controls both posterior-fitting and uncertainty-quantification
error (Theorem 4. We introduce KL-based metrics for
miscoverage and miscalibration, and use these to show that the DRO-NPE objective controls
overfitting and reduces posterior overconfidence. Our method is tractable, parallelisable, and
readily integrates with standard normalising flows.Where the paper says it
Where the paper says it
Performance across benchmarks
Across benchmark SBI tasks, DRO-NPE consistently improves coverage and calibration while narrowing the gap between empirical and population NPE loss. In experiments on Lotka–Volterra, SLCP, Two Moons, and Inverse Kinematics, DRO-NPE yields conservative posteriors for SLCP and near-perfect coverage for Two Moons and Inverse Kinematics. p. 8 3 for details. DRO-NPE yields conservative posteriors for SLCP and near-perfect coverage for Two Moons
and Inverse Kinematics. 5
kl
q
cal
Figure 4: Cosmology results: expected coverage,
NLPD, and klq
cal for DRO-NPE ( ), CR-NPE
( ), Bal-NPE ( ), and standard NPE ( ). Finally, we evaluate DRO-NPE on the
CAMELS suite (Villaescusa-Navarro et al.Where the paper says it
Where the paper says it
Analysis of design choices
The selection of hyperparameter epsilon is performed using a validation metric called KL-based miscalibration (klq cal), which is motivated by its link to entropy maximisation. Experiments show that selecting epsilon via klq cal yields coverage curves closest to the diagonal while maintaining competitive NLPD. p. 9 NLPD is a natural validation metric
because it equals the NPE loss and upper-bounds klq
cal, while absolute miscoverage directly
measures deviation from nominal coverage. As shown in Figure 3(b), selecting ε via klq
cal yields a
coverage curve closest to the diagonal. DRO-NPE is more expensive than standard NPE, but only by a
moderate constant factor. The main overhead comes from higher-order automatic differentiation
through ∇z log qϕ(θ | x), so the computational cost of this regularisation term grows with the
joint parameter–data dimension dZ.Where the paper says it
Where the paper says it
Conclusion
DRO-NPE provides a conservative yet efficient posterior inference in challenging low-simulation settings. It yields an objective that upper bounds the population NPE loss and KL-based metrics for uncertainty-quantification error, admits a tractable regularised objective, and enjoys statistical guarantees on excess risk, miscoverage, and miscalibration. p. 1 Second, we propose a Wasserstein
DRO objective that upper-bounds these metrics and the population NPE loss, so that, under
suitable conditions, minimising it controls both posterior-fitting and uncertainty-quantification
error (Theorem 4. We also derive a tractable regularised objective that upper-bounds the DRO
objective (Proposition 4. We also derive a tractable regularised objective that upper-bounds the DRO
objective (Proposition 4. 2), and verify its assumptions for standard normalising-flow architectures
used in NPE (Lemma 4. , 2024; Cabezas et al. , 2025)
post-hoc calibrate credible sets with coverage guarantees, but do not directly address poor
conditional density estimation.Where the paper says it
Where the paper says it
Where the paper says it
Acknowledgments
The authors thank Niall Jeffrey for help with using the CAMELS dataset. WL was supported by UCL’s Center for Doctoral Training in Data-Intensive Science and by the Alan Turing Institute. p. 0 Conservative neural posterior estimation via
distributionally robust training
William Laplante1,3,∗ Yuga Hikida2,∗ Charita Dellaporta1
François-Xavier Briol1 Ayush Bharti2
1Department of Statistical Science, University College London, UK
2Department of Computer Science, Aalto University, Finland
3The Alan Turing Institute, UK
Corresponding authors: william. hikida@aalto. Conservative neural posterior estimation via
distributionally robust training
William Laplante1,3,∗ Yuga Hikida2,∗ Charita Dellaporta1
François-Xavier Briol1 Ayush Bharti2
1Department of Statistical Science, University College London, UK
2Department of Computer Science, Aalto University, Finland
3The Alan Turing Institute, UK
Corresponding authors: william. hikida@aalto. By
contrast, Wasserstein balls are based on mass transport (Kuhn et al. , 2019), and can contain
distributions supported away from the observed simulations, including continuous distributions. Conservative neural posterior estimation via
distributionally robust training
William Laplante1,3,∗ Yuga Hikida2,∗ Charita Dellaporta1
François-Xavier Briol1 Ayush Bharti2
1Department of Statistical Science, University College London, UK
2Department of Computer Science, Aalto University, Finland
3The Alan Turing Institute, UK
Corresponding authors: william. hikida@aalto.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
References
Alquier, P. (2024). User-friendly introduction to PAC-Bayes bounds. p. 0 Across benchmark SBI tasks, DRO-NPE
consistently improves coverage and calibration, while narrowing the gap between empirical
and population NPE loss, leading to more reliable inference in low-simulation regimes. 1 Introduction
Simulation-based inference (SBI; Cranmer et al. 1 Introduction
Simulation-based inference (SBI; Cranmer et al. , 2020) is a powerful framework for inferring
parameters of scientific models whose likelihood functions are unavailable or computationally
prohibitive to evaluate, but for which simulating data is straightforward. Conservative neural posterior estimation via
distributionally robust training
William Laplante1,3,∗ Yuga Hikida2,∗ Charita Dellaporta1
François-Xavier Briol1 Ayush Bharti2
1Department of Statistical Science, University College London, UK
2Department of Computer Science, Aalto University, Finland
3The Alan Turing Institute, UK
Corresponding authors: william. hikida@aalto. fi
Abstract
Simulation-based inference with neural posterior estimation (NPE) often yields overconfident and unreliable posteriors under limited simulation budgets. To address this, we propose
DRO-NPE, a distributionally robust approach that replaces the standard NPE objective
with a worst-case loss over a Wasserstein ambiguity set. The goal of SBI is to approximate this posterior distribution Pθ|x0
given
observed data xo. NPE is an amortised SBI method that approximates the posterior using flexible conditional
density estimators such as mixture-density networks (Bishop, 1994) and normalising flows
(Papamakarios et al. Across benchmark SBI tasks, DRO-NPE
consistently improves coverage and calibration, while narrowing the gap between empirical
and population NPE loss, leading to more reliable inference in low-simulation regimes. 1 Introduction
Simulation-based inference (SBI; Cranmer et al. , 2020) is a powerful framework for inferring
parameters of scientific models whose likelihood functions are unavailable or computationally
prohibitive to evaluate, but for which simulating data is straightforward. The use of flexible
neural conditional density estimators has substantially expanded the applicability of SBI to
challenging problems, especially in fields such as particle physics (Brehmer, 2021), cognitive
neuroscience (Fengler et al. hikida@aalto. fi
Abstract
Simulation-based inference with neural posterior estimation (NPE) often yields overconfident and unreliable posteriors under limited simulation budgets. By contrast, conservative posteriors are often safer as they widen
uncertainty statements rather than overstating confidence. This has motivated several works on
conservative uncertainty quantification in neural SBI: Delaunoy et al. By contrast, conservative posteriors are often safer as they widen
uncertainty statements rather than overstating confidence. This has motivated several works on
conservative uncertainty quantification in neural SBI: Delaunoy et al. simulations can exhibit poor uncertainty quantification, leading Hermans et al. (2022) to warn of
the potential for a “trust crisis”. Instead
of regularising standard NPE, Delaunoy et al. (2024) use Bayesian neural networks for conservative
inference, at the cost of Bayesian inference over network weights. Instead
of regularising standard NPE, Delaunoy et al. (2024) use Bayesian neural networks for conservative
inference, at the cost of Bayesian inference over network weights. Provided qϕmin (θi
| xi) > 0,
this implies ∇zqϕmin (θi
| xi) = 0 ∀i = 1, . Thus, for large ε, the objective favours
conditional densities that vary less sharply in z at the simulated pairs. Other diagnostics have also been studied (Zhao et al. , 2021; Linhart
et al. Other diagnostics have also been studied (Zhao et al. , 2021; Linhart
et al. , 2020) to inflate estimator uncertainty,
and Delaunoy et al. (2023) extended this idea to contrastive NRE (Miller et al. hikida@aalto. fi
Abstract
Simulation-based inference with neural posterior estimation (NPE) often yields overconfident and unreliable posteriors under limited simulation budgets. 1 Introduction
Simulation-based inference (SBI; Cranmer et al. , 2020) is a powerful framework for inferring
parameters of scientific models whose likelihood functions are unavailable or computationally
prohibitive to evaluate, but for which simulating data is straightforward. , 2022) and NPE. Falkiewicz et al. By contrast, conservative posteriors are often safer as they widen
uncertainty statements rather than overstating confidence. This has motivated several works on
conservative uncertainty quantification in neural SBI: Delaunoy et al. hikida@aalto. fi
Abstract
Simulation-based inference with neural posterior estimation (NPE) often yields overconfident and unreliable posteriors under limited simulation budgets. , 2020) is a powerful framework for inferring
parameters of scientific models whose likelihood functions are unavailable or computationally
prohibitive to evaluate, but for which simulating data is straightforward. The use of flexible
neural conditional density estimators has substantially expanded the applicability of SBI to
challenging problems, especially in fields such as particle physics (Brehmer, 2021), cognitive
neuroscience (Fengler et al. , 2020) is a powerful framework for inferring
parameters of scientific models whose likelihood functions are unavailable or computationally
prohibitive to evaluate, but for which simulating data is straightforward. The use of flexible
neural conditional density estimators has substantially expanded the applicability of SBI to
challenging problems, especially in fields such as particle physics (Brehmer, 2021), cognitive
neuroscience (Fengler et al. We also derive a tractable regularised objective that upper-bounds the DRO
objective (Proposition 4. 2), and verify its assumptions for standard normalising-flow architectures
used in NPE (Lemma 4. , 2019), multi-fidelity simulations (Hikida et al. , 2025; Krouglova
et al. In practice, however, SBI is often performed on
handcrafted or learned summary statistics rather than raw observations (Radev et al. , 2022;
Deistler et al. In practice, however, SBI is often performed on
handcrafted or learned summary statistics rather than raw observations (Radev et al. , 2022;
Deistler et al. 1 Introduction
Simulation-based inference (SBI; Cranmer et al. , 2020) is a powerful framework for inferring
parameters of scientific models whose likelihood functions are unavailable or computationally
prohibitive to evaluate, but for which simulating data is straightforward. 1 is a high-probability concentration radius ensuring P ∈ Ap(Pn; ε), but it depends
on unknown constants and is not directly usable. We therefore follow common DRO practice
and treat ε as a hyperparameter selected using a validation set (Mohajerin Esfahani and Kuhn,
2018). Finally, we evaluate DRO-NPE on the
CAMELS suite (Villaescusa-Navarro et al. ,
2021, 2023), a state-of-the-art cosmology
dataset (dX = 39) for machine learning on
simulated universes, used previously by Hikida
et al. The function S allows SBC to be applied in higher-dimensional
settings. For example, S(θ; x) = ∥θ − θ0∥2 for a reference point θ0 is used in the BayesFlow
library (Kühmichel et al. (2024) for a recent review. In this work,
we focus on the widely used neural posterior estimation (NPE) method (Papamakarios and
Murray, 2016; Radev et al. Instead
of regularising standard NPE, Delaunoy et al. (2024) use Bayesian neural networks for conservative
inference, at the cost of Bayesian inference over network weights. We use 1,000 high-fidelity baryonic hydrodynamic simulations available to us, reserving
900 for training and 100 for testing. Conservative inference is particularly important in
this setting as simulations are scarce, and the
observables depend not only on Ωm and σ8, but also on nuisance astrophysical processes that
can produce similar simulated data (Villaescusa-Navarro et al. Our method is tractable, parallelisable, and
readily integrates with standard normalising flows. Across benchmark SBI tasks, DRO-NPE
consistently improves coverage and calibration, while narrowing the gap between empirical
and population NPE loss, leading to more reliable inference in low-simulation regimes. Assuming Pθ|x is absolutely continuous with respect to Qθ|x, the NPE
objective is obtained by minimising the expected KL divergence between Pθ|x and Qθ|x over ϕ:
Ex∼Px
h
KL
Pθ|x ∥ Qθ|x
i +K(P)
= E(θ,x)∼P [− log qϕ(θ | x)] := LNPE(qϕ; P), (1)
where z = (x, θ), and the additive constant K(P) does not depend on ϕ. Note that LNPE(qϕ; P)
can be thought of as a population risk, as it is defined as an expectation under P, induced by
the pointwise loss ℓNPE(qϕ; z) := − log qϕ(θ | x). As many scientific simulators are expensive to run, generating a sufficiently
large training set is often the main computational bottleneck. Prior work has sought to reduce
simulation cost using sequential training schemes (Papamakarios and Murray, 2016; Lueckmann
et al. Prior work has sought to reduce
simulation cost using sequential training schemes (Papamakarios and Murray, 2016; Lueckmann
et al. , 2017; Greenberg et al. Prior work has sought to reduce
simulation cost using sequential training schemes (Papamakarios and Murray, 2016; Lueckmann
et al. , 2017; Greenberg et al. , 2020) is a powerful framework for inferring
parameters of scientific models whose likelihood functions are unavailable or computationally
prohibitive to evaluate, but for which simulating data is straightforward. The use of flexible
neural conditional density estimators has substantially expanded the applicability of SBI to
challenging problems, especially in fields such as particle physics (Brehmer, 2021), cognitive
neuroscience (Fengler et al. (2024) use Bayesian neural networks for conservative
inference, at the cost of Bayesian inference over network weights. Complementarily, conformal
and Neyman-inversion methods (Masserano et al. This has motivated several works on
conservative uncertainty quantification in neural SBI: Delaunoy et al. (2022) proposed regularised
balanced neural ratio estimation (NRE; Hermans et al. , 2018; Säilynoja et al. ,
2022; Modrák et al. (b) Negative log predictive density (NLPD) on held-out test data. (c) KL-based calibration
metric klq
cal used for selecting ε. , 2018; Gneiting et al. For the pushforward
distributions (F
Q
S|x
)#PS|x and (F
Q
S|x
)#QS|x, where the latter is uniformly distributed under
mild assumptions, this deviation is often measured via Kolmogorov–Smirnov distances or via
Cramér–von Mises criteria (Rossi and Sekhposyan, 2019; Zhao et al. Provided qϕmin (θi
| xi) > 0,
this implies ∇zqϕmin (θi
| xi) = 0 ∀i = 1, . Thus, for large ε, the objective favours
conditional densities that vary less sharply in z at the simulated pairs. , 2022) and simulation-based calibration (Talts et al. , 2018; Säilynoja et al. (2026): SLCP (dX = 8, dΘ = 5),
Lotka–Volterra (dX = 20, dΘ = 4), Two Moons (dX = 2, dΘ = 2), and Inverse Kinematics
(dX = 2, dΘ = 4). Evaluation is based on three metrics: (i) coverage plots, which show empirical
versus nominal coverage, (ii) negative log predictive density (NLPD), which is the average negative
log-density assigned by the learned posterior to the true parameter on held-out test points, and
8 , 2026; Tatsuoka et al. , 2025; Saoulis et al. , 2024) and cosmology (Alsing et al. ,
2018; Jeffrey et al. Finally, we evaluate DRO-NPE on the
CAMELS suite (Villaescusa-Navarro et al. ,
2021, 2023), a state-of-the-art cosmology
dataset (dX = 39) for machine learning on
simulated universes, used previously by Hikida
et al. (2024) for a recent review. In this work,
we focus on the widely used neural posterior estimation (NPE) method (Papamakarios and
Murray, 2016; Radev et al. 1 Introduction
Simulation-based inference (SBI; Cranmer et al. , 2020) is a powerful framework for inferring
parameters of scientific models whose likelihood functions are unavailable or computationally
prohibitive to evaluate, but for which simulating data is straightforward. 3 below, these assumptions
hold for Masked Autoregressive Flows (MAFs) (Papamakarios et al. , 2017) and coupling flows,
the default density estimators in the sbi (Tejero-Cantero et al. As many scientific simulators are expensive to run, generating a sufficiently
large training set is often the main computational bottleneck. Prior work has sought to reduce
simulation cost using sequential training schemes (Papamakarios and Murray, 2016; Lueckmann
et al. Across benchmark SBI tasks, DRO-NPE
consistently improves coverage and calibration, while narrowing the gap between empirical
and population NPE loss, leading to more reliable inference in low-simulation regimes. 1 Introduction
Simulation-based inference (SBI; Cranmer et al. This notion is motivated by KL-based miscalibration discrepancies previously used in probabilistic regression (Utpala and Rai, 2020; Dheur and Taieb, 2023). As with klα
cov, klS
cal is weaker
than the full KL divergence between Pθ|x and Qθ|x by the data-processing inequality. This notion is motivated by KL-based miscalibration discrepancies previously used in probabilistic regression (Utpala and Rai, 2020; Dheur and Taieb, 2023). As with klα
cov, klS
cal is weaker
than the full KL divergence between Pθ|x and Qθ|x by the data-processing inequality. Finally, we evaluate DRO-NPE on the
CAMELS suite (Villaescusa-Navarro et al. ,
2021, 2023), a state-of-the-art cosmology
dataset (dX = 39) for machine learning on
simulated universes, used previously by Hikida
et al. 5
kl
q
cal
Figure 4: Cosmology results: expected coverage,
NLPD, and klq
cal for DRO-NPE ( ), CR-NPE
( ), Bal-NPE ( ), and standard NPE ( ). Finally, we evaluate DRO-NPE on the
CAMELS suite (Villaescusa-Navarro et al. Finally, we evaluate DRO-NPE on the
CAMELS suite (Villaescusa-Navarro et al. ,
2021, 2023), a state-of-the-art cosmology
dataset (dX = 39) for machine learning on
simulated universes, used previously by Hikida
et al. This is consistent with the fact that the baselines primarily encourage conservativeness, often
at the expense of predictive performance. DRO-NPE also often improves on standard NPE,
which we attribute to reduced generalisation error induced by distributional robustness. Conservative neural posterior estimation via
distributionally robust training
William Laplante1,3,∗ Yuga Hikida2,∗ Charita Dellaporta1
François-Xavier Briol1 Ayush Bharti2
1Department of Statistical Science, University College London, UK
2Department of Computer Science, Aalto University, Finland
3The Alan Turing Institute, UK
Corresponding authors: william. hikida@aalto. Conservative neural posterior estimation via
distributionally robust training
William Laplante1,3,∗ Yuga Hikida2,∗ Charita Dellaporta1
François-Xavier Briol1 Ayush Bharti2
1Department of Statistical Science, University College London, UK
2Department of Computer Science, Aalto University, Finland
3The Alan Turing Institute, UK
Corresponding authors: william. hikida@aalto. , 2020) is a powerful framework for inferring
parameters of scientific models whose likelihood functions are unavailable or computationally
prohibitive to evaluate, but for which simulating data is straightforward. The use of flexible
neural conditional density estimators has substantially expanded the applicability of SBI to
challenging problems, especially in fields such as particle physics (Brehmer, 2021), cognitive
neuroscience (Fengler et al.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
The paper introduces distributionally robust optimisation (DRO) to minimize an upper bound on the population risk, accounting for distributional uncertainty. The strong dual formulation of DRO-NPE is exact but computationally challenging, while a regularised form provides a tractable objective that upper-bounds the population NPE risk. p. 5 Both provide upper
bounds on the NPE population risk as in Theorem 4. 1; the strong dual formulation is exact but
challenging to obtain numerically, while the regularised form is computationally cheaper but
approximate. Focusing on the case p = 2, we therefore derive in Proposition 4. 2 a tractable
finite-sample upper bound on the dual objective based on a second-order Taylor argument and
growth control of the loss.Where the paper says it
Where the paper says it
The theoretical results establish that KL-based miscoverage upper-bounds miscalibration, and miscalibration in turn upper-bounds miscoverage. The DRO objective is shown to upper-bound the population NPE risk by relying on Wasserstein concentration results which show that the true distribution P belongs to the ambiguity set A(Pn; ε) with high probability when epsilon is chosen appropriately. p. 4 , 2019), stating that for a suitable radius ε decreasing with
n, the true distribution P belongs to the ambiguity set A(Pn) with high probability. This directly
implies that the DRO-NPE objective upper-bounds the population NPE risk, and therefore also
klα
cov and klS
cal; see Appendix B. Second, we propose a Wasserstein
DRO objective that upper-bounds these metrics and the population NPE loss, so that, under
suitable conditions, minimising it controls both posterior-fitting and uncertainty-quantification
error (Theorem 4. We also derive a tractable regularised objective that upper-bounds the DRO
objective (Proposition 4. 2 a tractable
finite-sample upper bound on the dual objective based on a second-order Taylor argument and
growth control of the loss. This yields a regularised objective well-suited for gradient-based
optimisation.Where the paper says it
Where the paper says it
Where the paper says it
Experimental details
The experiments evaluate DRO-NPE on common SBI tasks like SLCP, Lotka–Volterra, Two Moons, and Inverse Kinematics. The results show that DRO-NPE yields conservative posteriors for SLCP and near-perfect coverage for Two Moons and Inverse Kinematics. p. 8 3 for details. DRO-NPE yields conservative posteriors for SLCP and near-perfect coverage for Two Moons
and Inverse Kinematics. NLPD is a natural validation metric
because it equals the NPE loss and upper-bounds klq
cal, while absolute miscoverage directly
measures deviation from nominal coverage. As shown in Figure 3(b), selecting ε via klq
cal yields a
coverage curve closest to the diagonal. Finally, we apply DRO-NPE to a real-world cosmology problem in Section 5. The code to
reproduce our experiments is available at https://github. The code to
reproduce our experiments is available at https://github. com/yugahikida/dro-npe.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Additional results
The sensitivity analysis shows that increasing epsilon consistently reduces the generalisation gap across tasks and simulation budgets. The comparison with early stopping confirms that DRO-NPE remains more conservative in low-data regimes, whereas standard NPE improves both coverage and NLPD. p. 9 Figure 4 reports expected coverage, NLPD, and klq
cal across five independent training runs for
each method on the same fixed train–test split. Standard NPE ( ) is severely overconfident and
has much worse NLPD than all conservative methods, while Bal-NPE ( ) improves NLPD but
remains strongly undercovered. This explains why
selecting ε via absolute miscoverage can favour overly conservative posterior approximations. By contrast, klq
cal is minimised at an intermediate value of ε, making it more informative for
selecting ε in DRO-NPE. Thus, if P were available, the NPE
objective would provide a principled criterion for controlling these forms of posterior uncertainty. In finite-simulation settings, however, P is unavailable, so we instead construct a high-probability
upper bound on the population risk.Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Distributionally robust training of neural posterior estimation (DRO-NPE) improves uncertainty quantification by replacing standard NPE with a worst-case loss over a Wasserstein ambiguity set, which controls overfitting and reduces posterior overconfidence. This makes the method more reliable in low-simulation regimes where simulation budgets are limited.
-
The introduction of KL-based metrics for miscoverage and miscalibration provides a principled way to quantify uncertainty errors, allowing for the selection of an appropriate regularization hyperparameter epsilon based on these metrics rather than relying on prior blindness.
-
DRO-NPE can provide sharper bounds compared to standard NPE when the learned empirical minimiser has high functional variation relative to the oracle, meaning it can yield better risk guarantees by accounting for distributional uncertainty.
-
The method admits a tractable regularised objective, such as the regularised form derived in Proposition 4.2, which is computationally cheaper than the strong dual formulation while still providing upper bounds on population NPE risk.
-
For specific architectures like Masked Autoregressive Flows and coupling flows, DRO-NPE satisfies the necessary conditions for Proposition 4.2, ensuring that the training objective provides an upper bound on KL-based miscoverage and miscalibration rates relative to the oracle estimator.
-
The method can be used to select a robust regularization hyperparameter epsilon by minimizing klq cal, which is shown in Section C.1 to yield coverage curves closest to the diagonal while maintaining competitive negative log predictive density values across various tasks and simulation budgets.
-
By selecting epsilon using klq cal minus gamma epsilon˜, the system can be tuned to favor more conservative posteriors, allowing for a controlled trade-off between calibration and additional conservativeness in low-data regimes.
Abstract
Simulation-based inference with neural posterior estimation (NPE) often yields overconfident and unreliable posteriors under limited simulation budgets. To address this, we propose DRO-NPE, a distributionally robust approach that replaces the standard NPE objective with a worst-case loss over a Wasserstein ambiguity set. We introduce KL-based metrics for miscoverage and miscalibration, and use these to show that the DRO-NPE objective controls overfitting and reduces posterior overconfidence. Our method is tractable, parallelisable, and readily integrates with standard normalising flows. Across benchmark SBI tasks, DRO-NPE consistently improves coverage and calibration, while narrowing the gap between empirical and population NPE loss, leading to more reliable inference in low-simulation regimes.
Sources
- Amortised and provably-robust simulation-based inference
- Distributionally Robust Optimization and Robust Statistics
- CP4SBI: Local Conformal Calibration of Credible Sets in Simulation-Based Inference
- Simulation-Based Inference: A Practical Guide
- Low-Budget Simulation-Based Inference with Bayesian Neural Networks
- Benchmarking Invertible Architectures on Inverse Problems
- BayesFlow 2: Multi-Backend Amortized Bayesian Inference in Python
- Decoupled Weight Decay Regularization
- Transfer learning for multifidelity simulation-based inference in cosmology
- On Regularization via Early Stopping for Least Squares Regression
- Validating Bayesian Inference Algorithms with Simulation-Based Calibration
- Bifidelity Parameter Estimation Using Conditional Diffusion Models
- Quantile Regularization: Towards Implicit Calibration of Regression Models
- Neural Methods for Amortized Inference
Related papers
- Uniform-in-Time Weak Propagation-of-Chaos in Shallow Neural Networks
- Diffusion Models Observe Only Gradients: A Geometric Perspective on Score Matching Errors
- Quantifying Error Propagation and Model Collapse in Diffusion Models
- Behavior of prediction performance metrics with rare events
- Statistical Inference and Learning for Shapley Additive Explanations (SHAP)
- Control Variate Design for Score-Function Gradient Estimators in Importance-Weighted Variational Inference