All Emulators are Wrong, Many are Useful, and Some are More Useful Than Others: A Reproducible Comparison of Computer Model Surrogates
stat.CO, stat.ML
Submitted: 2025-12-09
Updated: 2026-08-30
Code: https://github.com/knrumsey/spareGParts
Project page: https://garthtarr.github.io/BayesianLasso
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Accurate and efficient surrogate modeling is essential for modern computational science, and there are a staggering number of emulation methods to choose from.
Terminology
Abstract
Accurate and efficient surrogate modeling is essential for modern computational science, and there are a staggering number of emulation methods to choose from. With new methods being developed all the time, comparing the relative strengths and weaknesses of different methods remains a challenge due to inconsistent benchmarking practices and (sometimes) limited reproducibility and transparency. In this work, we present a large-scale, fully reproducible comparison of 29 distinct emulators across 60 canonical test functions and 40 real emulation datasets. To facilitate rigorous, apples-to-apples comparisons, we introduce the R package duqling, which streamlines reproducible simulation studies using a consistent, simple syntax, and automatic internal scaling of inputs. This framework allows researchers to compare emulators in a unified environment and makes it possible to replicate or extend previous studies with minimal effort, even across different publications. Our results provide detailed empirical insight into the strengths and weaknesses of state-of-the-art emulators and offer guidance for both method developers and practitioners selecting a surrogate for new data. We discuss best practices for emulator comparison and highlight how duqling can accelerate research in emulator design and application.
Sources
- Estimating Shapley Effects in Big-Data Emulation and Regression Settings using Bayesian Additive Regression Trees
- Generative Bayesian Computation as a Scalable Alternative to Gaussian Process Surrogates
- Nonstationary Gaussian Process Surrogates
Related papers
- A fast non-reversible sampler for Bayesian mixture models
- BKP: An R Package for Beta Kernel Process Modeling
- Edgewise Envelopes Between Balanced Forman and Ollivier-Ricci Curvature
- Amortized quadrature for posterior expectations in inverse problems
- A Non-asymptotic Analysis for Learning and Applying a Preconditioner in MCMC
- Prob-GParareal: A Probabilistic Numerical Parallel-in-Time Solver for Differential Equations