Generalized and Scalable Deep Gaussian Process Emulation

arXiv:2603.24538 · stat.CO · Submitted 2026-03-25 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.

Tom: We're looking at the paper: "Generalized and Scalable Deep Gaussian Process Emulation".

Jane: Here's the gist of the paper. Gaussian process (GP) emulators have become essential tools for approximating complex simulators, significantly reducing computational demands in optimization, sensitivity analysis and model calibration.

Tom: In this work, we introduce a scalable Generalized Deep Gaussian Process (GDGP) emulation framework designed to accommodate simulators with heteroskedastic Gaussian outputs and a wide range of non-Gaussian response distributions, including Poisson, negative binomial, and categorical distributions.

Jane: The GDGP framework leverages the expressiveness of DGPs and extends them to latent GP structures, enabling it to capture the complex, non-stationary behavior inherent in many simulators while also modeling non-Gaussian simulator outputs.

Tom: Framework Introduction The proposed GDGP framework generalizes the DGP formulation by introducing a likelihood layer at the top of the DGP hierarchy to explicitly model stochasticity and distributional form.

Title and authors: Tom: So we're looking at this paper today titled "Generalized and Scalable Deep Gaussian Process Emulation." It’s about taking Gaussian process emulators—which are usually pretty good for modeling continuous outputs with a certain type of noise—and making them work for way more complicated, real-world simulator outputs.

Jane: That makes sense. So we’re talking about how to handle those simulators that spit out discrete things, or things where the noise isn't just simple Gaussian stuff? It sounds like they’re trying to expand the toolkit beyond what traditional GP emulators can do.

Lu: Exactly. The paper introduces a Generalized Deep Gaussian Process emulation framework, which is designed specifically for those tricky outputs—things like Poisson counts or categorical responses—and it’s built on top of the expressiveness of deep Gaussian processes, DGPs.

Meng: But what’s the actual mechanism there? Does it just slap a layer on top, or is this a fundamental change to how we think about these simulators? Because when I look at deployment, I need to know if this adds just more complexity or if it actually solves the scaling issues we run into.

Tom: Well, they introduce a likelihood layer right at the top of the deep Gaussian process hierarchy. This allows them to explicitly model that stochasticity and whatever kind of distribution we’re dealing with, whether it's heteroskedastic Gaussian noise or something non-Gaussian like Poisson or negative binomial.

Jane: That sounds powerful because it lets you tell the model exactly how the output is supposed to be distributed. So, for someone listening who isn't a math whiz, what does that mean practically? Does it just let you model more real-world scenarios?

Lu: It means they can capture complex, non-stationary behavior inherent in many simulators because they extend the DGP structure into latent GP layers. This is key because it lets the model learn patterns that aren't just simple, fixed relationships across the entire simulation space.

Meng: Latent structures sound good for modeling complexity, but scaling is my real concern. When we’re running these emulators on large sets of input locations, how do you keep the training time from exploding? Does this framework handle big data well?

Tom: That’s where they make a big push for scalability by incorporating the Vecchia approximation. They use this to enable inference for a large number of input locations in a way that is much more practical than standard methods. It replaces sampling from GP nodes with something closed form within the Stochastic Expectation-Maximization algorithm.

Jane: Closed form expressions sound like a win, especially when you’re trying to get results fast. So, what are the actual results they show? Are these improvements just theoretical, or did they test it against something concrete?

Lu: They validated it across various likelihoods—heteroskedastic Gaussian, categorical, and count distributions. In the case of heteroskedastic Gaussian outputs, their GDGP framework substantially outperforms competing models like hetGP and bhetGP when measured by both normalized root mean square error and normalized cross prediction squared error.

Meng: Outperforming other GP models is one thing, but how much faster is it? And what about those count likelihoods? I’m used to seeing Poisson likelihoods in count data; if this framework handles negative binomial or zero-inflated outputs better, that’s where the real production value is.

Tom: They showed that for count likelihoods, their model achieved significantly lower negative log-likelihood than models that used a simple Poisson likelihood. Plus, when they applied the Vecchia approximation to GDGP, they found it keeps accuracy comparable to the full GDGP as the training size increases.

Jane: So, if I’m listening and I only care about the bottom line, what does this mean for an engineer or a scientist who needs to build a fast tool? Is this something that actually moves beyond just theoretical papers?

Lu: It's moving towards practical implementation because they provide analytical structure. For instance, in the heteroskedastic Gaussian case, they have closed-form expressions for both the predictive mean and variance under different likelihood choices like Poisson or exponential.

Meng: Closed-form means you don't have to run expensive Expectation Propagation methods just to get an estimate on the next output. That saves a lot of computation time when you’re iterating through simulations, which is exactly what we need for production work.

Tom: And they even mentioned that by exploiting some analytical structure in the heteroskedastic Gaussian case, they can get closed-form conditional updates in the imputation step, which gives them further computational gains during training.

Jane: It sounds like this paper is really about making these complex emulators usable for a wider variety of messy data and making sure they don't take all day to train when you have thousands of simulation runs.

Lu: That’s the core idea, moving from just continuous Gaussian noise to handling any kind of non-Gaussian response while maintaining scalability through techniques like the Vecchia approximation.

Meng: So, to sum up for deployment, it’s a framework that handles diverse output types and scales reasonably well using approximations that keep the complexity down for large input sets.

Tom: Precisely. The paper "Generalized and Scalable Deep Gaussian Process Emulation" shows how to build something robust enough for real simulators that don't fit the standard textbook molds.

Jane: It certainly sounds like a lot of heavy lifting was done to make this framework flexible enough to handle everything from simple Gaussian noise up through complex count distributions.

Lu: The extension into latent GP structures, as described on page three is what really gives it that flexibility to capture non-stationarity in the simulator behavior itself.

Meng: I still need more evidence on how stable these approximations are when you introduce those non-Gaussian likelihoods at scale; that’s where I worry about things breaking in production.

Tom: The authors do address that by showing the Vecchia approximation leads to posterior distribution forms with complexity O of N cubed or O of N times M cubed plus O of N times M, which is feasible even when the number of replicates is large.

Jane: Feasible for inference is one thing, but what about the limitations they state? Where does this framework stop working on its own?

Lu: They admit that while the framework handles a broad class of distributions, it still relies on specific structural assumptions within the DGP hierarchy to capture those non-stationary behaviors effectively.

Meng: So, it’s not a magic box that solves every modeling problem for every simulator; you still need to design the right DGP structure for your specific problem.

Tom: Right. But overall, this paper "Generalized and Scalable Deep Gaussian Process Emulation" provides a concrete method to tackle the challenge of non-Gaussian, complex simulators with scalable inference methods.

Jane: It certainly gives us a clear path forward for using these powerful emulators in areas where the simulation outputs are anything but simple.

The paper's summary: Tom: So, to recap, this paper is about taking those Deep Gaussian Process models—which are great for learning complex relationships in simulation data—and making them work when the output isn't just simple continuous numbers, but something discrete or messy.

Jane: Right. The main thing they’re doing is introducing this Generalized Deep Gaussian Process emulation framework to handle things like Poisson counts or categorical results, which used to be really tough for standard emulators.

Lu: They’re essentially adding a likelihood layer right on top of the deep structure, so it can explicitly model that specific type of output distribution you need. It’s about making the model smarter about what it's predicting beyond just a smooth curve.

Meng: So, if I’m thinking about production, what does that mean for how quickly I can get a reliable estimate when my simulator spits out these tricky outputs?

Tom: Well, they tackle scaling with this Stochastic Imputation framework and Vecchia approximation. That means even if you have thousands of input locations—that’s your simulation data—the model can still train efficiently without crashing.

Jane: So, they're solving two big problems at once: modeling messy outputs and keeping the training process fast enough for real-world data volumes.

Lu: Exactly. And they show that by using this structure, you can capture those non-stationary behaviors in the simulation itself while simultaneously handling these different output types. It’s about linking the simulator's internal complexity to the emulator's structure.

Tom: And when they test it, especially with heteroskedastic Gaussian outputs—which is just noise that changes across locations—their method actually beats other models like hetGP and bhetGP in terms of error metrics. That’s a solid number behind it.

Jane: So, for someone just listening, this paper means we can move past using emulators only for simple continuous outputs and start using them for the whole messy world of simulation data.

Lu: And the way they derive those closed-form expressions in some cases—like the heteroskedastic Gaussian example—that’s pretty cool because it means you don't have to use super slow methods like Expectation Propagation just for every single imputation step.

Tom: It speeds up the training process significantly, which is huge when you’re trying to calibrate a model quickly.

Jane: So, the implication here is that we’re getting a tool that doesn't just fit data; it understands *how* the simulation is behaving and what kind of messy output it’s going to produce.

Lu: That ability to handle both non-stationarity and distributional shifts in one framework is what makes this GDGP approach powerful for complex computer models.

Tom: It’s a lot of engineering put into making sure the math actually runs fast enough when you crank up the number of input locations.

Jane: And they give us a concrete tool, the open source package dgpsi, so other researchers can try out this framework themselves to see if it works for their specific kind of simulation.

Lu: It’s a solid extension of DGP theory that shows how deep learning structures can be adapted for more realistic scientific modeling tasks.

Tom: So, the bottom line is: if your simulator has complicated outputs and you need to run this emulator on big datasets, this GDGP framework gives you a path forward with better accuracy and faster training times than many of the existing options.

The paper's improvements: Tom: So, we're looking at how they suggest improving this GDGP framework for future work, and honestly, it’s just more ways to make it run smoother when you’re actually in production.

Jane: Right. The authors pointed out that while the Vecchia approximation helps with scaling the input locations, there are still avenues to explore for making the imputation steps even more efficient.

Lu: They suggest exploring those analytical structures we talked about earlier, specifically looking at how they can get closed-form conditional updates in the imputation step when you’re dealing with heteroskedastic Gaussian outputs.

Meng: Closed-form updates mean less computation per step, which is good because every extra millisecond counts when we're running thousands of simulations.

Tom: That’s right. And they mentioned that by exploiting that analytical structure, you can get faster conditional updates just in the imputation part itself, which saves time during training.

Jane: It seems like they're suggesting a hybrid approach—using the deep structure for modeling complexity and then using those mathematical shortcuts to speed up the inference mechanics.

Lu: It’s about refining the Stochastic Imputation framework, making sure that even with all these complex non-Gaussian outputs, you can leverage existing analytical forms to keep the computational cost low.

Tom: I like that focus on keeping things grounded in actual computation rather than just adding more layers of complexity.

Jane: And they also touched on the limitations of their current setup, which is important because it tells us where we still need to work.

Lu: They flag that while the framework is flexible for many distributions, you still have to carefully choose your deep Gaussian process structure to capture non-stationary behavior correctly in your specific simulator.

Meng: So they’re essentially saying: this tool is powerful, but you can't just plug and play; you still need to design the model architecture based on what you know about your simulator’s physics.

Tom: It’s a fair point. And I think the open source package dgpsi is really important because it lets us see those analytical solutions in action without having to build everything from scratch ourselves.

Jane: So, if you take away the complexity of handling non-Gaussian noise and the scaling issues with large datasets, what else does this framework enable scientists to do with their models?

Lu: It opens up possibilities for using these emulators in more challenging scientific domains where simulation outputs are anything but simple or smooth.

Tom: It means we can start doing sensitivity analysis on those complex, non-stationary computer models that used to be too slow or too hard to approximate with traditional methods.

Jane: So the future direction seems to be pushing these tools into areas of science that require more nuanced modeling than just basic continuous outputs.

Lu: Exactly. The work points toward using these deep GP techniques not just for prediction, but as a way to understand and model the underlying complex physics driving those simulations.

Conclusion: Tom: So we're wrapping up our look at "Generalized and Scalable Deep Gaussian Process Emulation," which is about using deep learning structures to handle messy, non-Gaussian simulation outputs while keeping the training fast for large datasets.

Jane: To recap, this paper shows how to build a framework that not only handles complicated output types like Poisson counts but also scales up efficiently using methods like the Vecchia approximation.

Lu: Yeah, it’s essentially a unified approach that links the complexity of deep learning representations with the mathematical tools needed for scalable inference.

Meng: I just want to stress how much they focused on making sure this actually runs without needing super massive computational resources to train it up.

Tom: They did mention that in the heteroskedastic Gaussian case, they found analytical shortcuts that give them closed-form expressions for the mean and variance, which is a big win for speed.

Jane: That means when you’re doing those imputation steps, you don't have to use slow sampling methods; you can just calculate what you need directly.

Lu: And they pointed out that while it’s flexible, the choice of the deep process structure still matters a lot for capturing those non-stationary behaviors in the simulation itself.

Meng: So, it’s not a magic solution that works for every single simulator; you still have to design your model architecture based on what you know about your specific system.

Tom: Exactly. And they provided the open source R package dgpsi so everyone can try to see those analytical solutions in action themselves.

Jane: It seems like this work is really pushing the boundary of what we can expect from GP emulators when dealing with real-world, messy scientific data instead of just textbook examples.

Lu: I think it sets a good precedent for how we might adapt these deep process ideas to other complex modeling problems where the output isn't a simple continuous thing.

Tom: Definitely. And I’m looking forward to seeing how this kind of scalability translates into faster model calibration and sensitivity analysis in the coming months.

Jane: Next week, we’ll be diving into something totally different, looking at how partitioning time in target trial emulation helps us get unbiased results in clinical trials.

Deyu Ming, Daniel Williamson

University College London · University of Exeter

stat.CO

Submitted: 2026-03-25

Updated: 2026-11-03

Journal ref: Computational Statistics & Data Analysis 227 (2027) 108485

DOI: 10.1016/j.csda.2026.108485

Code: https://github.com/mingdeyu/dgpsi-R

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 82/100

The gist: Gaussian process (GP) emulators have become essential tools for approximating complex simulators, significantly reducing computational demands in optimization…

Key concepts

Generalized Deep Gaussian Process (GDGP)
This framework modifies standard Deep GPs by adding a likelihood layer at the top. This allows it to model complex, non-Gaussian outputs such as Poisson or categorical distributions, moving beyond simple Gaussian predictions to handle diverse simulation results.
Stochastic Imputation (SI) Framework
Developed for inference, this framework treats the problem similarly to fully Bayesian methods for standard GPs. It involves prediction using a Linked GP and imputation of latent layers using an ESS-within-Gibbs sampler to estimate hidden simulator parameters.
Vecchia Approximation
This technique is used to make inference scalable for large datasets. It replaces sampling from many GP nodes with a closed-form expression applied to the log-likelihood function, significantly reducing computational complexity during training via the Stochastic Expectation-Maximization (SEM) algorithm.

Terminology

Summary

Gaussian process (GP) emulators have become essential tools for approximating complex simulators, significantly reducing computational demands in optimization, sensitivity analysis and model calibration. p. 0 In this work, we introduce a scalable Generalized Deep Gaussian Process (GDGP) emulation framework designed to accommodate simulators with heteroskedastic Gaussian outputs and a wide range of non-Gaussian response distributions, including Poisson, negative binomial, and categorical distributions. p. 0 The GDGP framework leverages the expressiveness of DGPs and extends them to latent GP structures, enabling it to capture the complex, non-stationary behavior inherent in many simulators while also modeling non-Gaussian simulator outputs. p. 0

Framework Introduction

The proposed GDGP framework generalizes the DGP formulation by introducing a likelihood layer at the top of the DGP hierarchy to explicitly model stochasticity and distributional form. This construction allows GDGP to accommodate a broad class of response distributions, including heteroskedastic, categorical, count, and zero-inflated outputs. p. 0 The framework is implemented in the open-source R package dgpsi. p. 0

Inference Methodology

Inference for GDGP was developed within the Stochastic Imputation (SI) framework, which is analogous to fully Bayesian inference for DGPs. The SI framework involves three components: prediction, imputation, and training. p. 3 Prediction leverages the linked Gaussian process (LGP) to give a closed form expression to the posterior predictive distribution. p. 3 Imputation utilizes an ESS-within-Gibbs sampler for imputing latent layers F and W(p)l from their conditional posteriors. p. 5

Scalability Enhancements

To improve practicality in large-scale settings, the authors incorporated the Vecchia approximation to enable scalable inference for large numbers of input locations. The Vecchia approximation is applied to the log-likelihood function in the Stochastic Expectation-Maximization (SEM) algorithm and replaces sampling from GP nodes with a closed form expression. p. 7 This reduces the complexity of evaluating likelihood functions and sampling from GP nodes, making training scalable via SEM. p. 7

Empirical Validation

Experiments demonstrate the efficacy of GDGP under various likelihoods, including heteroskedastic Gaussian, categorical, and count distributions. In the heteroskedastic Gaussian case, GDGP substantially outperforms competing models like hetGP and bhetGP in terms of both NRMSE and NCRPS across different replicate settings. p. 1 Furthermore, the Vecchia-approximated GDGP shows computational benefits, achieving accuracy comparable to the full GDGP as training size increases. p. 5 The framework is also effective for count likelihoods, where it achieves significantly lower Negative Log-Likelihood (NLL) than models using a Poisson likelihood. p. 6

Conclusion

In this work, we proposed a Generalized Deep Gaussian Process (GDGP) framework for emulating nonstationary computer models with non-Gaussian outputs. By introducing a likelihood layer at the top of the DGP hierarchy, the proposed framework extends DGP emulation beyond deterministic and homogeneous Gaussian settings to accommodate a broad class of response distributions. p. 2 Inference for GDGP was developed within the Stochastic Imputation (SI) framework, incorporating Vecchia approximation to enable scalable inference for large numbers of input locations. p. 0 The authors also demonstrated that in the heteroskedastic Gaussian case, additional analytical structure can be exploited to obtain closed-form conditional updates in the imputation step, yielding further computational gains. p. 9 The authors provide a careful implementation of the full GDGP framework in the open-source R package dgpsi. p. 0

References

Baker, E., Barbillon, P., Fadikar, A., Gramacy, R. p. 1 p. 1 p. 1 p. 1 p. 1 p. 1 B., Herbei, R., Higdon, D., Huang, J., Johnson, L. R., Ma, P., Mondal, A. et al. (2022), ‘Analyzing stochastic computer models: A review with opportunities’, Statistical Science 37(1), 64–89. p. 1

Binois, M. & Gramacy, R. p. 1 p. 1 p. 1 B. (2021), ‘hetgp: Heteroskedastic Gaussian process modeling and sequential design in R’, Journal of Statistical Software 98, 1–44. p. 0

Binois, M., Gramacy, R. B. & Ludkovski, M. (2018), ‘Practical heteroscedastic Gaussian process modeling for large simulation experiments’, Journal of Computational and Graphical Statistics 27(4), 808–821. p. 0

Celeux, G. & Diebolt, J. p. 5 (1985), ‘The SEM algorithm: a probabilistic teacher algorithm derived from the EM algorithm for the mixture problem’, Computational Statistics Quarterly 2, 73–82. p. 5

Chang, W., Haran, M., Applegate, P. & Pollard, D. (2016), ‘Calibrating an ice sheet model using high-dimensional binary spatial data’, Journal of the American Statistical Association 111(513), 57–72.

Cole, D. A., Gramacy, R. B. & Ludkovski, M. (2022), ‘Large-scale local surrogate modeling of stochastic simulation experiments’, Computational Statistics & Data Analysis 174, 107537. p. 0

Cooper, A., Booth, A. S. & Gramacy, R. B. (2026), ‘Modernizing full posterior inference for surrogate modeling of categorical-output simulation experiments’, Quality Engineering 38(1), 91–110. p. 3

Gelfand, A. E., Diggle, P., Guttorp, P. & Fuentes, M., eds (2010), Handbook of Spatial Statistics, 1 edn, CRC Press. p. 5

Gneiting, T. & Raftery, A. E. (2007), ‘Strictly proper scoring rules, prediction, and estimation’, Journal of the American statistical Association 102(477), 359–378.

Goldberg, P. p. 1 W., Williams, C. K. & Bishop, C. M. (1997), ‘Regression with input-dependent noise: a Gaussian process treatment’, Advances in Neural Information Processing Systems 10, 493–499.

Gramacy, R. B. & Lee, H. K. H. (2008), ‘Bayesian treed Gaussian process models with an application to computer modeling’, Journal of the American Statistical Association 103(483), 1119–1130. p. 0

Guinness, J. p. 1 (2021), ‘Gaussian process learning via Fisher scoring of Vecchia’s approximation’, Statistics and Computing 31(3), 25. p. 3

Hensman, J., Fusi, N. & Lawrence, N. D. (2013), Gaussian processes for big data, in ‘Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence’, UAI’13, AUAI Press, Arlington, Virginia, USA.

Katzfuss, M. p. 1 p. 1 p. 1 & Guinness, J. p. 1 (2021), ‘A general framework for vecchia approximations of Gaussian processes’, Statistical Science 36(1), 124–141. p. 0

Katzfuss, M., Guinness, J. p. 1 p. 1, Gong, W. & Zilber, D. (2020), ‘Vecchia approximations of Gaussian-process predictions’, Journal of Agricultural, Biological and Environmental Statistics 25, 383–414. p. 1

Katzfuss, M., Guinness, J. & Lawrence, E. (2022), ‘Scaled Vecchia approximation for fast computer-model emulation’, SIAM/ASA Journal on Uncertainty Quantification 10(2), 537–554.

Kyzyurova, K. p. 3 N., Berger, J. O. & Wolpert, R. L. (2018), ‘Coupling computer models through linking their statistical emulators’, SIAM/ASA Journal on Uncertainty Quantification 6(3), 1151–1171. p. 0

MacKay, D. J. (1992), ‘The evidence framework applied to classification networks’, Neural computation 4(5), 720–736. p. 8

Ming, D. & Guillas, S. p. 3 p. 3 (2021), ‘Linked Gaussian process emulation for systems of computer models using Matérn kernels and adaptive design’, SIAM/ASA Journal on Uncertainty Quantification. p. 0 arXiv:1912. p. 009468. In press.

Ming, D., Williamson, D. p. 0 & Guillas, S. (2023), ‘Deep Gaussian process emulation using stochastic imputation’, Technometrics 65(2), 150–161. p. 0

Murph, A. C., Strait, J. D., Moran, K. R., Hyman, J. D., Viswanathan, H. S. & Stauffer, P. H. (2024), ‘Sensitivity analysis in the presence of intrinsic stochasticity for discrete fracture network simulations’, Journal of Geophysical Research: Machine Learning and Computation 1(3), e2023JH000113. p. 0

Murray, I., Adams, R. & MacKay, D. (2010), Elliptical slice sampling, in ‘Proceedings of the thirteenth international conference on artificial intelligence and statistics’, JMLR Workshop and Conference Proceedings, pp. p. 5 541–548.

Patil, P. V., Gramacy, R. B., Carey, C. C. & Thomas, R. Q. (2025), ‘Vecchia approximated bayesian heteroskedastic Gaussian processes’, arXiv preprint arXiv:2507. p. 107815.

Pineda-Krch, M. (2008), ‘GillespieSSA: implementing the Gillespie stochastic simulation algorithm in R’, Journal of Statistical Software 25, 1–18. p. 0

Rosenzweig, M. L. & MacArthur, R. H. (1963), ‘Graphical representation and stability conditions of predator-prey interactions’, The American Naturalist 97(895), 209–223.

Salimbeni, H. p. 1 & Deisenroth, M. p. 1 (2017), Doubly stochastic variational inference for deep Gaussian processes, in ‘Advances in Neural Information Processing Systems’, pp. p. 1 4588–4599.

Salter, J. M., McKinley, T. J., Xiong, X. & Williamson, D. p. 0 B. (2025), ‘Emulating computer models with highdimensional count output’, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 383(2292). p. 2

Sauer, A., Cooper, A. p. 1 & Gramacy, R. B. (2023), ‘Vecchia-approximated deep Gaussian processes for computer experiments’, Journal of Computational and Graphical Statistics 32(3), 824–837. p. 2

Sauer, A., Gramacy, R. B. & Higdon, D. (2020), ‘Active learning for deep Gaussian process surrogates’, arXiv:2012. p. 008015.

Spiller, E. p. 1 T., Wolpert, R. L., Tierz, P. & Asher, T. G. (2023), ‘The zero problem: Gaussian process emulators for range-constrained computer models’, SIAM/ASA Journal on Uncertainty Quantification 11(2), 540–566. p. 0

Tanner, M. A. (1993), Tools for Statistical Inference: Methods for the Exploration of Posterior Distributions and Likelihood Functions, 2nd edn, Springer.

Titsias, M. p. 1 (2009), Variational learning of inducing variables in sparse Gaussian processes, in ‘Artificial Intelligence and Statistics’, pp. p. 1 567–574.

Vecchia, A. p. 0 V. (1988), ‘Estimation and model identification for continuous spatial processes’, Journal of the Royal Statistical Society Series B: Statistical Methodology 50(2), 297–312.

Williams, C. K. & Barber, D. (1998), ‘Bayesian classification with Gaussian processes’, IEEE Transactions on pattern analysis and machine intelligence 20(12), 1342–1351. p. 0

Yi, S.-r. & Taflanidis, A. p. 1 A. (2024), ‘Stochastic emulation with enhanced partial-and no-replication strategies for seismic response distribution estimation’, Earthquake Engineering & Structural Dynamics 53(7), 2354–2381. p. 0

Appendix A Closed-form Predictive Means and Variances

Let µ (q)0,k and σ 2(q)0,k denote the mean and variance of the LGP approximation, pb(f(q)0x0;fk, w(p)l), at input location x0, for q = 1..., Q. Then, under a range of parametric choices for p(yϕ), the predictive mean µ˜Y 0,k and variance (˜σ 2Y 0,k) admit closed-form expressions. p. 4 Table A.1 lists several representative distributions for which such expressions are available. p. 4

A.4 Heteroskedastic Gaussian

µ˜Y 0,k = µ(1)0,k, (˜σ 2Y 0,k)2 = exp (µ(2)0,k+σ 2(2)0,k2) + σ 2(1)0,k.

A.5 Negative Binomial

µ˜Y 0,k = exp (µ(1)0,k+σ 2(1)0,k2), (˜σ 2Y 0,k)2 = e σ 2(1)0,k-1 exp n 2µ(1)0,k+σ 2(1)0,ko + exp (µ(1)0,k+σ 2(1)0,k2) + exp (µ(2)0,k+σ 2(2)0,ko

  • 2µ(1)0,k+2σ 2(1)0,ko.

A.6 Zero-Inflated Poisson

µ˜Y 0,k = (1−π¯0,k) exp (µ(1)0,k+σ 2(1)0,k2), (˜σ 2Y 0,k)2 = (1−π¯0,k)"exp (µ(1)0,k+σ 2(1)0,ko + e σ 2(1)0,k-1 exp n 2µ(1)0,ko +¯π0,k (1−π¯0,k) exp n 2µ(1)0,k+σ 2(1)0,ko.

A.7 Zero-Inflated Negative Binomial

µ˜Y 0,k = (1−π¯0,k) exp (µ(1)0,k+σ 2(1)0,k2), (˜σ 2Y 0,k)2 = (1−π¯0,k)"exp (µ(1)0,k+σ 2(1)0,ko + exp (µ(2)0,k+σ 2(2)0,ko + e σ 2(1)0,k-1 exp n 2µ(1)0,ko +¯π0,k (1−π¯0,k) exp n 2µ(1)0,k+σ 2(1)0,ko.

Table A.1: Different choices of p(yϕ) for which predictive mean µ˜Y 0,k and variance (˜σ 2Y 0,k) admit closed-form expressions. p. 4

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Scalable inference for complex simulators: The Generalized Deep Gaussian Process (GDGP) framework, enhanced with Stochastic Imputation (SI), enables efficient inference for large numbers of input locations by incorporating Vecchia approximation to handle large training datasets while preserving computational complexity.

  2. Handling diverse output distributions: The GDGP framework accommodates a wide range of non-Gaussian response distributions, including Poisson, negative binomial, and categorical distributions, providing a unified methodology capable of addressing diverse modeling challenges beyond homogeneous Gaussian outputs.

  3. Capturing non-stationarity: By leveraging the flexibility of Deep Gaussian Processes (DGPs) within the GDGP structure, the framework can capture non-stationary behaviors inherent in many simulators while simultaneously modeling diverse response distributions.

  4. Efficient prediction via closed-form expressions: In the heteroskedastic Gaussian case, additional analytical structure allows for closed-form expressions for predictive mean and variance under various likelihood functions (Poisson, Exponential, Gamma, Heteroskedastic Gaussian), enabling direct sampling in the imputation step and avoiding expensive Expectation Propagation methods like ESS.

  5. Scalable posterior sampling: Propositions 4.3 through 4.5 provide analytical forms for the posterior distribution of mean parameters under Vecchia approximation, allowing for efficient sampling with computational complexity O(N3) or O(NM3+NM), which is feasible even when the number of replicates is large.

Sources

Related papers