Stochastic Derivative Estimation for Discontinuous Sample Performances: A Leibniz Integration Perspective

arXiv:2511.00006 · stat.ME, math.PR, stat.CO · Submitted 2025-10-09 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.

Tom: We're looking at the paper: "Stochastic Derivative Estimation for Discontinuous Sample Performances".

Jane: Here's the gist of the paper. Stochastic derivative estimation for discontinuous sample performances based on multidimensional Leibniz rules provides novel estimators that handle discontinuities arising from indicator functions and general structural parameters by generalizing existing methods like generalized likelihood ratio (GLR) and push-out likelihood ratio (LR) techniques Framework Development The paper develops a novel stochastic derivative estimation framework for sample performance functions that are discontinuous in the parameter of interest, based on the multidimensional Leibniz integral rule When discontinuities arise from indicator functions, the authors embed these indicator functions into the sample space to yield a continuous performance function over a parameter-dependent domain For general discontinuous functions, they apply a change of variables to shift parameter dependence into the sample space and the underlying probability measure This leads to two terms from applying the Leibniz integral rule: a standard likelihood ratio (LR) term and a surface integral from differentiating the boundary of the domain The framework generalizes GLR and provides intuition on when the surface integral vanishes, enabling single-run estimators.

Tom: Leibniz Rules in Rn The paper introduces the Leibniz integral rule and the Leibniz divergence rule, illustrating their applications to stochastic derivative estimation through two simple examples The general Leibniz integral rule is stated as: d/dθ ∫ Dtheta G(x, θ)dx = ∫ ∂Dtheta G(x, θ) (v®(x, θ) · ®n(x, θ)) dx + ∫ Dtheta ∂θ G(x, θ)dx The Leibniz divergence rule is an equivalent form obtained by applying the divergence theorem: d/dθ ∫ Dtheta G(x, θ)dx = ∫ Dtheta (div(G(x, θ)®v(x, θ)) + ∂θ G(x, θ)) dx The distinction between the two forms is crucial for stochastic derivative estimation as the divergence rule avoids surface integrals which may require multiple sample paths.

Jane: Handling Indicator Discontinuities For sample performances with discontinuities arising from indicator functions, the authors show that applying the Leibniz divergence rule (2) yields a novel single-run unbiased derivative estimator This is achieved by absorbing indicator functions directly into the integration domain, leaving a differentiable integrand In practice, this involves constructing a differentiable mapping h over a set V to parametrize the restricted domain The resulting Leibniz divergence estimator is given by equation (6), which combines the standard IPA-LR term with a divergence term that captures the effect of discontinuities introduced by the indicator function.

Tom: Push-out and General Discontinuities For general discontinuous sample performances admitting a change of variables, the authors propose a framework combining push-out LR with the Leibniz integral rule This approach applies the Leibniz integral rule (1) to avoid differentiation with respect to the input, using smooth approximations and reversing the change of variables via Proposition 2 The final result is expressed in terms of a volume integral and a surface integral This framework generalizes GLR methods but requires simpler, more easily verifiable conditions for the surface integral to vanish, facilitating single-run estimators.

Title and authors: Tom: Okay, so we've seen that they are using the Leibniz integral rule to tackle discontinuities in sample performance functions. Jane, can you explain what the paper is actually proposing as the main solution?

Jane: The paper proposes a unified framework for stochastic derivative estimation using this rule. When it comes to indicator functions causing those breaks, they embed them into the sample space so you get a continuous function over the parameter-dependent domain, and applying that Leibniz integral rule gives you a single-run, unbiased derivative estimator <ref:2511.00006#pg1>.

Tom: So it’s about turning something messy—discontinuous—into something manageable by redefining the space where you integrate. What about the general cases, not just indicator functions?

Jane: For general discontinuous functions, they use a change of variables to move the parameter dependence into the sample space and then into the probability measure itself. This leads to two specific terms from applying that Leibniz integral rule: a standard likelihood ratio term and another term that looks like a surface integral from differentiating the boundary of that new domain.

Lu: That surface integral is key because it gives us intuition about when we can actually get away with just one sample path instead of needing multiple, which is what methods like generalized likelihood ratio often require <ref:2511.00006#pg3>.

Meng: If that surface integral vanishes under certain conditions, that means we can deploy an estimator that's much faster to run and more stable at scale because it avoids those extra simulations.

Lalam: The paper shows how this general framework combines push-out likelihood ratio with the Leibniz integral rule to handle general discontinuities <ref:2511.00006#pg3>.

The paper's summary: Tom: So, summarizing the main idea again, they are using the Leibniz integral rule to find derivatives for sample performance functions that have discontinuities. Jane, how does this compare to what we usually do with methods like IPA or likelihood ratio?

Jane: Traditional methods like infinitesimal perturbation analysis or likelihood ratio estimate derivatives by differentiating the underlying probability measure instead of the sample performance itself <ref:2511.00006#pg2>. But those methods often fail when you have distributional parameters or discontinuous sample performance functions, which is where this new approach steps in.

Tom: So, it handles things IPA can't? That sounds promising for real-world modeling where the underlying structure isn't perfectly smooth.

Lu: Exactly. They show that by using the Leibniz integral rule correctly, you get differentiation under the integral sign when both the integrand and the integration domain depend on your parameter, which gives us more flexibility to design new estimators <ref:2511.00006#pg3>.

Meng: Flexibility is good, but what does it cost? If this framework requires more complex setup than standard methods, it might not be deployable in a real production environment where we need something fast.

Lalam: The paper illustrates this by showing how the general Leibniz integral rule has two forms: one that involves surface integrals and another version obtained by applying the divergence theorem, which avoids those surface integrals entirely <ref:2511.00006#pg3>.

The paper's improvements: Tom: So, moving on to what they improved. They showed that for indicator functions causing discontinuities, this method yields a single-run unbiased estimator using the Leibniz divergence rule <ref:2511.00006#pg1>. That’s a big win if we can get around the need for multiple samples.

Jane: It’s a big win because they embed those indicator functions directly into the domain, which leaves them with a differentiable integrand, and that specific estimator is unbiased <ref:2511.00006#pg1>. They call this the Leibniz divergence estimator, which combines a standard likelihood ratio term with this new divergence term to capture the discontinuity effect.

Lu: And for the general discontinuous functions using change of variables, they propose a framework that combines push-out LR with the Leibniz integral rule <ref:2511.00006#pg3>. This allows them to express the result in terms of a volume integral and that surface integral we mentioned earlier.

Meng: I’m interested in this because if they can find simpler, more verifiable conditions for when that surface integral is zero, then we can deploy single-run estimators more reliably without needing heavy simulations.

Lalam: The paper also shows extensions for hyperrectangle support, where the surface integral turns into a sum of conditional expectations over faces <ref:2511.00006#pg3>. This gives them an unbiased estimator when you're dealing with those bounded inputs.

Conclusion: Tom: So we’ve seen they have methods for both indicator discontinuities and general ones, aiming for single-run estimates by controlling that surface integral. Jane, what’s the final word on the implications of this work?

Jane: The main implication is that it gives us a robust way to handle sample performance functions that aren't perfectly smooth, which is very common in many real-world applications. It extends existing techniques to a much broader range of problems <ref:2511.00006#pg3>.

Lu: I think the most significant part is the theoretical flexibility they provide in choosing how you set up the integration domain and applying these rules, which opens doors for designing entirely new estimation techniques.

Meng: From my side, if this method can be made numerically stable across different joint distributions, it means we might actually be able to use these estimators on real traffic data without worrying about them breaking down at scale <ref:2511.00006#pg3>.

Lalam: I’m confident that the Leibniz divergence estimator is numerically stable across various joint distributions, and in cases where other methods fail, like with certain shape parameters in Gamma marginals, it proves to be more robust <ref:2511.00006#pg1>.

Tom: So to wrap up on "Stochastic Derivative Estimation for Discontinuous Sample Performances: A Leibniz Integration Perspective", they’ve given us a new mathematical tool based on the Leibniz integral rule that tackles sample performance discontinuities by offering single-run unbiased estimators under specific conditions. Jane, Lu, Meng, Lalam, thanks for breaking that down with us.

University of Maryland · University of Montreal

stat.ME, math.PR, stat.CO

Submitted: 2025-10-09

Updated: 2026-11-04

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

Importance score: 83/100

The gist: Stochastic derivative estimation for discontinuous sample performances based on multidimensional Leibniz rules provides novel estimators that handle discontinuities arising from indicator functions…

Key concepts

Leibniz Integral Rule
This rule provides a mathematical way to differentiate an integral with respect to a parameter inside the integral. The paper uses two forms: one involving a standard surface integral and another involving a divergence term. The divergence form is preferred because it can avoid complex surface integrals, making estimation easier.
Indicator Functions
These are functions that return 1 if a condition is met and 0 otherwise, often causing discontinuities in sample performance. The authors address these by embedding them into the integration domain using a change of variables, effectively turning the discontinuous function into one that is continuous over the parameter space.
Push-out Likelihood Ratio (LR)
This technique is used to handle general discontinuities by shifting parameter dependence from the sample performance function into a different probability measure. Combining this with Leibniz rules allows for a flexible framework that can generalize standard likelihood ratio methods while providing better conditions for single-run estimators.

Terminology

Summary

Stochastic derivative estimation for discontinuous sample performances based on multidimensional Leibniz rules provides novel estimators that handle discontinuities arising from indicator functions and general structural parameters by generalizing existing methods like generalized likelihood ratio (GLR) and push-out likelihood ratio (LR) techniques

Framework Development

The paper develops a novel stochastic derivative estimation framework for sample performance functions that are discontinuous in the parameter of interest, based on the multidimensional Leibniz integral rule When discontinuities arise from indicator functions, the authors embed these indicator functions into the sample space to yield a continuous performance function over a parameter-dependent domain For general discontinuous functions, they apply a change of variables to shift parameter dependence into the sample space and the underlying probability measure This leads to two terms from applying the Leibniz integral rule: a standard likelihood ratio (LR) term and a surface integral from differentiating the boundary of the domain The framework generalizes GLR and provides intuition on when the surface integral vanishes, enabling single-run estimators.

Leibniz Rules in Rn

The paper introduces the Leibniz integral rule and the Leibniz divergence rule, illustrating their applications to stochastic derivative estimation through two simple examples The general Leibniz integral rule is stated as: d/dθ ∫ Dtheta G(x, θ)dx = ∫ ∂Dtheta G(x, θ) (v®(x, θ) · ®n(x, θ)) dx + ∫ Dtheta ∂θ G(x, θ)dx The Leibniz divergence rule is an equivalent form obtained by applying the divergence theorem: d/dθ ∫ Dtheta G(x, θ)dx = ∫ Dtheta (div(G(x, θ)®v(x, θ)) + ∂θ G(x, θ)) dx The distinction between the two forms is crucial for stochastic derivative estimation as the divergence rule avoids surface integrals which may require multiple sample paths.

Handling Indicator Discontinuities

For sample performances with discontinuities arising from indicator functions, the authors show that applying the Leibniz divergence rule (2) yields a novel single-run unbiased derivative estimator This is achieved by absorbing indicator functions directly into the integration domain, leaving a differentiable integrand In practice, this involves constructing a differentiable mapping h over a set V to parametrize the restricted domain The resulting Leibniz divergence estimator is given by equation (6), which combines the standard IPA-LR term with a divergence term that captures the effect of discontinuities introduced by the indicator function.

Push-out and General Discontinuities

For general discontinuous sample performances admitting a change of variables, the authors propose a framework combining push-out LR with the Leibniz integral rule This approach applies the Leibniz integral rule (1) to avoid differentiation with respect to the input, using smooth approximations and reversing the change of variables via Proposition 2 The final result is expressed in terms of a volume integral and a surface integral This framework generalizes GLR methods but requires simpler, more easily verifiable conditions for the surface integral to vanish, facilitating single-run estimators.

Implementation and Extensions

The paper details extensions of the Leibniz integral estimator to various settings, including hyperrectangle support and unbounded sample spaces For hyperrectangle support, the surface integral is expressed as a sum of conditional expectations over faces, yielding an unbiased estimator (15) The Leibniz divergence estimator is shown to be numerically stable across different joint distributions, outperforming the Leibniz integral estimator when strong dependencies are present The framework allows for extensions to sample performances involving stage-wise rewards and stopping thresholds in optimal stopping problems.

Simulation Results

Simulation experiments evaluate the proposed Leibniz estimators using various joint distributions for random vectors, including independent exponential, FGM copula with exponential marginals, joint log-normal, and Clayton copula with Gamma marginals The results highlight that the Leibniz divergence estimator remains numerically stable and consistently yields low-variance estimates across all scenarios compared to the finite-difference (FD) estimator. In cases where the Leibniz integral estimator fails, such as when shape parameters are small in Gamma marginals, the Leibniz divergence estimator is shown to be more robust and stable. p. 2

Where the paper says it

Although the GLR method also introduces a surface integral via integration by parts, the Leibniz framework reveals more geometric insights, enabling the identification of broader conditions under which the surface integral vanishes. We summarize the main contributions of our work as follows: • For sample performance functions with discontinuities arising from indicator functions, under suitable conditions, applying the Leibniz divergence rule yields a novel single-run unbiased derivative estimator.

The comparison between estimators demonstrates that avoiding surface integrals by choosing transformations that yield parameter-independent supports can reduce simulation cost. p. 6
Where the paper says it

In addition, it offers a clearer geometric interpretation: the surface integral arises from differentiating the parameter-dependent domain, a geometric origin that cannot be revealed by the GLR approach. This insight motivates the use of transformations that yield parameter-independent domains, thereby eliminating the surface integral and reducing simulation cost.

Conclusions

The paper concludes by highlighting two straightforward extensions of the Leibniz integral estimator: its natural extension to sample performances involving a differentiable outer function, which encompasses the SLRIPA estimator and its ability to be extended to cases where the input dimension exceeds that of g Future research directions include developing systematic techniques for parameterizing the integration domain and identifying effective changes of variables to eliminate surface integrals. The paper suggests that combining these Leibniz estimators with conditioning can further improve efficiency in settings like sequential decision-making problems. p. 8

Where the paper says it

Ren, Fu, and L’Ecuyer: Stochastic Derivative Estimation through Leibniz Integration 9 This formulation extends Example 2 and often arises in sequential decision-making problems, such as option pricing (Wang et al. 2012) and inventory management (Pflug and Rubinstein 2002).

References

Amann H, Escher J, Brookfield G (2005) Analysis (Verlag: Birkhauser)

Broadie M, Glasserman P (1996) Estimating security price derivatives using simulation. Management Science 42(2):269–285

Cao XR (1985) Convergence of parameter sensitivity estimates in a stochastic experiment. IEEE Transactions on Automatic Control 30(9):845–853

Cui Z, Fu MC, Hu JQ, Liu Y, Peng Y, Zhu L (2020) On the variance of single-run unbiased stochastic derivative estimators. INFORMS Journal on Computing 32(2):390–407

Flanders H (1973) Differentiation under the integral sign. The American Mathematical Monthly 80(6):615–627

Folland GB (1999) Real Analysis: Modern Techniques and Their Applications (New York: John Wiley & Sons)

Frankel T (2011) The Geometry of Physics: An Introduction (New York: Cambridge University Press)

Fu MC (2006a) Gradient estimation. Henderson SG, Nelson BL, eds., Handbooks in Operations Research and Management Science: Simulation, chapter 19 (Elsevier)

Fu MC (2006b) Sensitivity analysis in Monte Carlo simulation of stochastic activity networks. Alt FB, Fu MC, Golden BL, eds., Perspectives in Operations Research: Papers in Honor of Saul Gass’ 80 th Birthday, 351–366 (Springer)

Fu MC (2008) What you should know about simulation and derivatives. Naval Research Logistics (NRL) 55(8):723–736

Fu MC (2015) Stochastic gradient estimation. Fu MC, ed., Handbook on Simulation Optimization, chapter 5 (New York: Springer)

Fu MC, Hu JQ (1995) Sensitivity analysis for Monte Carlo simulation of option pricing. Probability in the Engineering and Informational Sciences 9(3):417–446

Fu MC, Hu JQ (1997) Conditional Monte Carlo: Gradient Estimation and Optimization Applications (Boston: Kluwer Academic)

Glasserman P (1991a) Gradient Estimation via Perturbation Analysis (Boston: Kluwer Academic)

Glasserman P (1991b) Structural conditions for perturbation analysis derivative estimation: Finite-time performance indices. Operations Research 39(5):724–738

Glasserman P (2003) Monte Carlo Methods in Financial Engineering (New York: Springer)

Glynn PW (1987) Likelihood ratio gradient estimation: An overview. Proceedings of the 1987 Winter Simulation Conference (WSC), 366–375 (ACM)

Heidelberger P, Cao XR, Zazanis MA, Suri R (1988) Convergence properties of infinitesimal perturbation analysis estimates. Management Science 34(11):1281–1302

Heidergott B, Peng Y (2023) Gradient estimation for smooth stopping criteria. Advances in Applied Probability 55(1):29–55

Heidergott B, Vazquez-Abad F (2006) Measure valued differentiation for random horizon problems. ´ Markov Processes and Related Fields 12:509–536

Huh WT, Janakiraman G (2008) (s, S) optimality in joint inventory-pricing control: An alternate approach. Operations Research 56(3):783–790

L’Ecuyer P (1990) A unified view of the IPA, SF, and LR gradient estimation techniques. Management Science 36(11):1364–1383

L’Ecuyer P (1991) An overview of derivative estimation. Proceedings of the 1991 Winter Simulation Conference (WSC), 207–217 (IEEE)

L’Ecuyer P, Puchhammer F, Ben Abdellah A (2022) Monte Carlo and quasi–Monte Carlo density estimation via conditioning. INFORMS Journal on Computing 34(3):1729–1748

Peng Y, Fu MC, Hu J, L’Ecuyer P, Tuffin B (2022) Variance reduction for generalized likelihood ratio method by conditional Monte Carlo and randomized Quasi-Monte Carlo methods. Journal of Management Science and Engineering 7(4):550–577

Peng Y, Fu MC, Hu JQ, Heidergott B (2018) A new unbiased stochastic derivative estimator for discontinuous sample performances with structural parameters. Operations Research 66(2):487–499

Pflug GC (1996) Optimization of Stochastic Models: The Interface Between Simulation and Optimization (Boston: Kluwer Academic)

Pflug GC, Rubinstein RY (2002) Inventory processes: Quasi- regenerative property and sensitivity estimation. Stochastic Models 18(3):469–496

Puchhammer F, L’Ecuyer P (2022) Likelihood ratio density estimation for simulation models. Proceedings of the 2022 Winter Simulation Conference (WSC), 109–120 (IEEE)

Reiman MI, Weiss A (1989) Sensitivity analysis for simulations via likelihood ratios. Operations Research 37(5):830–844

Rubinstein RY (1986) The score function approach for sensitivity analysis of computer simulation models. Mathematics and Computers in Simulation 28(5):351–379

Rubinstein RY (1992) Sensitivity analysis of discrete event systems by the “push out” method. Annals of Operations Research 39(1):229–250

Shapiro VL (1958) The divergence theorem for discontinuous vector fields. Annals of Mathematics 68(3):604–624

Shi L (1996) Discontinuous perturbation analysis of discrete-event dynamic systems. IEEE Transactions on Automatic Control 41(11):1676–1681

Spivak M (1965) Calculus on Manifolds: A Modern Approach to Classical Theorems of Advanced Calculus (Reading, Massachusetts: Addison-Wesley)

Suri R, Zazanis MA (1988) Perturbation analysis gives strongly consistent sensitivity estimates for the M/G/1 queue. Management Science 34(1):39–64

Wang Y, Fu MC, Marcus SI (2012) A new stochastic derivative estimator for discontinuous payoff functions with application to financial derivatives. Operations Research 60(2):447–460

Zorich VA (2004a) Mathematical Analysis I (Berlin: Springer)

Zorich VA (2511.9999) Mathematical Analysis II (Berlin: Springer) p. 0

Where the paper says it

Numerical experiments demonstrate the effectiveness and robustness of our methods. Key words: Stochastic derivative estimation, Simulation optimization, Leibniz integral rule, Perturbation analysis, Likelihood ratio 1.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Develop a single-run unbiased derivative estimator for sample performance functions with discontinuities arising from indicator functions by applying the Leibniz divergence rule when an appropriate change of variables is used to embed indicators into the integration domain.

  2. Implement a framework that combines push-out likelihood ratio methods with the Leibniz integral rule to handle general discontinuous sample performance functions, providing flexibility in choosing changes of variables while reducing reliance on multiple sample paths compared to standard GLR methods.

  3. Design estimators that exploit geometric insights from the Leibniz framework by identifying broader conditions under which the surface integral vanishes, enabling single-run estimators when input distributions are independent or when the transformed input has a parameter-independent support.

  4. Create a method for estimating sensitivity of expected payoffs in optimal stopping problems by leveraging conditioning on past inputs and applying the univariate Leibniz integral rule to inner conditional expectations, which recovers the Infinitesimal Perturbation Analysis (IPA) estimator.

  5. Build robust estimators for complex dependency structures like FGM copulas that maintain numerical stability and low variance, contrasting them against the Leibniz integral estimator which becomes unstable under certain conditions when compared to the Leibniz divergence estimator.

  6. Extend existing derivative estimation methods to handle unbounded hyperrectangle-supported inputs by employing a truncation technique and taking a limit, ensuring that the resulting estimators remain valid under appropriate integrability conditions.

Abstract

Based on the multidimensional Leibniz integral rule, we propose a novel unbiased stochastic derivative estimation framework for sample performance functions that are discontinuous in the parameter of interest. In this framework, for general discontinuities, a change of variables followed by the Leibniz integral rule generally yields a multi-run estimator, whereas for indicator-type discontinuities, a suitable transformation of the sample performance leads directly to a single-run estimator. The new framework includes existing likelihood ratio (LR)-based approaches as special cases and applies more broadly. In particular, it encompasses generalized LR (GLR) methods and the new formulation permits us to obtain much milder and simpler sufficient regularity conditions for these estimators to be unbiased, compared with the conditions given in previous papers. We also provide guidance on how to construct efficient single-run estimators in settings where existing methods may require the simulation of multiple sample paths or may be inapplicable. We validate the effectiveness and robustness of the proposed framework through an inventory-price control problem.

Related papers