E-backtesting
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Quantitative Finance. The day in quantitative finance: models of markets, risk and prices.
Ruth: I'm Ruth.
Warren: And I'm Warren, here with Iris.
Ruth: We're looking at the paper: "E-backtesting".
Warren: Here's the gist of the paper. The paper introduces a model-free backtesting method utilizing e-values and e-processes to address challenges in backtesting risk measures, particularly Expected Shortfall (ES), which has become the standard risk measure under recent Basel Accords.
Ruth: The primary contributions are multifaceted: first, defining backtest e-statistics and proposing novel backtesting methods, yielding specific statistics for ES (Theorem 1) and other risk measures (Theorem 2); second, proposing three data-based betting process calculations—GRO, GREE, GREL, and GREM—which are shown to be asymptotically optimal in various scenarios (Theorem 3); third, characterizing the backtest e-statistics for mean, variance, VaR (Theorem 4), and ES (Theorem 5); and finally, demonstrating detailed procedures for backtesting VaR and ES using e-values in practical financial operations.
Warren: The methodology centers on constructing a stochastic process M t defined by M 0 = 1 and M t = (1 - lambda t + lambda t X t) M t-1 = Y s = (1 - lambda s + lambda X s), for t in
T: .
Ruth: Backtest e-statistics are defined, and the paper establishes that for ES, the function e ES p is a monotone backtest e-statistic for (ES p, VaR p) (Theorem 1).
Title and authors: Ruth: Okay, so we’ve seen the basic setup of E-backtesting and how they’re defining their statistics. Now let’s look at what exactly this paper is saying in its summary about the methods they developed and why it matters to us right now >
Warren: Basically, the core idea is building a stochastic process called M t starting at one, where M t depends on some lambda and X t, which represents the actual losses or risk factors over time >
Iris: They show that for ES specifically, this function e ES p is a monotone backtest e-statistic for both Expected Shortfall and Value-at-Risk simultaneously using theorem one >
Ruth: That monotonicity is important because it means as you test, the statistic moves in a predictable way, which simplifies things when you're trying to see if a risk measure is being underestimated or overestimated >
Warren: The paper also discusses the choice of betting process being crucial, and they propose GRO for growth-rate optimal, GREE for empirical e-statistics, GREL for empirical losses, and GREM for empirical mixture >
Iris: They show that GREM is recommended as a stable default because it performs well across different testing situations >
Ruth: It sounds like they’re giving practitioners a clear path on which betting process to choose depending on what kind of data or risk scenario they're facing when backtesting ES >
The paper's summary: Warren: Moving past the basic summary, the authors point out several real improvements they are proposing for this whole E-backtesting approach, which I think are pretty significant for how we actually use these tools >
Iris: One big improvement is that you can perform backtest ES for market risk using model-free e-values and e-processes, which lets you test underestimation of tail risk without needing to have a specific distributional forecast >
Ruth: That’s really useful because it allows regulators to get alerted early because the "e-process accumulates to a reasonably large value," rather than waiting for some complex model output >
Warren: They also suggest using the GREM method for choosing the betting process, which they propose as an average of GREE and GREL, providing a robust approach when there isn't a clear pattern in losses or forecasts >
Iris: They suggest that the e-backtesting procedure can be used with mean-variance criteria to determine portfolio weights, and this is interesting because it doesn't require you to know any specific trading strategy or statistical model for that >
Ruth: So, the idea is making the process flexible enough that you don't need a perfect model of the underlying economy just to do a decent risk check >
The paper's improvements: Ruth: We’ve gone over how they defined their statistics and which betting processes are best for ES, but let’s wrap up with what this all means for the practical application in the real world >
Warren: The main implication is that e-backtesting is presented as the first model-free and non-asymptotic backtest for Expected Shortfall, which means it's something you can actually run without needing a fixed sample size or assuming a specific model structure on your data >
Iris: They also characterized the necessary conditions for the existence of backtest e-statistics, revealing that for risk measures like ES, the functional has to be quasi-linear >
Ruth: So while this method offers validity across all stopping times and feasibility without underlying model assumptions, they admit a trade-off where these e-backtests might be less powerful than traditional model-based approaches >
Warren: They also pointed out that the characterization results identify essentially unique optimal choices for backtest e-statistics for VaR and Expected Shortfall, which is good because it helps narrow down the search space for us >
Iris: And finally, they showed detailed procedures for practical use in financial operations, which is where we get to see how this actually works on a portfolio level >
Ruth: So the paper E-backtesting really gives us a robust way to check if our risk forecasts are underestimating tail risk without having to commit to a specific distributional assumption on the losses >
Conclusion: Ruth: So we’ve seen how they’re building this model called E-backtesting, and what it means for testing if risk measures like Expected Shortfall are being underreported >
Warren: Right, so you're saying it lets us test if tail risk is being underestimated without having to guess the exact shape of the loss distribution >
Iris: That's the core idea. They show this function e ES p is monotone for both expected shortfall and value at risk simultaneously, which means we can track how well our forecasts are working over time >
Ruth: And they’ve got these betting processes like GRO, GREE, GREL, and GREM to choose from; they even recommend GREM as a stable default because it handles different testing situations well >
Warren: The authors also show that you can use this method to detect evidence against forecasts with thresholds of two or five or ten on the e-process value >
Iris: That early warning mechanism is pretty strong, much better than just doing some traditional p-tests, because it watches the accumulation of risk evidence >
Ruth: It’s interesting how they talk about institutional gaming here; they show that using a properly calculated betting process like GREM can help institutions stop under-reporting risks >
Warren: They suggest that GREM is superior in that context because it keeps the e-process near one during over-reporting and grows fast during under-reporting, unlike some of the others >
Iris: It’s important to remember they also characterized the necessary conditions for these statistics to exist, showing that for risk measures like ES, you need a quasi-linear functional >
Ruth: So while this method is flexible—it works across all stopping times and doesn't require an underlying model—they admit it might not be as powerful as some of the traditional model-based approaches >
Warren: It’s a trade-off, I guess. But they also point out that their characterization results help us find the most optimal choices for these backtest statistics for value at risk and expected shortfall >
Iris: Exactly. And in terms of practical application, they showed how you can use this e-backtesting procedure with mean variance criteria to determine portfolio weights without needing any specific trading strategy >
Ruth: It’s pretty broad then, moving beyond just a specific financial product to general portfolio risk management using these principles >
Warren: So the takeaway is that E-backtesting gives us a way to check risk underestimation robustly, even when we don't have perfect models of the losses themselves >
Iris: It’s still important to remember the authors flag that this method doesn't replace a full model-based approach; it supplements it by checking for estimation errors >
Ruth: That’s right. So we have this paper, "E-backtesting," showing a new way to test risk without needing perfect assumptions about the loss distribution >
Warren: It’s a practical tool that helps spot when institutions might be hiding tail risks in their reports >
Iris: Next time, we’ll look at how AI is actually messing with market data and where those outages are hiding things.
Qiuqi Wang, Ruodu Wang, Johanna Ziegel
Georgia State University · University of Waterloo · ETH Zurich
q-fin.RM, math.ST, stat.ME, stat.TH
Submitted: 2022-08-27
Updated: 2026-11-06
Comments: 68 pages
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
Importance score: 100/100
The gist: The paper introduces a model-free backtesting method utilizing e-values and e-processes to address challenges in backtesting risk measures, particularly Expected Shortfall (ES)…
Key concepts
- Backtest e-statistics
- These are statistical measures derived from a stochastic process used to test if a risk forecast is accurate. The paper defines specific backtest e-statistics for ES and other risks, showing how they behave when testing predictions.
- Betting Processes (GRO, GREE, GREL, GREM)
- These are different ways to calculate the underlying process used in backtesting. They are designed to capture different aspects of risk behavior—like growth rate or empirical losses—and GREM is recommended for stability.
- e-values and e-processes
- These are tools that allow for model-free backtesting. They help detect evidence against forecasts by accumulating values in an e-process, providing an early warning signal when risk estimates might be underestimated.
Terminology
Summary
The paper introduces a model-free backtesting method utilizing e-values and e-processes to address challenges in backtesting risk measures, particularly Expected Shortfall (ES), which has become the standard risk measure under recent Basel Accords. p. 0 E-backtesting
Qiuqi Wang∗ Ruodu Wang† Johanna Ziegel‡
16th April 2026
Abstract
In the recent Basel Accords, the Expected Shortfall (ES) replaces the Value-at-Risk (VaR) as
the standard risk measure for market risk in the banking sector, making it the most important
risk measure in financial regulation. One of the most challenging tasks in risk modeling practice
is to backtest ES forecasts provided by financial institutions. It turns out that these methods are asymptotically optimal
(equivalent to an oracle betting process) in different situations (Theorem 3). Third, we characterize
backtest e-statistics for the mean, the variance, VaR (Theorem 4), and ES (Theorem 5) in Section
6.Where the paper says it
Where the paper says it
The methodology centers on constructing a stochastic process M t defined by M 0 = 1 and M t = (1 - lambda t + lambda t X t) M t-1 = Y s = (1 - lambda s + lambda X s), for t in [T]. p. 5 (iv) Construct a stochastic process: M0 = 1 and
Mt = (1 − λt + λtXt)Mt−1 =
Y
t
s=1
(1 − λs + λsXs), t ∈ [T]. (v) Reject H0 at time t if we observe Mt > 1/α. Third, we characterize
backtest e-statistics for the mean, the variance, VaR (Theorem 4), and ES (Theorem 5) in Section
6. All backtest e-statistics for these functionals take similar forms as mixtures between the constant
function equal to 1 and a simple backtest e-statistic.Where the paper says it
Where the paper says it
Simulation studies validate the method's advantages, showing that evidence against forecasts is detected using thresholds of 2, 5, or 10. The e-backtesting procedure is presented as the first model-free and non-asymptotic backtest for ES. p. 1 Table 1 summarizes the main features of existing methods backtesting
ES. To the best of our knowledge, there is no model-free non-asymptotic backtesting method for
ES. To design a model-free backtesting
procedure for ES, we make use of the recently developed techniques of e-values and e-processes. Backtest e-statistics are introduced to formulate e-processes for risk measure forecasts, and
unique forms of backtest e-statistics for VaR and ES are characterized using recent results on
identification functions. Fourth, the tester does not necessarily accurately know the underlying
model used by a financial institution to produce risk predictions. In financial practice, a well-adopted simple approach exists for backtesting the Value-at-Risk
(VaR), which is the so-called three-zone approach based on binomial tests described in BCBS (2013);
this approach is model-free in the sense that one directly tests the risk forecast without testing
any specific family of models. To design a model-free backtesting
procedure for ES, we make use of the recently developed techniques of e-values and e-processes. Backtest e-statistics are introduced to formulate e-processes for risk measure forecasts, and
unique forms of backtest e-statistics for VaR and ES are characterized using recent results on
identification functions.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
A significant practical application demonstrated is addressing the issue of financial institutions Five sections are put in the e-companion. Section A shows the betting processes calculated via
Taylor approximation for VaR and ES. As a particularly relevant feature to our context, our proposed e-tests allow regulators to
get alerted early as the e-process accumulates to a reasonably large value. This is different from
scientific discoveries (such as genome studies) where a scientist may not be entitled to reject a
1Quoting BCBS (2016, p. One of the most challenging tasks in risk modeling practice
is to backtest ES forecasts provided by financial institutions. To design a model-free backtesting
procedure for ES, we make use of the recently developed techniques of e-values and e-processes.gaming
regulators by producing conservative forecasts initially and then under-reporting risks. This is mitigated by using a properly calculated betting process. p. 3Where the paper says it
Where the paper says it
Where the paper says it
The forecasting procedure involves estimating model parameters using standard Gaussian QML on presampled losses, followed by forecasts for VaR and ES based on empirical forecasts derived from presampled residuals under various innovation distributions (normal, t-, or skewed-t). p. 0 To design a model-free backtesting
procedure for ES, we make use of the recently developed techniques of e-values and e-processes. Backtest e-statistics are introduced to formulate e-processes for risk measure forecasts, and
unique forms of backtest e-statistics for VaR and ES are characterized using recent results on
identification functions. Parametric or dependence assumptions refer to those on loss distributions, time series models, stationarity, or strong mixing. Forecast
structural assumptions refer to requirements on the forms and properties of risk forecasts. Table 1: Comparison of existing backtesting methods for ES
Literature
Parametric or
dependence
assumptions
Forecast
structural
assumptions
Fixed sample
size
Asymptotic
test
Reliance on VaR
or distributional
forecasts
MF00 yes yes yes yes yes
AS14 yes yes yes yes yes
DE17 yes yes yes yes yes
NZ17 yes yes yes yes yes
SQPQ21 yes yes yes yes yes
BD22 yes yes yes yes no
HD22 yes yes no no yes
This paper no no no no yes
Notes: We use shortcuts MF00 for McNeil and Frey (2000), AS14 for Acerbi and Sz´ekely (2014), DE17
for Du and Escanciano (2017), NZ17 for Nolde and Ziegel (2017), SQPQ21 for Su et al. (2021), BD22 for
Bayer and Dimitriadis (2022), and HD22 for Hoga and Demetrescu (2023).Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Backtest ES for market risk can be performed using model-free e-values and e-processes, which allows for testing under estimation of tail risk rather than requiring specific distributional forecasts. This enables regulators to get alerted early as
the e-process accumulates to a reasonably large value.
-
The AI system can utilize the GREM method for choosing the betting process, which is proposed by taking the average of the GREE and GREL methods, providing a robust approach when there is no clear pattern in losses or forecasts.
-
The system can detect evidence against risk forecasts with high probability using thresholds 2, 5, or 10 for e-processes exceeding these values, offering an early warning mechanism superior to traditional p-tests.
-
The AI system can monitor structural changes in time series data by comparing the results of the e-backtesting method with the sequential monitoring method, which is shown to outperform the e-backtesting methods in terms of detection percentage and speed.
-
For portfolio risk management, the system can apply the e-backtesting procedure using mean-variance criteria to determine portfolio weights, without requiring knowledge of a specific trading strategy or statistical model.
Abstract
In the recent Basel Accords, the Expected Shortfall (ES) replaces the Value-at-Risk (VaR) as the standard risk measure for market risk in the banking sector, making it the most important risk measure in financial regulation. One of the most challenging tasks in risk modeling practice is to backtest ES forecasts provided by financial institutions. To design a model-free backtesting procedure for ES, we make use of the recently developed techniques of e-values and e-processes. Backtest e-statistics are introduced to formulate e-processes for risk measure forecasts, and unique forms of backtest e-statistics for VaR and ES are characterized using recent results on identification functions. For a given backtest e-statistic, a few criteria for optimally constructing the e-processes are studied. The proposed method can be naturally applied to many other risk measures and statistical quantities. We conduct extensive simulation studies and data analysis to illustrate the advantages of the model-free backtesting method, and compare it with the ones in the literature.
Sources
Related papers
- Diversified-Risk-Perception Risk Measures and Portfolio Selection
- Robust distortion riskmetrics over Bregman--Wasserstein balls and their intersections
- Distributions or Quantiles? A Convex Variational Theory of Worst-Case Risk
- Learned Monotone Recurrent Features in Governed Credit Scoring: The Price of the Frame and the Necessity of Macro Conditioning
- Correlation Ambiguity, Diversification Fragility, and Insurance Pricing
- Universal Value-at-Risk superadditivity