E-backtesting
summary
The gist
The paper introduces a model-free backtesting method utilizing e-values and e-processes to address challenges in backtesting risk measures, particularly Expected Shortfall (ES)…
In short
The paper introduces a model-free method using e-values and e-processes to backtest risk measures like Expected Shortfall (ES). It defines specific statistics and proposes four data-based betting processes (GRO, GREE, GREL, GREM) to detect forecast errors. This allows for testing risk under estimation without needing specific distributional assumptions.
Key concepts
- Backtest e-statistics
- These are statistical measures derived from a stochastic process used to test if a risk forecast is accurate. The paper defines specific backtest e-statistics for ES and other risks, showing how they behave when testing predictions.
- Betting Processes (GRO, GREE, GREL, GREM)
- These are different ways to calculate the underlying process used in backtesting. They are designed to capture different aspects of risk behavior—like growth rate or empirical losses—and GREM is recommended for stability.
- e-values and e-processes
- These are tools that allow for model-free backtesting. They help detect evidence against forecasts by accumulating values in an e-process, providing an early warning signal when risk estimates might be underestimated.
Terminology used across episodes
This episode discusses
The paper
Qiuqi Wang, Ruodu Wang, Johanna Ziegel
Georgia State University · University of Waterloo · ETH Zurich
In the recent Basel Accords, the Expected Shortfall (ES) replaces the Value-at-Risk (VaR) as the standard risk measure for market risk in the banking sector, making it the most important risk measure in financial regulation. One of the most challenging tasks in risk modeling practice is to backtest ES forecasts provided by financial institutions. To design a model-free backtesting procedure for ES, we make use of the recently developed techniques of e-values and e-processes. Backtest e-statistics are introduced to formulate e-processes for risk measure forecasts, and unique forms of backtest e-statistics for VaR and ES are characterized using recent results on identification functions. For a given backtest e-statistic, a few criteria for optimally constructing the e-processes are studied. The proposed method can be naturally applied to many other risk measures and statistical quantities. We conduct extensive simulation studies and data analysis to illustrate the advantages of the model-free backtesting method, and compare it with the ones in the literature.
Transcript
Introduction to the show: ident: Quantitative Finance. The day in quantitative finance: models of markets, risk and prices.
Ruth: I'm Ruth.
Warren: And I'm Warren, here with Iris.
Ruth: We're looking at the paper: "E-backtesting".
Warren: Here's the gist of the paper. The paper introduces a model-free backtesting method utilizing e-values and e-processes to address challenges in backtesting risk measures, particularly Expected Shortfall (ES), which has become the standard risk measure under recent Basel Accords.
Ruth: The primary contributions are multifaceted: first, defining backtest e-statistics and proposing novel backtesting methods, yielding specific statistics for ES (Theorem 1) and other risk measures (Theorem 2); second, proposing three data-based betting process calculations—GRO, GREE, GREL, and GREM—which are shown to be asymptotically optimal in various scenarios (Theorem 3); third, characterizing the backtest e-statistics for mean, variance, VaR (Theorem 4), and ES (Theorem 5); and finally, demonstrating detailed procedures for backtesting VaR and ES using e-values in practical financial operations.
Warren: The methodology centers on constructing a stochastic process M t defined by M 0 = 1 and M t = (1 - lambda t + lambda t X t) M t-1 = Y s = (1 - lambda s + lambda X s), for t in
T: .
Ruth: Backtest e-statistics are defined, and the paper establishes that for ES, the function e ES p is a monotone backtest e-statistic for (ES p, VaR p) (Theorem 1).
Title and authors: Ruth: Okay, so we’ve seen the basic setup of E-backtesting and how they’re defining their statistics. Now let’s look at what exactly this paper is saying in its summary about the methods they developed and why it matters to us right now >
Warren: Basically, the core idea is building a stochastic process called M t starting at one, where M t depends on some lambda and X t, which represents the actual losses or risk factors over time >
Iris: They show that for ES specifically, this function e ES p is a monotone backtest e-statistic for both Expected Shortfall and Value-at-Risk simultaneously using theorem one >
Ruth: That monotonicity is important because it means as you test, the statistic moves in a predictable way, which simplifies things when you're trying to see if a risk measure is being underestimated or overestimated >
Warren: The paper also discusses the choice of betting process being crucial, and they propose GRO for growth-rate optimal, GREE for empirical e-statistics, GREL for empirical losses, and GREM for empirical mixture >
Iris: They show that GREM is recommended as a stable default because it performs well across different testing situations >
Ruth: It sounds like they’re giving practitioners a clear path on which betting process to choose depending on what kind of data or risk scenario they're facing when backtesting ES >
The paper's summary: Warren: Moving past the basic summary, the authors point out several real improvements they are proposing for this whole E-backtesting approach, which I think are pretty significant for how we actually use these tools >
Iris: One big improvement is that you can perform backtest ES for market risk using model-free e-values and e-processes, which lets you test underestimation of tail risk without needing to have a specific distributional forecast >
Ruth: That’s really useful because it allows regulators to get alerted early because the "e-process accumulates to a reasonably large value," rather than waiting for some complex model output >
Warren: They also suggest using the GREM method for choosing the betting process, which they propose as an average of GREE and GREL, providing a robust approach when there isn't a clear pattern in losses or forecasts >
Iris: They suggest that the e-backtesting procedure can be used with mean-variance criteria to determine portfolio weights, and this is interesting because it doesn't require you to know any specific trading strategy or statistical model for that >
Ruth: So, the idea is making the process flexible enough that you don't need a perfect model of the underlying economy just to do a decent risk check >
The paper's improvements: Ruth: We’ve gone over how they defined their statistics and which betting processes are best for ES, but let’s wrap up with what this all means for the practical application in the real world >
Warren: The main implication is that e-backtesting is presented as the first model-free and non-asymptotic backtest for Expected Shortfall, which means it's something you can actually run without needing a fixed sample size or assuming a specific model structure on your data >
Iris: They also characterized the necessary conditions for the existence of backtest e-statistics, revealing that for risk measures like ES, the functional has to be quasi-linear >
Ruth: So while this method offers validity across all stopping times and feasibility without underlying model assumptions, they admit a trade-off where these e-backtests might be less powerful than traditional model-based approaches >
Warren: They also pointed out that the characterization results identify essentially unique optimal choices for backtest e-statistics for VaR and Expected Shortfall, which is good because it helps narrow down the search space for us >
Iris: And finally, they showed detailed procedures for practical use in financial operations, which is where we get to see how this actually works on a portfolio level >
Ruth: So the paper E-backtesting really gives us a robust way to check if our risk forecasts are underestimating tail risk without having to commit to a specific distributional assumption on the losses >
Conclusion: Ruth: So we’ve seen how they’re building this model called E-backtesting, and what it means for testing if risk measures like Expected Shortfall are being underreported >
Warren: Right, so you're saying it lets us test if tail risk is being underestimated without having to guess the exact shape of the loss distribution >
Iris: That's the core idea. They show this function e ES p is monotone for both expected shortfall and value at risk simultaneously, which means we can track how well our forecasts are working over time >
Ruth: And they’ve got these betting processes like GRO, GREE, GREL, and GREM to choose from; they even recommend GREM as a stable default because it handles different testing situations well >
Warren: The authors also show that you can use this method to detect evidence against forecasts with thresholds of two or five or ten on the e-process value >
Iris: That early warning mechanism is pretty strong, much better than just doing some traditional p-tests, because it watches the accumulation of risk evidence >
Ruth: It’s interesting how they talk about institutional gaming here; they show that using a properly calculated betting process like GREM can help institutions stop under-reporting risks >
Warren: They suggest that GREM is superior in that context because it keeps the e-process near one during over-reporting and grows fast during under-reporting, unlike some of the others >
Iris: It’s important to remember they also characterized the necessary conditions for these statistics to exist, showing that for risk measures like ES, you need a quasi-linear functional >
Ruth: So while this method is flexible—it works across all stopping times and doesn't require an underlying model—they admit it might not be as powerful as some of the traditional model-based approaches >
Warren: It’s a trade-off, I guess. But they also point out that their characterization results help us find the most optimal choices for these backtest statistics for value at risk and expected shortfall >
Iris: Exactly. And in terms of practical application, they showed how you can use this e-backtesting procedure with mean variance criteria to determine portfolio weights without needing any specific trading strategy >
Ruth: It’s pretty broad then, moving beyond just a specific financial product to general portfolio risk management using these principles >
Warren: So the takeaway is that E-backtesting gives us a way to check risk underestimation robustly, even when we don't have perfect models of the losses themselves >
Iris: It’s still important to remember the authors flag that this method doesn't replace a full model-based approach; it supplements it by checking for estimation errors >
Ruth: That’s right. So we have this paper, "E-backtesting," showing a new way to test risk without needing perfect assumptions about the loss distribution >
Warren: It’s a practical tool that helps spot when institutions might be hiding tail risks in their reports >
Iris: Next time, we’ll look at how AI is actually messing with market data and where those outages are hiding things.
More episodes
- The day in conversation for 2026-11-19
- Daily Summary for 2026-11-19
- The day in conversation for 2026-11-18
- Daily Summary for 2026-11-18
- 2601.05274-On the use of case estimate and transactional payment data in neural networks for individual loss reserving
- 2009.00868-Loss-Given-Default Modeling by Post-Last Passage Time Process
- 2606.30070-Financial Resilience Evaluation: From Conditional Expectations to Dynamic Convex Risk Measures
- 2212.12687-Measuring price impact and information content of trades in a time-varying setting
- Daily Summary for 2026-11-17
- 2401.08323-Dynamic portfolio selection under generalized disappointment aversion