Comparative e-backtesting
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.
Tom: We're looking at the paper: "Comparative e-backtesting".
Jane: Here's the gist of the paper. Comparative e-backtests for general risk measures develop a model-free sequential framework using e-values and e-processes to provide anytime-valid inference and robustness under dependence and model misspecification The main contribution of this paper is four-fold, including developing a model-free sequential framework for comparative e-backtests of elicitable risk measures for financial regulatory practice Framework Development The authors develop a non-parametric sequential framework for comparative backtests of general elicitable risk measures using e-values and e-processes They propose a modified three-zone approach based on weak dominance, which yields more informative conclusions in comparative backtesting The framework is built upon the concept of e-values and e-processes, which allow for sequential anytime-valid inference and robustness to model misspecification and dependence Key Contributions
Paper summary: Tom: So, to recap, this paper is "Comparative e-backtesting." The core thesis is that they've developed a non-parametric sequential framework utilizing e-values and e-processes to perform comparative backtests of general elicitable risk measures.
Jane: And the key claim here is that these proposed methods deliver anytime-valid inference, meaning you can get reliable conclusions whenever you need them, and they maintain robustness against both model misspecification and dependence in the data >
Lu: They propose a specific technical modification to an existing three-zone approach called weak dominance. This modification is designed to produce more informative conclusions when comparing models against each other in this comparative testing environment >
Meng: So, it’s not just about a single test; it’s a whole sequential system that handles the comparisons dynamically as the data arrives. That sounds like a lot of computational coordination to manage correctly >
Lalam: From what I've seen, they characterize possible forms of e-values for standard e-tests of identifiable risk measures and also construct the associated e-processes as technical building blocks for this comparative framework >
Jane: That means they aren't just proposing a new way to run one test; they are giving you the tools to build a whole system around comparing models against each other >
Tom: And they show detailed procedures for common measures like Value-at-Risk, Expected Shortfall, and expectiles in their numerical studies which serve as examples for practical manipulation in financial regulation >
Meng: But how certain are these results? When you're doing comparative testing on real data, you have to worry about the assumptions they make regarding dependence and misspecification >
Conclusion: Tom: So, looking at "Comparative e-backtesting" by Zhanyi Jiao, Qiuqi Wang, and Yimiao Zhao March six two thousand twenty-six it seems the main implication is providing a more rigorous foundation for how financial regulators validate internal models against standard benchmarks > <ref:2511.05840#pg1,Yimiao Zhao March 6 2026>
Jane: It moves beyond just checking if a model is generally consistent with losses; it allows for a direct comparison between two different models in an adaptive way that handles uncertainty well >
Lu: The title emphasizes "comparative," which is important because the authors are not just testing one model in isolation, but setting up a framework to test hypotheses like whether one internal model is at least as accurate as another benchmark >
Meng: From a deployment standpoint, it suggests that if we can build these e-value and e-process characterizations into our systems, we get a more reliable way to assess risk management performance in production >
Lalam: The framework offers a path toward sequential anytime valid inference and robustness under dependence which is exactly what practitioners need when dealing with complex financial data >
Zhanyi Jiao, Qiuqi Wang, Yimiao Zhao
School of Mathematical and Statistical Sciences, Arizona State University · Maurice R. Greenberg School of Risk Science, Georgia State University · Royal Bank of Canada
stat.ME, econ.EM, stat.AP
Submitted: 2025-11-08
Updated: 2026-11-09
Comments: Substantially revised; 37 pages including appendices
Code: https://github.com/qwangan/Standard-and-comparative-e-backtests-for-general-risk-measures
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
Importance score: 83/100
The gist: Comparative e-backtests for general risk measures develop a model-free sequential framework using e-values and e-processes to provide anytime-valid inference and robustness under dependence and model…
Key concepts
- e-values
- These are values derived from a sequential process that help in comparing different risk measures. They are used within the proposed framework to make decisions about which risk measure is superior at each step of the comparison.
- e-processes
- E-processes describe the steps or procedures involved in calculating and comparing risk measures sequentially. They form the backbone of the model-free framework, enabling sequential decision-making without needing a complex, predefined mathematical model.
- model misspecification
- This refers to situations where the mathematical model used to estimate risk might not perfectly match the actual behavior of financial data. The proposed framework is designed to be robust against this issue, meaning it can still provide reliable comparisons even if the initial assumptions about the data are slightly wrong.
- sequential framework
- This is a step-by-step approach to backtesting risk measures. Instead of testing all measures at once, this method tests them sequentially, allowing for continuous learning and inference as new information becomes available.
Terminology
Summary
Comparative e-backtests for general risk measures develop a model-free sequential framework using e-values and e-processes to provide anytime-valid inference and robustness under dependence and model misspecification The main contribution of this paper is four-fold, including developing a model-free sequential framework for comparative e-backtests of elicitable risk measures for financial regulatory practice
Framework Development
The authors develop a non-parametric sequential framework for comparative backtests of general elicitable risk measures using e-values and e-processes They propose a modified three-zone approach based on weak dominance, which yields more informative conclusions in comparative backtesting The framework is built upon the concept of e-values and e-processes, which allow for sequential anytime-valid inference and robustness to model misspecification and dependence
Key Contributions
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Develop a model-free sequential framework for comparative e-backtests of general elicitable risk measures using e-values and e-processes to provide anytime valid inference and robustness under dependence and model misspecification. This allows AI systems to perform real-time validation of internal models against benchmark models as new data arrives sequentially.
-
Implement a modified three-zone approach based on weak dominance for comparative backtesting, which yields more informative conclusions in comparing internal and standard models over time. This enables the AI to distinguish between cases where both traditional null hypotheses are rejected or not rejected simultaneously by analyzing the growth rate of e-processes.
-
Construct general standard e-backtests for identifiable risk measures, including mean, variance, Value-at-Risk, Expected Shortfall, and expectiles, characterized by their associated e-values and e-processes. This provides a technical building block for performing specific backtests on these common measures within the comparative framework.
-
Enable sequential monitoring of risk measure performance by constructing stochastic processes like Mt(λ) which are supermartingales under certain null hypotheses, allowing for rejection based on a pre-specified threshold 1/alpha. This supports dynamic evaluation of model performance during evolving market conditions.
-
Utilize the GREL method to determine optimal betting processes lambda t, which are asymptotically optimal when data are iid, to ensure the power of e-tests is guaranteed under proper choice conditions. This allows AI systems to select appropriate betting strategies for maximizing the expected log growth rate of the e-process.
-
Perform real-time comparative backtesting on live financial data by restarting the e-process whenever a rejection threshold is reached, enabling adaptation to structural changes like financial crises or pandemics in real time. This provides adaptive insights into which model performs better during periods of market turmoil.
Abstract
Backtesting risk measures is a central task in financial regulation. While traditional backtests evaluate whether a forecasting model is statistically consistent with observed losses, regulatory practice often requires assessing the performance of an internal model relative to standard models. In this paper, we develop a parametric anytime-valid framework for comparative backtests of general risk measures using e-values and e-processes. We construct likelihood-based e-processes for comparative e-backtests with validity and power guarantees. As practical guidelines, we propose the e-three-zone approach based on weak dominance, and introduce the overall and resetting test schemes for different practical purposes. Simulation studies and empirical analyses are conducted to show practical manipulations in financial regulation.
Sources
Related papers
- Learning Robust Treatment Rules for Censored Data
- Neural Estimation of Summary Statistics for Simulation-Based Inference with Non-Asymptotic Guarantees
- Spectral divide-and-conquer MCMC for long stationary time series
- Improving the Efficiency of Subgroup Analysis in Randomized Controlled Trials with TMLE
- Personalized Federated Vector Autoregression with Personalization Diversity
- Doubly Robust and Efficient Calibration of Prediction Sets for Right-Censored Time-to-Event Outcomes