Comparative e-backtesting
summary
The gist
Comparative e-backtests for general risk measures develop a model-free sequential framework using e-values and e-processes to provide anytime-valid inference and robustness under dependence and model…
In short
This research develops a model-free sequential framework to compare different risk measures using e-values and e-processes. The framework allows for anytime-valid inference, meaning conclusions can be drawn at any point during the process, ensuring robustness even when the underlying financial models are imperfect or data exhibits dependence. This provides a practical method for regulatory practice.
Key concepts
- e-values
- These are values derived from a sequential process that help in comparing different risk measures. They are used within the proposed framework to make decisions about which risk measure is superior at each step of the comparison.
- e-processes
- E-processes describe the steps or procedures involved in calculating and comparing risk measures sequentially. They form the backbone of the model-free framework, enabling sequential decision-making without needing a complex, predefined mathematical model.
- model misspecification
- This refers to situations where the mathematical model used to estimate risk might not perfectly match the actual behavior of financial data. The proposed framework is designed to be robust against this issue, meaning it can still provide reliable comparisons even if the initial assumptions about the data are slightly wrong.
- sequential framework
- This is a step-by-step approach to backtesting risk measures. Instead of testing all measures at once, this method tests them sequentially, allowing for continuous learning and inference as new information becomes available.
Terminology used across episodes
This episode discusses
The paper
Comparative e-backtesting · Read on arXiv
Zhanyi Jiao, Qiuqi Wang, Yimiao Zhao
School of Mathematical and Statistical Sciences, Arizona State University · Maurice R. Greenberg School of Risk Science, Georgia State University · Royal Bank of Canada
Backtesting risk measures is a central task in financial regulation. While traditional backtests evaluate whether a forecasting model is statistically consistent with observed losses, regulatory practice often requires assessing the performance of an internal model relative to standard models. In this paper, we develop a parametric anytime-valid framework for comparative backtests of general risk measures using e-values and e-processes. We construct likelihood-based e-processes for comparative e-backtests with validity and power guarantees. As practical guidelines, we propose the e-three-zone approach based on weak dominance, and introduce the overall and resetting test schemes for different practical purposes. Simulation studies and empirical analyses are conducted to show practical manipulations in financial regulation.
Transcript
Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.
Tom: We're looking at the paper: "Comparative e-backtesting".
Jane: Here's the gist of the paper. Comparative e-backtests for general risk measures develop a model-free sequential framework using e-values and e-processes to provide anytime-valid inference and robustness under dependence and model misspecification The main contribution of this paper is four-fold, including developing a model-free sequential framework for comparative e-backtests of elicitable risk measures for financial regulatory practice Framework Development The authors develop a non-parametric sequential framework for comparative backtests of general elicitable risk measures using e-values and e-processes They propose a modified three-zone approach based on weak dominance, which yields more informative conclusions in comparative backtesting The framework is built upon the concept of e-values and e-processes, which allow for sequential anytime-valid inference and robustness to model misspecification and dependence Key Contributions
Paper summary: Tom: So, to recap, this paper is "Comparative e-backtesting." The core thesis is that they've developed a non-parametric sequential framework utilizing e-values and e-processes to perform comparative backtests of general elicitable risk measures.
Jane: And the key claim here is that these proposed methods deliver anytime-valid inference, meaning you can get reliable conclusions whenever you need them, and they maintain robustness against both model misspecification and dependence in the data >
Lu: They propose a specific technical modification to an existing three-zone approach called weak dominance. This modification is designed to produce more informative conclusions when comparing models against each other in this comparative testing environment >
Meng: So, it’s not just about a single test; it’s a whole sequential system that handles the comparisons dynamically as the data arrives. That sounds like a lot of computational coordination to manage correctly >
Lalam: From what I've seen, they characterize possible forms of e-values for standard e-tests of identifiable risk measures and also construct the associated e-processes as technical building blocks for this comparative framework >
Jane: That means they aren't just proposing a new way to run one test; they are giving you the tools to build a whole system around comparing models against each other >
Tom: And they show detailed procedures for common measures like Value-at-Risk, Expected Shortfall, and expectiles in their numerical studies which serve as examples for practical manipulation in financial regulation >
Meng: But how certain are these results? When you're doing comparative testing on real data, you have to worry about the assumptions they make regarding dependence and misspecification >
Conclusion: Tom: So, looking at "Comparative e-backtesting" by Zhanyi Jiao, Qiuqi Wang, and Yimiao Zhao March six two thousand twenty-six it seems the main implication is providing a more rigorous foundation for how financial regulators validate internal models against standard benchmarks > <ref:2511.05840#pg1,Yimiao Zhao March 6 2026>
Jane: It moves beyond just checking if a model is generally consistent with losses; it allows for a direct comparison between two different models in an adaptive way that handles uncertainty well >
Lu: The title emphasizes "comparative," which is important because the authors are not just testing one model in isolation, but setting up a framework to test hypotheses like whether one internal model is at least as accurate as another benchmark >
Meng: From a deployment standpoint, it suggests that if we can build these e-value and e-process characterizations into our systems, we get a more reliable way to assess risk management performance in production >
Lalam: The framework offers a path toward sequential anytime valid inference and robustness under dependence which is exactly what practitioners need when dealing with complex financial data >
More episodes
- Daily Summary for 2026-11-19
- 2310.07973-Statistical Performance Guarantee for Subgroup Identification with Generic Machine Learning
- 2408.09155-Learning Robust Treatment Rules for Censored Data
- Daily Summary for 2026-11-19
- 2603.23397-Kinetic Langevin Splitting Schemes for Constrained Sampling
- 2606.09906-An information-geometric framework for mapping maximum potential biodiversity
- 2603.27718-Induced replication and the assessment of models
- 1307.0898-Entropy of a Zipfian Distributed Lexicon
- 2406.19974-Generalizing self-normalized importance sampling with couplings
- 2412.10213-Collaborative Optimal Design of Controlled Experiments with Subject Covariates