Model Selection in High-Dimensional Linear Regression using Boosting with Multiple Testing
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.
Ruth: I'm Ruth.
Warren: And I'm Warren, here with Iris.
Ruth: We're looking at the paper: "Model Selection in High-Dimensional Linear Regression using Boosting with Multiple Testing".
Warren: Here's the gist of the paper. The paper proposes Boosting with Multiple Testing (BMT), a novel method that integrates forward stepwise variable selection (boosting) with a multiple testing framework derived from Chudik et al.
Ruth: (2018).
Warren: The core mechanism involves sequentially adding only the most significant regressor at each stage, conditional on previously selected variables, while simultaneously applying a family-wise multiple testing filter to the remaining candidates.
Ruth: This approach is designed to maintain strong screening properties while operating in a less greedy manner concerning proxy and noise variables.
Paper summary: Ruth: Okay, so to recap where we are: this paper proposes Boosting with Multiple Testing, or BMT. The core thesis is that you can combine forward stepwise variable selection with a multiple testing filter from Chudik et al., two thousand eighteen <ref:2602.19705#pg1>. This keeps the good screening properties but makes the process less greedy concerning proxy and noise variables.
Warren: It claims that using sharp probability inequalities for processes called heterogeneous strongly mixing, they can prove BMT has oracle type properties relative to an approximating model that includes all true signals and excludes pure noise variables.
Iris: The mechanism is sequential: you add only the most significant regressor conditionally, while applying a family-wise multiple testing filter to the remaining candidates at every stage. This is designed to maintain strong screening properties while being less greedy about proxy and noise variables.
Ruth: And they establish three main theoretical results from this framework. First, under assumptions similar to Chudik et al., BMT has oracle type properties relative to an approximating model containing all true signals and at most a finite number of proxy signals.
Warren: The second result is about the comparison with OCMT. They show that the sequential updating in BMT is less greedy than OCMT because there’s a positive probability of stopping before all proxy signal regressors are included.
Iris: And under a tractable signal-to-proxy dominance condition, BMT asymptotically recovers the full true model, while OCMT tends to retain all proxy variables that pass their marginal testing step. That's a big distinction in how they handle those non-essential predictors.
Ruth: The third result connects the selection process to estimation: the post-selection least squares estimator from the BMT-selected model is asymptotically equivalent to an infeasible oracle estimator if you knew the true model upfront.
Warren: So, what this means for us right now is that this method provides a principled way to handle model selection in high dimensions by balancing accuracy and stability across sequential steps. It’s not just one trick; it builds on established frameworks while refining the selection criteria for real-world complexity.
Iris: And the work is robust because it handles dependence structures like heterogeneous strongly mixing processes, and even allows for polynomial probability tails, which is a nice relaxation of standard assumptions.
Conclusion: Ruth: So looking at this paper, "Model Selection in High-Dimensional Linear Regression using Boosting with Multiple Testing," the authors are showing how to make the variable selection process more stable when you’re dealing with huge datasets where you have many predictors.
Warren: They are proposing BMT as a way to manage that complexity by combining boosting's ability to iteratively pick variables with a multiple testing filter that controls greediness in proxy variables.
Iris: The main implication is that this method gives us confidence that the selection process will converge towards the correct model structure under reasonable assumptions, even when things get complicated.
Ruth: It’s about moving away from just relying on penalized regression or purely greedy boosting algorithms when you have many strongly correlated variables; BMT offers a way to keep the estimation stable and interpretable.
Warren: For someone running these models, it means they can expect better model selection accuracy and more stable coefficient estimates than with current methods like OCMT or standard Lasso when multicollinearity is severe.
Iris: The limitation they state is that the recovery of the true model under their signal-to-proxy dominance condition is local and tailored specifically to that proxy-signal setting, which isn't as broad as global conditions needed for Lasso consistency.
Ruth: That’s a fair point. So, while BMT is powerful, you need to be mindful of the specific conditions under which it achieves its best results. It's a tool with defined boundaries rather than a magic solution for every single high-dimensional problem out there.
George Kapetanios, Vasilis Sarafidis, Alexia Ventouri
King’s College, London · Brunel University London
econ.EM
Submitted: 2026-02-23
Updated: 2026-11-03
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 92/100
The gist: The paper proposes Boosting with Multiple Testing (BMT), a novel method that integrates forward stepwise variable selection (boosting) with a multiple testing framework derived from Chudik et al.
Key concepts
- Boosting with Multiple Testing (BMT)
- A novel method that combines sequential variable selection (boosting) with a multiple testing filter. It selects the most important variable at each step based on previous selections, while simultaneously filtering remaining candidates to control false discoveries from noise and proxy variables.
- Oracle-type Properties
- This means BMT performs very well relative to an ideal model that includes all true signals and excludes pure noise. The theory shows that BMT tends to select the correct true model and achieves standard, optimal error rates for prediction and coefficient estimation.
- Signal-to-Proxy Dominance Condition
- A specific condition under which BMT is guaranteed to recover the full true model. This condition is tailored to the relationship between signal variables and proxy variables, making it a less strict requirement than global conditions needed for Lasso consistency.
Terminology
Summary
The paper proposes Boosting with Multiple Testing (BMT), a novel method that integrates forward stepwise variable selection (boosting) with a multiple testing framework derived from Chudik et al. p. 0 Model Selection in High-Dimensional Linear Regression
using Boosting with Multiple Testing∗
George Kapetanios
King’s College, London
Vasilis Sarafidis
Brunel University London
Alexia Ventouri
King’s College, London
Abstract
High-dimensional regression specification and analysis is a complex and active area of
research in statistics, machine learning, and econometrics. This paper proposes a new approach, Boosting with Multiple Testing (BMT), which combines forward stepwise variable
selection with the multiple testing framework of Chudik et al. This paper proposes a new approach, Boosting with Multiple Testing (BMT), which combines forward stepwise variable
selection with the multiple testing framework of Chudik et al. At each stage, the
model is updated by adding only the most significant regressor conditional on those already
included, while a family-wise multiple testing filter is applied to the remaining candidates. In this way, the method retains the strong screening properties of Chudik et al. (2018)
while operating in a less greedy manner with respect to proxy and noise variables.Where the paper says it
Where the paper says it
Where the paper says it
The theoretical foundation of BMT relies on sharp probability inequalities for heterogeneous strongly mixing processes from Dendramis et al. p. 0 Using
sharp probability inequalities for heterogeneous strongly mixing processes from Dendramis
et al. (2021), we show that BMT enjoys oracle type properties relative to an approximating model that includes all true signals and excludes pure noise variables: this model
is selected with probability tending to one, and the resulting estimator achieves standard
parametric rates for prediction error and coefficient estimation. (2021), we show that BMT enjoys oracle type properties relative to an approximating model that includes all true signals and excludes pure noise variables: this model
is selected with probability tending to one, and the resulting estimator achieves standard
parametric rates for prediction error and coefficient estimation. Additional results establish
conditions under which BMT recovers the exact true model and avoids selection of proxy
signals. (2021), we show that BMT enjoys oracle type properties relative to an approximating model that includes all true signals and excludes pure noise variables: this model
is selected with probability tending to one, and the resulting estimator achieves standard
parametric rates for prediction error and coefficient estimation. Additional results establish
conditions under which BMT recovers the exact true model and avoids selection of proxy
signals.Where the paper says it
Where the paper says it
Where the paper says it
The method establishes three main theoretical results:
-
Under assumptions parallel to Chudik et al. p. 3
(2018), BMT possesses oracle-type properties with respect to an approximating model containing all true signals and at most a finite number of proxy signals, where the probability that any pure noise regressor is selected converges to zero, while the probability that every true signal is retained converges to one. p. 3Where the paper says it
On the theoretical side, the paper establishes three main results. First, under assumptions that closely parallel and generalise those in Chudik et al.
Where the paper says it
(2018), the proposed BMT procedure enjoys oracle-type properties with respect to an approximating model that contains all true signals and at most a finite number of proxy signals. In particular, the probability that any pure noise regressor is selected converges to zero, while the probability that every true signal is retained converges to one.
-
The sequential updating in BMT makes the multiple-testing filter less greedy than in OCMT (Orthogonal Conditional Multiple Testing), with a positive probability of stopping before all proxy signal regressors are included. p. 3
Under a tractable signal-to-proxy dominance condition, BMT asymptotically recovers the full true model, whereas OCMT tends to retain all proxy variables that pass the marginal testing step. p. 3Where the paper says it
Second, the sequential updating in BMT makes the multiple-testing filter less greedy than in OCMT. As a consequence, with positive probability, BMT stops before all proxy signal regressors are included.
Where the paper says it
More generally, we derive a tractable signal-to-proxy dominance condition under which this no-proxy selection property extends to the case of multiple true signals. Under this condition, BMT asymptotically recovers the full true model, whereas OCMT continues to retain all proxy variables that pass the marginal testing step.
-
The post-selection least squares estimator derived from the BMT-selected model is asymptotically equivalent to the infeasible oracle estimator obtained if the true model were known beforehand. p. 3
Where the paper says it
Under this condition, BMT asymptotically recovers the full true model, whereas OCMT continues to retain all proxy variables that pass the marginal testing step. 1 Third, it is shown that the post-selection least squares estimator based on the BMT-selected model is asymptotically equivalent to the infeasible oracle estimator that would be obtained if the true model were known in advance.
The framework is robust to dependence structures such as heterogeneous strongly mixing processes and allows for polynomial probability tails, relaxing standard subexponential tail assumptions. p. 8 The main departures are in the treatment of dependence and tail behaviour. Both regressors
and errors are allowed to be heterogeneous strongly mixing processes rather than martingale
difference sequences (Assumptions B-C), which permits dynamic specifications with autoregressive behaviour that are ruled out under the original OCMT conditions. We show that the global design conditions required
for Lasso consistency, such as the irrepresentable condition, are substantially more restrictive
than the local, stage-wise dominance condition underpinning BMT. In a stylised proxy-signal
design, this leads to a sharp region of the correlation space in which BMT recovers the true
model with probability approaching one, while the Lasso is asymptotically inconsistent.Where the paper says it
Where the paper says it
Empirical evidence from Monte Carlo experiments and simulation studies indicates that BMT performs very well relative to OCMT and Lasso-type procedures, yielding higher model selection accuracy and smaller Root Mean Square Error (RMSE) for coefficient estimates, especially when multicollinearity among regressors is strong. p. 0 Monte Carlo experiments indicate that BMT performs very well relative to OCMT
and Lasso type procedures, delivering higher model selection accuracy and smaller RMSE
for the estimated coefficients, especially under strong multicollinearity of the regressors. Two empirical illustrations based on a large set of macro–financial indicators as covariates,
show that BMT yields sparse, interpretable specifications with favourable out-of-sample
performance. MCC can be interpreted as a correlation between the true and selected inclusion indicators, aggregating all four
classification outcomes (true positives, true negatives, false positives and false negatives) into
a single balanced index that remains informative in sparse, highly unbalanced settings. Across
all designs, BMT delivers MCC values close to one, with high TPR and TDR, low FPR and
FDR, and model sizes that closely track the true sparsity level. Across both applications, BMT achieves the strongest in-sample fit,
selects sparse and interpretable specifications, and delivers lower out-of-sample forecast errors
than OCMT and Lasso-based alternatives. These results highlight the ability of BMT to balance
parsimony, interpretability, and predictive performance in high-dimensional settings.Where the paper says it
Where the paper says it
Where the paper says it
The key implication is that proxy regressors do not necessarily need to remain in the selected model once relevant signals have been incorporated. p. 3 model selection criteria, so that the resulting procedure acts as a sharp filter for pure noise
variables. A key implication of BMT is that proxy regressors need not remain in the selected model
once the relevant signals have entered. (2021), we show that BMT enjoys oracle type properties relative to an approximating model that includes all true signals and excludes pure noise variables: this model
is selected with probability tending to one, and the resulting estimator achieves standard
parametric rates for prediction error and coefficient estimation. Additional results establish
conditions under which BMT recovers the exact true model and avoids selection of proxy
signals. These procedures construct the
model in a stepwise fashion by adding or adjusting regressors sequentially in a direction that
improves a chosen in-sample fit criterion. The aforementioned penalized-regression and greedy methods can be highly effective when
the number of candidate regressors is very large.Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
- Enhanced Model Selection Procedure
The AI system can implement Boosting with Multiple Testing (BMT), which combines forward stepwise variable selection with a family-wise multiple testing filter to select regressors sequentially, adding only one variable at a time per stage. This prevents the accumulation of proxy variables that are correlated with true signals, leading to more stable and interpretable final specifications.
- Oracle Type Performance Guarantee
The system can leverage the theoretical results showing that BMT enjoys oracle type properties relative to an approximating model including all true signals and excluding pure noise variables, meaning the probability that every true signal is retained converges to one. This ensures the estimator achieves standard parametric rates for prediction error and coefficient estimation.
- Superior Proxy Signal Mitigation
The system can exploit the signal-to-proxy dominance condition in Theorem 4, which provides a local, stage-wise comparison to ensure that true signals are selected ahead of proxies with probability approaching one. This allows the system to recover the exact true model by excluding proxy variables and noise variables with high probability.
- Robustness under Dependence
By utilizing sharp probability inequalities for heterogeneous strongly mixing processes from Dendramis et al. (2021), the AI system can handle non-trivial temporal dependence and heavier tailed distributions, which are often present in complex macroeconomic or financial time series data.
- Comparison with Existing Methods
The system can outperform greedy procedures like OCMT and penalized regression methods by delivering higher Matthews Correlation Coefficient values close to one and achieving smaller root mean squared errors for estimated coefficients, especially under strong multicollinearity of the regressors.
Abstract
This paper studies model selection in high dimensional regression settings. We propose a Boosting with Multiple Testing (BMT) approach designed to address proxy signal contamination in environments with strongly correlated regressors, including settings driven by latent common factors. At each stage, at most one regressor is selected conditional on those already included, using a multiple testing threshold for the remaining candidates. Under stated dependence, detectability and design conditions, BMT recovers the exact true model with probability approaching one. A separate extension permits growing signal counts under additional restrictions. For fixed numbers of signals and proxies, we also establish recovery of an approximating model containing all true signals and no noise variables under a separate set of conditions. Following exact selection, fixed dimensional least squares has the oracle asymptotic distribution under the stated moment conditions. A theoretical comparison identifies designs and tuning sequences for which BMT is consistent while the Lasso is not. In the main Monte Carlo comparison, BMT has the highest median Matthews correlation coefficient and exact recovery rate and the lowest median coefficient root mean squared error among the compared specifications. Applications to U.S. emissions and inflation show that BMT combines sparse specifications with favourable forecast performance, including the lowest emissions forecasting errors in the reported comparisons and the best one quarter ahead forecasts in the baseline inflation comparison.
Sources
Related papers
- Stationarity with Occasionally Binding Constraints
- Backtesting High-Dimensional Expected Shortfall
- Higher-Order Neyman Orthogonality in Moment-Condition Models
- Online Bayesian Inference for a Dirichlet Process Mixture of Binary Logit Models
- Debiased Conditional Independence Tests for Causal Effects
- Sharp Testable Implications of Encouragement Designs