Model Selection in High-Dimensional Linear Regression using Boosting with Multiple Testing

arXiv:2602.19705 · econ.EM · Submitted 2026-02-23 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.

Ruth: I'm Ruth.

Warren: And I'm Warren, here with Iris.

Ruth: We're looking at the paper: "Model Selection in High-Dimensional Linear Regression using Boosting with Multiple Testing".

Warren: Here's the gist of the paper. The paper proposes Boosting with Multiple Testing (BMT), a novel method that integrates forward stepwise variable selection (boosting) with a multiple testing framework derived from Chudik et al.

Ruth: (2018).

Warren: The core mechanism involves sequentially adding only the most significant regressor at each stage, conditional on previously selected variables, while simultaneously applying a family-wise multiple testing filter to the remaining candidates.

Ruth: This approach is designed to maintain strong screening properties while operating in a less greedy manner concerning proxy and noise variables.

Paper summary: Ruth: Okay, so to recap where we are: this paper proposes Boosting with Multiple Testing, or BMT. The core thesis is that you can combine forward stepwise variable selection with a multiple testing filter from Chudik et al., two thousand eighteen <ref:2602.19705#pg1>. This keeps the good screening properties but makes the process less greedy concerning proxy and noise variables.

Warren: It claims that using sharp probability inequalities for processes called heterogeneous strongly mixing, they can prove BMT has oracle type properties relative to an approximating model that includes all true signals and excludes pure noise variables.

Iris: The mechanism is sequential: you add only the most significant regressor conditionally, while applying a family-wise multiple testing filter to the remaining candidates at every stage. This is designed to maintain strong screening properties while being less greedy about proxy and noise variables.

Ruth: And they establish three main theoretical results from this framework. First, under assumptions similar to Chudik et al., BMT has oracle type properties relative to an approximating model containing all true signals and at most a finite number of proxy signals.

Warren: The second result is about the comparison with OCMT. They show that the sequential updating in BMT is less greedy than OCMT because there’s a positive probability of stopping before all proxy signal regressors are included.

Iris: And under a tractable signal-to-proxy dominance condition, BMT asymptotically recovers the full true model, while OCMT tends to retain all proxy variables that pass their marginal testing step. That's a big distinction in how they handle those non-essential predictors.

Ruth: The third result connects the selection process to estimation: the post-selection least squares estimator from the BMT-selected model is asymptotically equivalent to an infeasible oracle estimator if you knew the true model upfront.

Warren: So, what this means for us right now is that this method provides a principled way to handle model selection in high dimensions by balancing accuracy and stability across sequential steps. It’s not just one trick; it builds on established frameworks while refining the selection criteria for real-world complexity.

Iris: And the work is robust because it handles dependence structures like heterogeneous strongly mixing processes, and even allows for polynomial probability tails, which is a nice relaxation of standard assumptions.

Conclusion: Ruth: So looking at this paper, "Model Selection in High-Dimensional Linear Regression using Boosting with Multiple Testing," the authors are showing how to make the variable selection process more stable when you’re dealing with huge datasets where you have many predictors.

Warren: They are proposing BMT as a way to manage that complexity by combining boosting's ability to iteratively pick variables with a multiple testing filter that controls greediness in proxy variables.

Iris: The main implication is that this method gives us confidence that the selection process will converge towards the correct model structure under reasonable assumptions, even when things get complicated.

Ruth: It’s about moving away from just relying on penalized regression or purely greedy boosting algorithms when you have many strongly correlated variables; BMT offers a way to keep the estimation stable and interpretable.

Warren: For someone running these models, it means they can expect better model selection accuracy and more stable coefficient estimates than with current methods like OCMT or standard Lasso when multicollinearity is severe.

Iris: The limitation they state is that the recovery of the true model under their signal-to-proxy dominance condition is local and tailored specifically to that proxy-signal setting, which isn't as broad as global conditions needed for Lasso consistency.

Ruth: That’s a fair point. So, while BMT is powerful, you need to be mindful of the specific conditions under which it achieves its best results. It's a tool with defined boundaries rather than a magic solution for every single high-dimensional problem out there.

George Kapetanios, Vasilis Sarafidis, Alexia Ventouri

King’s College, London · Brunel University London

econ.EM

Submitted: 2026-02-23

Updated: 2026-11-03

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 92/100

The gist: The paper proposes Boosting with Multiple Testing (BMT), a novel method that integrates forward stepwise variable selection (boosting) with a multiple testing framework derived from Chudik et al.

Key concepts

Boosting with Multiple Testing (BMT)
A novel method that combines sequential variable selection (boosting) with a multiple testing filter. It selects the most important variable at each step based on previous selections, while simultaneously filtering remaining candidates to control false discoveries from noise and proxy variables.
Oracle-type Properties
This means BMT performs very well relative to an ideal model that includes all true signals and excludes pure noise. The theory shows that BMT tends to select the correct true model and achieves standard, optimal error rates for prediction and coefficient estimation.
Signal-to-Proxy Dominance Condition
A specific condition under which BMT is guaranteed to recover the full true model. This condition is tailored to the relationship between signal variables and proxy variables, making it a less strict requirement than global conditions needed for Lasso consistency.

Terminology

Summary

The paper proposes Boosting with Multiple Testing (BMT), a novel method that integrates forward stepwise variable selection (boosting) with a multiple testing framework derived from Chudik et al. p. 0

Where the paper says it

Model Selection in High-Dimensional Linear Regression using Boosting with Multiple Testing∗ George Kapetanios King’s College, London Vasilis Sarafidis Brunel University London Alexia Ventouri King’s College, London Abstract High-dimensional regression specification and analysis is a complex and active area of research in statistics, machine learning, and econometrics. This paper proposes a new approach, Boosting with Multiple Testing (BMT), which combines forward stepwise variable selection with the multiple testing framework of Chudik et al.

(2018). The core mechanism involves sequentially adding only the most significant regressor at each stage, conditional on previously selected variables, while simultaneously applying a family-wise multiple testing filter to the remaining candidates. p. 0
Where the paper says it

This paper proposes a new approach, Boosting with Multiple Testing (BMT), which combines forward stepwise variable selection with the multiple testing framework of Chudik et al. At each stage, the model is updated by adding only the most significant regressor conditional on those already included, while a family-wise multiple testing filter is applied to the remaining candidates.

This approach is designed to maintain strong screening properties while operating in a less greedy manner concerning proxy and noise variables. p. 0
Where the paper says it

In this way, the method retains the strong screening properties of Chudik et al. (2018) while operating in a less greedy manner with respect to proxy and noise variables.

The theoretical foundation of BMT relies on sharp probability inequalities for heterogeneous strongly mixing processes from Dendramis et al. p. 0

Where the paper says it

Using sharp probability inequalities for heterogeneous strongly mixing processes from Dendramis et al. (2021), we show that BMT enjoys oracle type properties relative to an approximating model that includes all true signals and excludes pure noise variables: this model is selected with probability tending to one, and the resulting estimator achieves standard parametric rates for prediction error and coefficient estimation.

(2021). Under these assumptions, BMT exhibits oracle-type properties relative to an approximating model that includes all true signals and excludes pure noise variables, meaning the selection process tends toward selecting the true model and achieving standard parametric rates for prediction error and coefficient estimation. p. 0
Where the paper says it

(2021), we show that BMT enjoys oracle type properties relative to an approximating model that includes all true signals and excludes pure noise variables: this model is selected with probability tending to one, and the resulting estimator achieves standard parametric rates for prediction error and coefficient estimation. Additional results establish conditions under which BMT recovers the exact true model and avoids selection of proxy signals.

Furthermore, additional results establish conditions under which BMT recovers the exact true model while avoiding the selection of proxy signals. p. 0
Where the paper says it

(2021), we show that BMT enjoys oracle type properties relative to an approximating model that includes all true signals and excludes pure noise variables: this model is selected with probability tending to one, and the resulting estimator achieves standard parametric rates for prediction error and coefficient estimation. Additional results establish conditions under which BMT recovers the exact true model and avoids selection of proxy signals.

The method establishes three main theoretical results:

  1. Under assumptions parallel to Chudik et al. p. 3

    Where the paper says it

    On the theoretical side, the paper establishes three main results. First, under assumptions that closely parallel and generalise those in Chudik et al.

    (2018), BMT possesses oracle-type properties with respect to an approximating model containing all true signals and at most a finite number of proxy signals, where the probability that any pure noise regressor is selected converges to zero, while the probability that every true signal is retained converges to one. p. 3
    Where the paper says it

    (2018), the proposed BMT procedure enjoys oracle-type properties with respect to an approximating model that contains all true signals and at most a finite number of proxy signals. In particular, the probability that any pure noise regressor is selected converges to zero, while the probability that every true signal is retained converges to one.

  2. The sequential updating in BMT makes the multiple-testing filter less greedy than in OCMT (Orthogonal Conditional Multiple Testing), with a positive probability of stopping before all proxy signal regressors are included. p. 3

    Where the paper says it

    Second, the sequential updating in BMT makes the multiple-testing filter less greedy than in OCMT. As a consequence, with positive probability, BMT stops before all proxy signal regressors are included.

    Under a tractable signal-to-proxy dominance condition, BMT asymptotically recovers the full true model, whereas OCMT tends to retain all proxy variables that pass the marginal testing step. p. 3
    Where the paper says it

    More generally, we derive a tractable signal-to-proxy dominance condition under which this no-proxy selection property extends to the case of multiple true signals. Under this condition, BMT asymptotically recovers the full true model, whereas OCMT continues to retain all proxy variables that pass the marginal testing step.

  3. The post-selection least squares estimator derived from the BMT-selected model is asymptotically equivalent to the infeasible oracle estimator obtained if the true model were known beforehand. p. 3

    Where the paper says it

    Under this condition, BMT asymptotically recovers the full true model, whereas OCMT continues to retain all proxy variables that pass the marginal testing step. 1 Third, it is shown that the post-selection least squares estimator based on the BMT-selected model is asymptotically equivalent to the infeasible oracle estimator that would be obtained if the true model were known in advance.

The framework is robust to dependence structures such as heterogeneous strongly mixing processes and allows for polynomial probability tails, relaxing standard subexponential tail assumptions. p. 8

Where the paper says it

The main departures are in the treatment of dependence and tail behaviour. Both regressors and errors are allowed to be heterogeneous strongly mixing processes rather than martingale difference sequences (Assumptions B-C), which permits dynamic specifications with autoregressive behaviour that are ruled out under the original OCMT conditions.

The signal-to-proxy dominance condition (Equation 28) under which BMT recovers the true model is local and tailored to the proxy-signal setting, making it less stringent than global design conditions required for Lasso consistency. p. 3
Where the paper says it

We show that the global design conditions required for Lasso consistency, such as the irrepresentable condition, are substantially more restrictive than the local, stage-wise dominance condition underpinning BMT. In a stylised proxy-signal design, this leads to a sharp region of the correlation space in which BMT recovers the true model with probability approaching one, while the Lasso is asymptotically inconsistent.

Empirical evidence from Monte Carlo experiments and simulation studies indicates that BMT performs very well relative to OCMT and Lasso-type procedures, yielding higher model selection accuracy and smaller Root Mean Square Error (RMSE) for coefficient estimates, especially when multicollinearity among regressors is strong. p. 0

Where the paper says it

Monte Carlo experiments indicate that BMT performs very well relative to OCMT and Lasso type procedures, delivering higher model selection accuracy and smaller RMSE for the estimated coefficients, especially under strong multicollinearity of the regressors. Two empirical illustrations based on a large set of macro–financial indicators as covariates, show that BMT yields sparse, interpretable specifications with favourable out-of-sample performance.

Specifically, BMT attains Mean Classifier Correlation (MCC) values close to one across various settings, demonstrating reliable detection of true signals with high precision and recall even in severely collinear, high-dimensional regimes. p. 4
Where the paper says it

MCC can be interpreted as a correlation between the true and selected inclusion indicators, aggregating all four classification outcomes (true positives, true negatives, false positives and false negatives) into a single balanced index that remains informative in sparse, highly unbalanced settings. Across all designs, BMT delivers MCC values close to one, with high TPR and TDR, low FPR and FDR, and model sizes that closely track the true sparsity level.

The method consistently delivers sparse, interpretable specifications with favorable out-of-sample performance compared to existing methods like OCMT and Lasso. p. 4
Where the paper says it

Across both applications, BMT achieves the strongest in-sample fit, selects sparse and interpretable specifications, and delivers lower out-of-sample forecast errors than OCMT and Lasso-based alternatives. These results highlight the ability of BMT to balance parsimony, interpretability, and predictive performance in high-dimensional settings.

The key implication is that proxy regressors do not necessarily need to remain in the selected model once relevant signals have been incorporated. p. 3

Where the paper says it

model selection criteria, so that the resulting procedure acts as a sharp filter for pure noise variables. A key implication of BMT is that proxy regressors need not remain in the selected model once the relevant signals have entered.

BMT prioritizes the largest conditional gain in fit at each step, which limits proxy accumulation and stabilizes estimation. p. 0
Where the paper says it

(2021), we show that BMT enjoys oracle type properties relative to an approximating model that includes all true signals and excludes pure noise variables: this model is selected with probability tending to one, and the resulting estimator achieves standard parametric rates for prediction error and coefficient estimation. Additional results establish conditions under which BMT recovers the exact true model and avoids selection of proxy signals.

The framework offers a principled alternative to both greedy multiple-testing procedures and penalized regression methods when dealing with numerous, strongly correlated, and potentially dynamic regressors. p. 1
Where the paper says it

These procedures construct the model in a stepwise fashion by adding or adjusting regressors sequentially in a direction that improves a chosen in-sample fit criterion. The aforementioned penalized-regression and greedy methods can be highly effective when the number of candidate regressors is very large.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Enhanced Model Selection Procedure

The AI system can implement Boosting with Multiple Testing (BMT), which combines forward stepwise variable selection with a family-wise multiple testing filter to select regressors sequentially, adding only one variable at a time per stage. This prevents the accumulation of proxy variables that are correlated with true signals, leading to more stable and interpretable final specifications.

  1. Oracle Type Performance Guarantee

The system can leverage the theoretical results showing that BMT enjoys oracle type properties relative to an approximating model including all true signals and excluding pure noise variables, meaning the probability that every true signal is retained converges to one. This ensures the estimator achieves standard parametric rates for prediction error and coefficient estimation.

  1. Superior Proxy Signal Mitigation

The system can exploit the signal-to-proxy dominance condition in Theorem 4, which provides a local, stage-wise comparison to ensure that true signals are selected ahead of proxies with probability approaching one. This allows the system to recover the exact true model by excluding proxy variables and noise variables with high probability.

  1. Robustness under Dependence

By utilizing sharp probability inequalities for heterogeneous strongly mixing processes from Dendramis et al. (2021), the AI system can handle non-trivial temporal dependence and heavier tailed distributions, which are often present in complex macroeconomic or financial time series data.

  1. Comparison with Existing Methods

The system can outperform greedy procedures like OCMT and penalized regression methods by delivering higher Matthews Correlation Coefficient values close to one and achieving smaller root mean squared errors for estimated coefficients, especially under strong multicollinearity of the regressors.

Abstract

This paper studies model selection in high dimensional regression settings. We propose a Boosting with Multiple Testing (BMT) approach designed to address proxy signal contamination in environments with strongly correlated regressors, including settings driven by latent common factors. At each stage, at most one regressor is selected conditional on those already included, using a multiple testing threshold for the remaining candidates. Under stated dependence, detectability and design conditions, BMT recovers the exact true model with probability approaching one. A separate extension permits growing signal counts under additional restrictions. For fixed numbers of signals and proxies, we also establish recovery of an approximating model containing all true signals and no noise variables under a separate set of conditions. Following exact selection, fixed dimensional least squares has the oracle asymptotic distribution under the stated moment conditions. A theoretical comparison identifies designs and tuning sequences for which BMT is consistent while the Lasso is not. In the main Monte Carlo comparison, BMT has the highest median Matthews correlation coefficient and exact recovery rate and the lowest median coefficient root mean squared error among the compared specifications. Applications to U.S. emissions and inflation show that BMT combines sparse specifications with favourable forecast performance, including the lowest emissions forecasting errors in the reported comparisons and the best one quarter ahead forecasts in the baseline inflation comparison.

Sources

Related papers