L 0 Feature Selection via Monte Carlo Tree Search in High-Dimensional Financial Regression

arXiv:2610.13574 · cs.LG, q-fin.CP, q-fin.ST · Submitted 2026-10-09 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "L 0 Feature Selection via Monte Carlo Tree Search in High-Dimensional Financial Regression".

Jane: The gist The proposed method identifies sparse feature subsets with higher out-of-sample explanatory power than an optimized Lasso baseline,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So we're talking about the paper "L zero Feature Selection via Monte Carlo Tree Search in High-Dimensional Financial Regression" <ref:2610.13574#pg1,Feature Selection via Monte Carlo Tree Search in High-Dimensional Financial Regression>. The core idea here is using this adaptive search framework to pick specific feature subsets that do a better job on out-of-sample data compared to standard Lasso regularization.

Jane: That makes sense. The authors are proposing an approach that avoids the bias you get from convex shrinkage methods when you have a lot of variables and strong correlations among them.

Lu: The paper frames this as a sequential decision process, essentially setting up feature selection like a Markov Decision Process, where candidate subsets are scored by their out-ofsample validation performance rather than just minimizing the training error.

Meng: I'm curious about the scale of this search space. They mention there are two to the one thousand seventy-two possible subsets to check, which sounds computationally heavy for something we need to deploy quickly.

Lalam: The main result they highlight is that their ensemble average R squared hits zero point six nine six one, which is better than the Lasso baseline of zero point six three one three they used on the NASDAQ-one hundred index data > <ref:2610.13574#pg2>.

Tom: That improvement over the shrinkage method is what's interesting for me; it shows that myopic approaches can miss good combinations of variables that a simple penalty function would ignore because it forces everything to shrink equally.

Jane: So, if you only listen to this, the big picture is that for complex financial models, finding the right *set* of features matters more than just finding the best single coefficient value.

Lu: Exactly. The method also incorporates a Minimum Redundancy Maximum Relevance heuristic during the search to make sure they’re picking features that aren't just relevant but also not highly correlated with each other > <ref:2610.13574#pg3>.

Meng: I wonder how much tuning is actually required for this GNRPA policy adaptation, because if the computational cost and tuning burden are too high, it just doesn't work when we need to deploy something fast.

Lalam: The paper does admit that while their method finds these sparse subsets better out-ofsample, the process itself has a higher computational cost and tuning requirement than simpler convex baselines > <ref:2610.13574#pg3>.

Tom: So we have a method that gets better performance but demands more effort to set up and run. It’s like having a super accurate map, but the GPS needs constant recalibration on your phone.

Jane: The implication is that for researchers building these kinds of high-dimensional models, they have an alternative path away from just relying on traditional shrinkage techniques.

Lu: And the natural next step for this kind of work is applying this Monte Carlo Tree Search approach to other combinatorial problems in finance, like setting up index tracking with constraints > <ref:2610.13574#pg3>.

Meng: That’s interesting—moving it from regression feature selection to something like index tracking changes the scale of the problem entirely.

Lalam: The paper presents a feature selection framework based on reinforcement learning for high-dimensional financial regression, and its contributions include introducing an adaptive search framework that optimizes discrete subsets for out-ofsample performance, avoiding convex shrinkage > <ref:2610.13574#pg1>.

Tom: That means they are actively searching through the feature space rather than just applying a fixed penalty to every single variable.

Jane: They are trying to solve that problem where too many features make a model unstable or hard to read by using this search method instead of just shrinking all the coefficients down > <ref:2610.13574#pg1>.

Lu: They formulate it as a sequential decision problem, and they use GNRPA to approximate L zero optimization using validation-guided tree search > <ref:2610.13574#pg2>.

Meng: From my side, I’m interested in how they handle the search space—they're looking at something like two one thousand seventy-two possible subsets. That sounds computationally heavy for real-world deployment.

Lalam: The key result is that their ensemble average R squared hits zero point six nine six one, which is better than the Lasso baseline of zero point six three one three they used in comparison on the NASDAQ-one hundred index data > <ref:2610.13574#pg2>.

Tom: That improvement over the shrinkage method is what's neat for me; it shows that myopic approaches can miss good combinations of variables that a simple penalty function would ignore because it forces everything to shrink equally.

Jane: The implication is that for researchers building these kinds of high-dimensional models, they have an alternative path away from just relying on traditional shrinkage techniques > <ref:2610.13574#pg2>.

Meng: That’s interesting—moving it from regression feature selection to something like index tracking changes the scale of the problem entirely.

Conclusion: Tom: So we’re looking at "L zero Feature Selection via Monte Carlo Tree Search in High-Dimensional Financial Regression" and this paper is about finding specific groups of features that perform better than standard Lasso regularization when you have a lot of variables.

Jane: The authors, they are proposing this adaptive search framework to find those discrete subsets, which avoids the bias you get from convex shrinkage methods like Lasso by looking at out-of-sample performance instead of just training error.

Lu: They frame the whole thing as a Markov Decision Process, an episodic one where candidate feature subsets get scored using out-of-sample validation performance.

Meng: It's interesting because they incorporate a Minimum Redundancy Maximum Relevance heuristic during the search to make sure they’re picking features that aren't just relevant but also not highly correlated with each other.

Lalam: The ensemble average R squared of zero point six nine six one is the main result they highlight for this specific high-dimensional financial regression problem, which beats the Lasso baseline of zero point six three one three on their NASDAQ-one hundred index data.

Tom: That improvement over the shrinkage method is what’s neat for me; it shows that myopic approaches can miss good combinations of variables that a simple penalty function would ignore because it forces everything to shrink equally.

Jane: So, if you only listen to this, the big picture is that for complex financial models, finding the right *set* of features matters more than just finding the best single coefficient value.

Lu: Exactly. The method also incorporates that Minimum Redundancy Maximum Relevance heuristic during the search to make sure they’re picking features that aren't just relevant but also not highly correlated with each other.

Lalam: The paper admits that while their method finds these sparse subsets better out-of-sample, the process itself has a higher computational cost and tuning requirement than simpler convex baselines.

Tom: So we have a method that gets better performance but demands more effort to set up and run, like having a super accurate map but the GPS needs constant recalibration on your phone.

Jane: The implication is that for researchers building these kinds of high-dimensional models, they have an alternative path away from just relying on traditional shrinkage techniques.

Lu: And the natural next step for this kind of work is applying this Monte Carlo Tree Search approach to other combinatorial problems in finance, like setting up index tracking with constraints.

Cameron Mouangue, Tristan Cazenave

LAMSADE, University Paris Dauphine-PSL, CNRS

cs.LG, q-fin.CP, q-fin.ST

Submitted: 2026-10-09

Updated: 2026-10-09

Comments: Accepted at the 7th ACM International Conference on AI in Finance (ICAIF '26). 8 pages, 2 figures

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

The gist: The gist The proposed method identifies sparse feature subsets with higher out-of-sample explanatory power than an optimized Lasso baseline, reaching an out-of-sample R2 of 0.6961 versus 0.6313 for

Key concepts

Lasso: Lasso is a regularization technique used in high-dimensional regression that shrinks some coefficients to zero, effectively performing feature selection. While popular for its sparsity, it can be restrictive when important signals are spread across correlated variables.
Monte Carlo Tree Search (MCTS): MCTS is an AI search algorithm used here as a framework to explore the vast space of possible feature subsets. It simulates decisions by building a search tree, allowing the system to navigate complex combinatorial problems effectively without exhaustive searching.
GNRPA: Generalized Nested Rollout Policy Adaptation (GNRPA) is the specific policy adaptation used within MCTS. It incorporates a Minimum Redundancy Maximum Relevance (mRMR) heuristic to guide the search, balancing the need for relevant features with avoiding highly correlated ones.
Out-of-Sample R²: This metric measures how well a model performs on new, unseen data after being trained. The paper focuses on maximizing this value because it indicates that the selected feature subset has genuine predictive power beyond just fitting the training data.

Terminology

Feature Selection — 1× in this paper · explained in 4 episodes

Summary

The gist The proposed method identifies sparse feature subsets with higher out-of-sample explanatory power than an optimized Lasso baseline, reaching an out-of-sample R2 of 0.6961 versus 0.6313 for Lasso

Problem and Motivation

Feature selection is a central problem in high-dimensional financial regression, where large numbers of engineered variables and strong correlations among predictors reduce model stability and interpretability L1 regularization (Lasso) is widely used for sparse regression, but its shrinkage can be restrictive when informative signals are spread across correlated variables Exact sparse regression avoids coefficient shrinkage but requires solving a combinatorial optimization problem that is computationally difficult and is typically fit to minimize in-sample rather than out-of-sample error This paper presents a feature selection framework based on Monte Carlo Tree Search with Generalized Nested Rollout Policy Adaptation (GNRPA) Feature selection is formulated as a sequential decision process, and candidate subsets are scored by their out-ofsample validation performance rather than their training error

Methodology

The framework formulates sparse feature selection as an episodic Markov Decision Process (MDP) defined by the tuple (S, A, R) The state is st = (flast, St), where St subset F is the current subset and flast is the most recently selected feature The action set A (st) = (F - St) U astop Rewards are evaluated only at terminal states For a feature subset S, the regression coefficients are estimated by ordinary least squares using a ridge-stabilized matrix inversion via NumPy The resulting model is evaluated on a separate validation set, and the corresponding validation performance defines the reward used during search

Search Heuristics

The complete search space contains 2 1072 ≈ 10322 possible subsets To effectively navigate this space, the paper incorporates a Minimum Redundancy Maximum Relevance (mRMR) heuristic into the GNRPA bias term β defined in Eq. (1) The heuristic score combines relevance and redundancy using a tradeoff parameter γ ∈ [0, 1], and is scaled by a magnitude parameter βmag to give the bias used in Gibbs sampling The reward R (S) = ⌊R2 val(S)⌉3, −S, −MaxVIF(S)

Experimental Results

The proposed method was evaluated on the contemporaneous modeling task of explaining the weekly return (5-day rolling sum) of the NASDAQ-100 index The mean out-of-sample R2 of individual GNRPA runs (0.6428) exceeded the Lasso baseline, suggesting that MCTS is better able to identify variable combinations that myopic or shrinkage-based methods might obscure The endogenous stopping configuration reaches a mean R2 of 0.6682 (p < 0.001 vs. Lasso), and even its worst run (0.650, Table 3) beats Lasso The ensemble-averaged R2 in Table 3 remains our primary evaluation metric throughout this paper, reflecting the reliable performance an independent user can expect

Conclusion

The contributions of this study are threefold First, it introduces an adaptive search framework that optimizes discrete subsets for out-ofsample performance, avoiding convex shrinkage Second, it demonstrates that Monte Carlo Tree Search readily accommodates domain-specific feature selection objectives, using the maximum Variance Inflation Factor to favor subsets with lower internal multicollinearity when explanatory performance is similar Third, it shows that the method finds sparse, interpretable feature subsets that outperform the convex baselines out of sample on this dataset A linear ensemble of these subsets reaches an out-of-sample R2 = 0.6961 while preserving additive performance attribution The improved out-of-sample fit comes at a higher computational cost and tuning burden than convex baselines Finally, the consistent validation and test gains over convex baselines support the method’s efficacy The natural extension of this work is applying MCTS to other combinatorial problems in finance, such as cardinality-constrained index tracking The GNRPA builds a solution incrementally and adapts its policy during the search This approach avoids the shrinkage bias of convex regularization methods such as Lasso, while retaining the interpretability of linear models The method finds sparse, interpretable feature subsets that outperform the convex baselines out of sample on this dataset This paper presents a feature selection framework, based on reinforcement learning, for high-dimensional financial regression The contributions of this study are threefold First, it introduces an adaptive search framework that optimizes discrete subsets for out-ofsample performance, avoiding convex shrinkage Second, it demonstrates that Monte Carlo Tree Search readily accommodates domain-specific feature selection objectives, using the maximum Variance Inflation Factor to favor subsets with lower internal multicollinearity when explanatory performance is similar Third, it shows that the method finds sparse, interpretable feature subsets that outperform the convex baselines out of sample on this dataset A linear ensemble of these subsets reaches an out-of-sample R2 = 0.6961 while preserving additive performance attribution This paper presents a feature selection framework, based on reinforcement learning, for high-dimensional financial regression The contributions of this study are threefold First, it introduces an adaptive search framework that optimizes discrete subsets for out-ofsample performance, avoiding convex shrinkage Second, it demonstrates that Monte Carlo Tree Search readily accommodates domain-specific feature selection objectives, using the maximum Variance Inflation Factor to favor subsets with lower internal multicollinearity when explanatory performance is similar Third, it shows that the method finds sparse, interpretable feature subsets that outperform the convex baselines out of sample on this dataset A linear ensemble of these subsets reaches an out-of-sample R2 = 0.

Improvements for AI systems

Bold header: Adaptive Feature Selection Framework

This system can identify sparse feature subsets with higher out-of-sample explanatory power than an optimized Lasso baseline, moving beyond the limitations of coefficient shrinkage by evaluating candidate subsets using validation performance during the search.

Bold header: Lexicographic Objective Optimization

The system supports a lexicographic objective that accounts for subset size and multicollinearity during the search, allowing it to simultaneously optimize for sparsity, minimize redundancy, and penalize high variance in coefficient estimates via MaxVIF.

Bold header: Endogenous Stopping Mechanism

The framework implements an endogenous stopping mechanism where the search policy determines the stopping depth automatically, avoiding pre-specification of cardinality by using a heuristic bias that grows with the subset’s proximity to the maximum capacity.

Bold header: Ensemble Stability and Interpretation

By averaging predictions from independent OLS regressions (soft voting), the system achieves an ensemble out-of-sample R2 of 0.6961, which preserves linear performance attribution while providing a more reliable estimate than any single stochastic search run.

Bold header: Preservation of Non-Convex Interactions

The L0 selection process allows OLS to assign large, opposing weights to correlated variables to construct spread signals, which convex penalties like Lasso might fail to capture, leading to the identification of interpretable distributed-lag models.

Sources

The Montparnasse Algorithm for RNA Design