A Stationary-Distribution Theory for Triplet-Based Plateau Search in Random Forest Ensemble-Size Selection

arXiv:2606.30837 · cs.LG, cs.AI, math.PR, stat.ML · Submitted 2026-06-29 · Read on arXiv

Andrey A. Dukhovny, Andrey M. Lange

cs.LG, cs.AI, math.PR, stat.ML

Submitted: 2026-06-29

Comments: 34 pages, 4 figures, 2 tables

License: http://creativecommons.org/licenses/by/4.0/

The gist: The number of trees is a central computational parameter in Random Forests: increasing it reduces finite-ensemble variability but increases training and prediction cost.

Terminology

Abstract

The number of trees is a central computational parameter in Random Forests: increasing it reduces finite-ensemble variability but increases training and prediction cost. Plateau-based tuning adapts this parameter through local comparisons of out-of-bag scores at a geometric triplet of tree counts. After the remaining hyperparameters have stabilized, however, the central triplet point need not converge to a deterministic value; instead, it fluctuates around a stationary regime. This paper develops a stationary-distribution theory for this process. The central ensemble size B t is modeled as a birth-death Markov chain on a geometric grid, and its stationary distribution is derived through local balance. Under a leading centered folded-normal approximation, equilibrium equations are obtained for the original update rule and a symmetric modified variant, implying that the stationary center B*=O(epsilon-2) as epsilon 0. The stationary spread is also characterized. A local Gaussian approximation and a Fokker-Planck interpretation give grid-level variance constants. After conversion to the ensemble-size scale, sigma B,*=O(epsilon-2), while the variance is O(epsilon-4). The leading relative spread is independent of epsilon and controlled by the scale factor and update rule. These results interpret plateau-based Random Forest tuning as a stochastic process rather than a deterministic stopping rule.

Sources

Related papers