LoBoost: Model-Native Local Conformal Prediction for Gradient-Boosted Trees

arXiv:2602.22432 · stat.ML, cs.LG · Submitted 2026-02-25 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "LoBoost: Model-Native Local Conformal Prediction for Gradient-Boosted Trees".

Jane: Here's the gist of the paper. Gradient-boosted decision trees are among the strongest off-the-shelf predictors for tabular regression, but point predictions alone do not quantify uncertainty.

Tom: LoBoost proposes a model-native local conformal method that reuses the fitted ensemble’s leaf structure to define a multiscale partition of the feature space.

Jane: This approach allows for locally adaptive prediction intervals by estimating residual quantiles within these model-induced cells, requiring no auxiliary partition or retraining.

Tom: Model-Native Locality and Partitioning LoBoost leverages the sequential nature of gradient boosting to induce a hierarchical organization of the feature space, where early trees capture coarse structure and later trees introduce finer refinements.

Title and authors: Tom: So, what's the actual mechanism behind this model-native local conformal prediction approach? Jane, can you translate that for someone who isn't deep in the math? What is LoBoost doing differently compared to standard split conformal methods we’ve seen before?

Jane: Well, standard split conformal uses one single global residual quantile across the whole feature space. This often leads to intervals that are way too wide in places where things are noisy, or not adaptive enough when the noise changes depending on where you look <ref:2602.22432#pg1>. LoBoost fixes this by creating these nested partitions based on how well the first few trees classify a data point.

Lu: It leverages the sequential nature of gradient boosting; early trees capture the big structure, and later trees add finer details. By matching the leaf identifiers from a specific set of early trees, they define a local region around your input data <ref:2602.22432#pg1>.

Meng: So it’s essentially saying, "Okay, for this specific area in the feature space defined by these tree splits, we're going to estimate a different residual quantile than for some other area." That sounds like it could be very efficient if implemented well.

Lalam: Yes, they are estimating separate residual quantiles within those model-induced cells and then using those local estimates to define the prediction interval conditional on that specific cell <ref:2602.22432#pg1>. It’s a way to get local calibration without needing an auxiliary partition or retraining the base learner.

The paper's summary: Tom: Okay, so we know it uses the internal structure to create local prediction intervals, but what are the actual theoretical guarantees they provide about how good those intervals are? I want to hear about the validity and coverage control.

Jane: The theory connects the stability of that fitted ensemble and how well those induced cells localize residuals to things called local residual score homogeneity <ref:2602.22432#pg1>. This gives them finite-sample control over the error rate, meaning they can bound how often their intervals miss the true value.

Lu: Proposition one shows that when you are inside a region and the tail envelope is small, the nonconformity score—which is just how far your prediction is from the actual observed value—stays roughly constant across those model partitions <ref:2602.22432#pg1>.

Meng: That sounds promising because if that score stays consistent within a region, it means the method isn't wildly guessing its uncertainty based on local noise spikes; it’s using the structure to stay stable. But what happens when we have really complex interactions?

Lalam: Theorem one establishes finite-sample group-conditional validity for every cell in that partition with positive probability <ref:2602.22432#pg1>. And they show that asymptotic conditional coverage is achieved when the cell gets more and more calibration observations and its score distribution becomes homogeneous around the data point, specifically when ncal, npn(x) p goes to infinity and delta n(x) p goes to zero <ref:2602.22432#pg1>.

The paper's improvements: Tom: Alright, so to wrap up the whole thing. What’s the big picture here? LoBoost is bridging that gap between standard global methods and those heavy, auxiliary conformal methods by reusing what the model has already learned about its own structure.

Jane: It moves us toward a method that provides locally adaptive intervals for boosted trees without needing to re-train the core predictor or build a separate partition model <ref:2602.22432#pg1>.

Lu: The main implication is that we can handle heteroscedasticity and discontinuities in the data much better than just using one global residual quantile, as shown by their comparison with standard split conformal in Figure one <ref:2602.22432#pg3>.

Meng: From a deployment view, the computational gains are interesting; some settings show post-fit speedups up to one hundred times faster than heavier rules like MICP and WICP, which makes it feasible for real-time systems.

Lalam: The limitation they point out is their reliance on absolute residual nonconformity scores, which means the prediction intervals they produce are symmetric around the point estimate and might not be ideal if your response data is skewed <ref:2602.22432#pg1>.

Tom: So, to summarize this paper "LoBoost: Model-Native Local Conformal Prediction for Gradient-Boosted Trees," it’s a method that uses the model's internal tree structure to define local regions for estimating uncertainty, leading to better coverage control than global methods, but we have some known limitations with response skew.

Jane: It’s definitely a solid step toward making uncertainty quantification practical and fast for these kinds of powerful boosted models. We'll keep an eye on how they extend this idea into more asymmetric scores next time.

Conclusion: Tom: So we’ve talked about how LoBoost uses the internal tree structure to build local regions for uncertainty estimates, and now we’re wrapping up this paper, "LoBoost: Model-Native Local Conformal Prediction for Gradient-Boosted Trees."

Jane: Basically, it takes a fitted gradient boosted tree ensemble and lets you get a prediction interval that adapts locally based on how the trees are structured, without needing to build any extra partition model or retrain the main tree <ref:2602.22432#pg1>.

Lu: The core idea is leveraging the sequential nature of boosting; early trees define coarse structure, and later ones add finer detail. This creates a nested partition on the feature space where you can estimate different residual quantiles in each cell <ref:2602.22432#pg1>.

Meng: From an engineering standpoint, the efficiency is what stands out; some settings show speedups up to one hundred times faster than heavier methods like MICP or WICP, which means this could actually be deployable in production environments that need fast uncertainty checks.

Lalam: And theoretically, they provide guarantees on coverage control. They show that if you get enough calibration observations and the cell gets refined enough, the interval will hit the true value about one minus alpha percent of the time <ref:2602.22432#pg1>.

Tom: That's a big deal for practical deployment because we’re used to those auxiliary models being computationally expensive, so reusing the fitted structure makes it much more accessible. But Jane mentioned something about limitations with response skew.

Jane: Exactly, LoBoost is built around absolute residual scores, which means the intervals will be symmetric around your point estimate, and that might not be perfect if your actual data is skewed <ref:2602.22432#pg1>.

Lu: That's a known trade-off; they are relying on those specific score types, so you have to be careful with the shape of your output distribution if you want the absolute best local fit.

Meng: If you’re dealing with highly skewed data, LoBoost might not be the first thing you reach for without some adjustment to how those scores are calculated <ref:2602.22432#pg1>.

Lalam: I'm confident that for standard tabular regression where things aren't too weirdly skewed, these guarantees hold up well, and it’s a solid way to get good uncertainty quantification right out of the model <ref:2602.22432#pg1>.

Tom: It really bridges that gap between the heavy machinery of conformal methods and just using a standard global residual score for boosted trees. So, LoBoost is a clean way to get local adaptation without the heavy overhead.

Jane: It’s definitely a method that’s going to be interesting for anyone working on high-dimensional tabular data where those feature interactions create localized noise patterns <ref:2602.22432#pg1>.

Lu: It shows that you don't always need to build a whole new model just to get local calibration; you can use the existing structure for that purpose.

Meng: For me, if we can get those speedups across the board, it’s worth looking at integrating this into our standard prediction pipeline for faster anomaly detection or quality control checks.

Lalam: I think what matters most is that they’ve shown how to reuse the internal leaf structure effectively, and that's a powerful idea for making uncertainty estimation more scalable.

Tom: Alright, so we've covered the mechanics, the theory, and where it falls short with skewness. LoBoost is a strong tool for model-native local prediction on boosted trees. Next time we’ll be looking at how other recent work tackles those challenging multimodal reasoning problems in large language models.

Federal University of São Carlos · University of São Paulo · Université Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK

stat.ML, cs.LG

Submitted: 2026-02-25

Updated: 2026-11-10

Comments: Accepted paper version

Journal ref: Transactions on Machine Learning Research (11/2026)

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 93/100

The gist: Gradient-boosted decision trees are among the strongest off-the-shelf predictors for tabular regression, but point predictions alone do not quantify uncertainty.

Key concepts

Model-Native Locality and Partitioning
LoBoost uses the sequential nature of gradient boosting to organize the feature space hierarchically. It defines local regions around a data point by checking which leaf nodes it falls into across different trees. This creates nested cells, where coarse partitions are defined by early trees and finer ones by later trees, allowing for localized uncertainty estimation.
Retraining-free Calibration
The method applies conformal prediction directly to the fitted ensemble without retraining the original gradient boosting model. It only requires the fitted leaf indices from the model and a standard calibration set. This avoids expensive auxiliary partition models or repeated training, making post-hoc uncertainty quantification much faster.
Theoretical Guarantees for Coverage
Mathematical proofs show that LoBoost's prediction intervals are reliable. As the number of calibration observations within a specific local region grows large enough, the actual coverage error converges to the desired level. This ensures that the method provides statistically sound uncertainty estimates under appropriate conditions.
Resolution Selection Mechanism
The algorithm intelligently chooses how finely to partition the space based on calibration data. It uses leaf paths from calibration covariates and a threshold to define local sizes. Regions are then refined or aggregated using a weighted Hamming distance, penalizing disagreements in trees that have more predictive power.

Terminology

Summary

Gradient-boosted decision trees are among the strongest off-the-shelf predictors for tabular regression, but point predictions alone do not quantify uncertainty. p. 0

Where the paper says it

Cabezas Department of Statistics, Federal University of São Carlos, São Carlos, São Paulo, Brazil Institute of Mathematics and Computer Science, University of São Paulo, São Carlos, São Paulo, Brazil Université Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK, France Rafael Izbicki Department of Statistics, Federal University of São Carlos, São Carlos, São Paulo, Brazil Thiago R. Ramos Department of Statistics, Federal University of São Carlos, São Carlos, São Paulo, Brazil Abstract Gradient-boosted decision trees are among the strongest off-the-shelf predictors for tabular regression, but point predictions alone do not quantify uncertainty.

LoBoost proposes a model-native local conformal method that reuses the fitted ensemble’s leaf structure to define a multiscale partition of the feature space. p. 0
Where the paper says it

Conformal prediction provides distribution-free marginal coverage, yet standard split conformal uses a single global residual quantile and can adapt poorly to heteroscedasticity. We propose LoBoost, a modelnative local conformal method that reuses the fitted ensemble’s leaf structure to define a multiscale partition of the feature space.

This approach allows for locally adaptive prediction intervals by estimating residual quantiles within these model-induced cells, requiring no auxiliary partition or retraining. p. 1
Where the paper says it

• Model-native local conformal prediction for boosted trees. We use leaf-index prefixes from the fitted boosted ensemble to define a multiscale partition of X and perform conformal calibration within these cells, yielding locally adaptive intervals.

Model-Native Locality and Partitioning

LoBoost leverages the sequential nature of gradient boosting to induce a hierarchical organization of the feature space, where early trees capture coarse structure and later trees introduce finer refinements. This process associates each input with a sequence of leaf identifiers, denoted as lT (x) = l1(x),. p. 5

Where the paper says it

Let ℓt(x) denote the leaf identifier reached by a point x ∈ X in the t-th regression tree ht, as defined in (1). The boosted ensemble therefore associates each input with a sequence of leaf identifiers, ℓT (x) = ℓ1(x), ℓ2(x), .

.., lT (x). A local region around x is defined by requiring agreement on the first k trees, i. p. 1
Where the paper says it

Concretely, we represent each input x by the sequence of leaves it visits along the ensemble, ℓT (x) = ℓ1(x), . , ℓT (x) , and define a local region around x by requiring agreement on the first k trees, i.

e., matching the prefix lk(x) = l1(x),. p. 1
Where the paper says it

, ℓT (x) , and define a local region around x by requiring agreement on the first k trees, i. , matching the prefix ℓk(x) = ℓ1(x), .

.., lk(x) for k ≤ T. This yields a nested, multiscale partition—coarse when k is small, increasingly refined as k grows—on which residual quantiles are estimated locally. p. 1
Where the paper says it

, ℓk(x) for k ≤ T. This yields a nested, multiscale partition—coarse when k is small, increasingly refined as k grows—on which we calibrate residual quantiles.

Theoretical Guarantees and Validity

The theory connects the stability of the fitted ensemble and the geometry of its induced cells to local residualscore homogeneity, providing finite-sample coverage-error control and asymptotic pointwise validity as the local calibration size grows. Proposition 1 shows that when z ∈ Rk(x) and the tail envelope bT,k is small, the nonconformity score s(z, y) = y − gT (z) is approximately equal to s(x, y); that is, the absolute-residual score remains approximately constant across the boosting-induced partitions. p. 6

Where the paper says it

Proposition 1 shows that when z ∈ Rk(x) and the tail envelope bT ,k is small, the nonconformity score s(z, y) = |y − gT (z)| is approximately equal to s(x, y); that is, the absolute-residual score remains approximately constant across the boosting-induced partitions. Due to this property, this is the score we consider throughout this work.

Theorem 1 establishes finite-sample group-conditional validity for every cell A ∈ Π with positive probability. p. 6
Where the paper says it

The prediction interval is Cb(x) = [gT (x) − qbA(x) , gT (x) + qbA(x) ]. Theorem 1 (Finite-sample group-conditional validity).

Asymptotic conditional coverage is achieved when the cell contains increasingly many calibration observations and its score distribution becomes homogeneous around x, specifically when ncal,npn(x) p → ∞ and ∆n(x) p → 0. p. 6
Where the paper says it

We next show that the cell-level guarantee becomes pointwise asymptotically when the cell contains increasingly many calibration observations and its score distribution becomes homogeneous around x. Consider a sequence of fitted predictors gn and corresponding final model-induced partitions Πn, each measurable with respect to the training sigma-field Ftr,n, together with independent calibration samples Dcal,n of size ncal,n.

Resolution Selection Mechanism

The selection of the resolution level k is managed by a data-efficient variant where the calibration sample is used both to select the resolution and to estimate local quantiles. The algorithm extracts leaf paths from calibration covariates, defines an effective local-size threshold Nmin, and refines groups based on these thresholds. p. 8

Where the paper says it

define the effective local-size threshold Nmin := maxn Npart, ⌈pminncal⌉ o , (2) which combines an absolute floor Npart with a proportional threshold pmin. Groups with fewer than Nmin calibration observations become terminal, while the remaining groups proceed to the next tree.

Terminal regions are aggregated using a weighted Hamming distance dWH(R, R′) that penalizes disagreements in trees with greater influence. p. 8
Where the paper says it

For two regions R and R′ , define the weighted Hamming distance dWH(R, R′ ) = X T t=1 wt I[ℓt(R) ̸= ℓt(R ′ )] , wt := ηBt. (3) The range Bt measures the maximum prediction variation produced by tree t, so this weighting assigns a larger penalty to disagreements in trees with greater influence on the fitted predictor.

Performance and Practical Trade-offs

Experiments demonstrate that LoBoost achieves competitive interval quality, low post-hoc calibration costs, and stable behavior across local calibration-size settings. While LoBoost does not uniformly match the best adaptive baseline on each dataset, it improves interval quality (SMIS) relative to standard split conformal (ICP) on most datasets. p. 1

Where the paper says it

We use leaf-index prefixes from the fitted boosted ensemble to define a multiscale partition of X and perform conformal calibration within these cells, yielding locally adaptive intervals. Relative to standard split conformal (ICP), the natural baseline that shares the same fitted predictor and calibration split, LoBoost improves interval quality (SMIS) on most datasets, though it does not uniformly match the best adaptive baseline on each dataset.

The method offers substantial computational gains, with some settings showing post-fit speedups up to 100× faster than heavier rules like MICP and WICP.

Conclusion and Limitations

LoBoost bridges the gap between computationally expensive auxiliary conformal methods and standard global split conformal for boosted tree ensembles by reusing the internal leaf structure already learned by the model. The method is most effective in high-dimensional, heterogeneous tabular settings where complex feature interactions create localized residual dispersion along dimensions that the ensemble naturally splits on. p. 0

Where the paper says it

We propose LoBoost, a modelnative local conformal method that reuses the fitted ensemble’s leaf structure to define a multiscale partition of the feature space. Each input is represented by the sequence of leaves it visits along the boosting path, and matching leaf prefixes define nested groups in which residual quantiles are estimated locally.

A methodological limitation is its reliance on absolute-residual nonconformity scores, which yield prediction intervals symmetric around point estimates and may be sub-optimal for skewed responses. Future research could extend LoBoost to incorporate directional or asymmetric quantile scores within leaf-prefix cells. p. 7
Where the paper says it

This is a fairly reasonable assumption, since it simply requires that, on average, enough calibration points fall within each cell for the local quantile estimates to be reliable. Note that the sequence of cells in Theorem 2 is generated by a sequence of prefix depths An(x) = Rn,kn(x)(x).

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Model-native local conformal prediction for boosted trees: The improved system can perform uncertainty quantification on gradient-boosted trees by reusing their fitted leaf structure to define a multiscale partition of the feature space, leading to locally adaptive intervals that handle heteroscedasticity better than standard split conformal methods.

  2. Retraining-free calibration: The system can apply conformal prediction to an already fitted boosted model using only its leaf indices and a standard calibration set, avoiding the need for auxiliary partition models or retraining of the base learner, which significantly lowers post-hoc computational cost.

  3. Theoretical guarantees for coverage: The system's performance is backed by theoretical results showing that under certain conditions (vanishing geometric localization and sufficient local calibration size), the realized pointwise coverage converges to the nominal target of 1 minus alpha as the number of calibration points grows.

Abstract

Gradient-boosted decision trees are among the strongest off-the-shelf predictors for tabular regression, but point predictions alone do not quantify uncertainty. Conformal prediction provides distribution-free marginal coverage, yet standard split conformal uses a single global residual quantile and can adapt poorly to heteroscedasticity. We propose LoBoost, a model-native local conformal method that reuses the fitted ensemble's leaf structure to define a multiscale partition of the feature space. Each input is represented by the sequence of leaves it visits along the boosting path, and matching leaf prefixes define nested groups in which residual quantiles are estimated locally. By reusing the predictive structure already learned by the model, LoBoost requires no auxiliary partition or nuisance model, no retraining, and no additional data split beyond standard conformal calibration. Our theory connects the stability of the fitted ensemble and the geometry of its induced cells to local residual-score homogeneity, providing finite-sample coverage-error control and asymptotic pointwise validity as the local calibration size grows and the cells become sufficiently homogeneous. Experiments show a transparent quality-cost trade-off: LoBoost improves over global split conformal on several heterogeneous datasets, but it is not uniformly superior to the adaptive baselines and is not uniformly Pareto-optimal.

Sources

Related papers