CLAPS: Aleatoric-Epistemic Scaling via Last-Layer Laplace for Conformal Regression

summary

Video file (mp4)

The gist

The gist: CLAPS is a split conformal regression method that uses heteroscedastic last-layer Laplace uncertainty as an adaptive local scale to combine learned input-dependent noise with last-layer

In short

CLAPS is a split conformal regression method that uses heteroscedastic last-layer Laplace uncertainty to create an adaptive local scale for prediction intervals. It combines learned input noise with epistemic uncertainty, preserving standard conformal calibration while making interval widths sensitive to both data noise and feature support.

Key concepts

Heteroscedastic Likelihood
This is a statistical model where the observation noise variance changes depending on the input data point (x). In CLAPS, this captures how much inherent observation noise varies across different inputs, allowing the method to adapt its scaling based on local data quality.
Last-Layer Posterior Uncertainty
This refers to the uncertainty derived from approximating the final layer weights of a neural network using a Laplace approximation. It quantifies how much the model's learned representation itself is uncertain for a given input, reflecting lack of training data support in that region.
Split Conformal Calibration
This is the core validity mechanism ensuring that prediction intervals maintain coverage guarantees (e.g., 95% confidence) even when using machine learning models. It separates the fixed calibration step from the local adaptation step, ensuring reliability.
Predictive Scale v(x)
This is a combined scale used to define the width of the prediction interval. It merges two sources of uncertainty: h2(x), which represents input-dependent observation noise (aleatoric), and phi(x)⊤Σϕ(x), which represents epistemic uncertainty related to feature support.

Terminology used across episodes

This episode discusses

The paper

CLAPS: Aleatoric-Epistemic Scaling via Last-Layer Laplace for Conformal Regression · Read on arXiv

Dongseok Kim, Hyoungsun Choi, Mohamed Jismy Aashik Rasool, Gisung Oh

Gachon University

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "CLAPS: Aleatoric-Epistemic Scaling via Last-Layer Laplace for Conformal Regression".

Jane: Here's the gist of the paper. The gist: CLAPS is a split conformal regression method that uses heteroscedastic last-layer Laplace uncertainty as an adaptive local scale to combine learned input-dependent noise with last-layer epistemic uncertainty while retaining standard conformal calibration validity.

Tom: How it works CLAPS proposes a "split conformal regression method that uses heteroscedastic last-layer Laplace uncertainty as the local normalization scale" (Page 1).

Jane: This method combines a "learned input-dependent noise estimate with a last-layer posterior uncertainty estimate, and then uses the resulting scale inside a standard split conformal procedure" (Page 2).

Tom: The central idea is to "separate the source of validity from the source of local adaptivity" (Page 2).

Title and authors: Tom: Now we’re moving into the title and authors of the paper, "CLAPS: Aleatoric-Epistemic Scaling via Last-Layer Laplace for Conformal Regression." I think it’s a descriptive title, but let's unpack what that means for us.

Jane: It’s not just about scaling; it’s specifically tying the scaling mechanism to the last layer of the network and using Laplace uncertainty from that layer.

Lu: The authors are Dongseok Kim, Hyoungsun Choi, Mohamed Jismy Aashik Rasool, and Gisung Oh. They come from Computer Engineering at Gachon University.

Meng: That’s a small team of researchers who seem focused on the mechanics of how these models actually function under uncertainty.

Tom: What this title suggests is that they are making the specific choice about *where* to look for uncertainty information—right at the final layer weights.

Jane: It tells us that they aren't just plugging in some general noise estimate; they are using a specific, learned distribution from the model’s own final parameters.

Lu: This is interesting because it grounds the adaptive scaling in something derived directly from the trained model rather than an external heuristic.

Meng: So, they’re saying that by looking at those final weights through a Laplace approximation, you get a more informed local scale.

Tom: It sounds like they are building a bridge between deep learning uncertainty and rigorous statistical calibration methods.

Jane: It shows the authors are interested in how model representation affects the uncertainty landscape across different inputs.

Lu: And that connects back to their point about separating validity from local adaptivity, which is a fundamental design goal for this paper.

The paper's summary: Tom: So, let’s look at what the actual summary of "CLAPS: Aleatoric-Epistemic Scaling via Last-Layer Laplace for Conformal Regression" tells us about the mechanics.

Jane: Essentially, they introduce a split conformal regression method that uses heteroscedastic last-layer Laplace uncertainty as its local normalization scale.

Lu: They combine a learned input-dependent noise estimate with last-layer posterior uncertainty and then use that resulting scale inside a standard split conformal procedure.

Meng: So they are blending the learned noise estimate with the model’s internal posterior variance to create one unified scaling factor for the prediction interval.

Tom: The main idea is to preserve the validity mechanism of conformal calibration while giving that interval scale a more structured uncertainty interpretation than just aleatoric scaling.

Jane: They do this by characterizing this aleatoric–epistemic scale and deriving its heteroscedastic last-layer precision, which leads to a specific predictive scale formula.

Lu: That predictive scale v(x) equals h2(x) plus phi of x transpose sigma phi of x, which combines the heteroscedastic aleatoric variance with the last-layer epistemic variance.

Meng: So they are explicitly defining how this combined term adapts to both observation noise and training data support in a way that’s tied to the input features x.

Tom: This is where they claim they are adapting both to the estimated variability of the response and to the degree of support provided by those weighted training features.

Jane: It seems like they’re building a scale that directly reflects how hard it is for the model to predict a specific output given a specific input.

The paper's improvements: Tom: Okay, let’s talk about what they claim are the key improvements this method offers over previous work.

Jane: One big improvement is the ability to distinguish between uncertainty driven by intrinsic observation noise, which is aleatoric, and uncertainty caused by limited training support, which is epistemic.

Lu: This allows them to characterize how learned noise variance affects both the test-time aleatoric scale and the posterior geometry of epistemic uncertainty.

Meng: It gives a more structured interpretation of why an interval widens in noisy regions versus sparse regions, which is a diagnostic insight into model behavior.

Tom: And they showed that when training set size increases, the epistemic fraction decreases and the scale ratio moves toward one, showing a graceful reduction to aleatoric scaling when epistemic uncertainty contracts.

Jane: That means as you get more data, the system naturally focuses on just what we observe in the data without being overly conservative due to lack of evidence.

Lu: They also achieved better interval efficiency in controlled synthetic studies by giving the smallest region coverage gap, average width, and interval score compared to baseline methods like Split CP or CV+.

Meng: So if this is true on real data too, that means we get tighter intervals with better overall scores than existing popular methods.

Tom: And they found that on real-data benchmarks, CLAPS attained empirical coverage close to the nominal level while reducing average width and interval score relative to Split CP and CV+.

Jane: So it seems they’re achieving a good balance between being valid and being efficient in terms of interval size.

Conclusion: Tom: Alright, we've covered the main points of "CLAPS: Aleatoric-Epistemic Scaling via Last-Layer Laplace for Conformal Regression." To wrap up, this method successfully extends locally adaptive conformal prediction from noise-adaptive scaling to aleatoric–epistemic scaling.

Jane: This is significant because they’ve combined both sources of uncertainty into a single predictive scale, which offers an operational decomposition where wide intervals might arise from high estimated observation noise, weak training support, or both.

Lu: The implication is that last-layer Bayesian uncertainty can serve as a useful local scaling mechanism for conformal regression without replacing conformal calibration as the source of validity.

Meng: It’s a tool to help us diagnose the model's behavior by seeing if the interval widens because of noise, sparsity, or both.

Tom: So we have CLAPS here, which is a split conformal regression method that uses heteroscedastic last-layer Laplace uncertainty as an adaptive local scale.

Jane: It’s a way to get more nuanced control over prediction intervals in high-dimensional settings by incorporating the model's internal posterior information directly into the scaling logic.

More episodes

← Home