Location--Scale Calibration for Generalized Posteriors

arXiv:2511.15320 · stat.ME · Submitted 2025-11-19 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.

Tom: We're looking at the paper: "Location--Scale Calibration for Generalized Posteriors".

Jane: Here's the gist of the paper. General Bayesian updating replaces the likelihood with a loss scaled by a learning rate, but posterior uncertainty can depend sharply on that scale <ref:2511.15320#pg2>.

Tom: We propose a simple post-processing that aligns generalized posterior draws with their asymptotic target, yielding uncertainty quantification that is invariant to the learning rate <ref:2511.15320#pg5>.

Jane: The paper proves total-variation convergence for generalized posteriors with an effective sample size, allowing for sample-size-dependent priors, non-i.i.d.

Tom: observations, and convex penalties under model misspecification <ref:2511.15320#pg5>.

Title and authors: Tom: So, we're looking at this paper, "Location--Scale Calibration for Generalized Posteriors". It sounds like they’re dealing with a general Bayesian updating setup where the loss function is scaled by this learning rate, let's call it eta.

Jane: Right. The title suggests they are trying to fix that uncertainty issue where the posterior dispersion gets really sensitive to how big that learning rate eta is. Is that what it boils down to?

Tom: Exactly. The abstract says general Bayesian updating replaces the likelihood with a loss scaled by a learning rate, but posterior uncertainty can depend sharply on that scale, and they propose this post-processing to align generalized posterior draws with their asymptotic target, which keeps the uncertainty quantification invariant to the learning rate > Page one of THIS PAPER <ref:2511.15320#pg1,general Bayesian updating replaces the likelihood with a loss scaled by a>.

Lu: What's interesting is that they aren't just suggesting a fix; they are proving total-variation convergence for generalized posteriors with an effective sample size, which means it can handle things like sample-size dependent priors, non independent and identically distributed observations, and even convex penalties under model misspecification > Page five of THIS PAPER.

Meng: From an engineering standpoint, that sounds ambitious. It sounds like they're taking a complex setup and proving something fundamental about its convergence properties across different data regimes > Page five of THIS PAPER.

Lalam: I see the paper focuses heavily on defining target quantities like J star and V star target, which are related to the sandwich covariance for penalized M-estimators > Page three of THIS PAPER <ref:2511.15320#pg1>.

Tom: Right, so they define this matrix V star target based on something called J star and K star, and then they use that to construct a location--scale calibration map Omega that aligns the limiting distribution with this target law > Page five of THIS PAPER.

Jane: So, what does that alignment actually mean for us when we're just trying to get a reliable uncertainty estimate from our posterior draws?

Tom: It means that the asymptotic uncertainty quantification becomes invariant to the choice of learning rate eta, which is huge because it means we don't have to obsess over picking the perfect learning rate for our specific application > Page five of THIS PAPER.

Lu: And they prove that conditionally on the data, s n of theta calibrated minus theta tilde n converges in distribution to a normal distribution with covariance V star target as n goes to infinity > Page nine of THIS PAPER.

Meng: That convergence result is solid, but what about the practical side? If we have to implement this calibration, how much work are we talking about?

Title and authors: Jane: They present a fully implementable version that only requires posterior draws and empirical moment estimators, which means you don't need to solve the whole complex population equation just to get an estimate > Page five of THIS PAPER.

Tom: They estimate the working curvature from the posterior sample and plug in consistent estimators for the target curvature and variability to define an empirical operator Omega hat > Page five of THIS PAPER.

Lalam: And Proposition four point three shows that this estimated calibration yields calibrated draws whose limiting law is invariant to the learning rate eta > Page nine of THIS PAPER.

Jane: That’s a big win for usability if it works as well as they claim, especially when we're dealing with real-world data that isn't perfectly behaved > Page seven of THIS PAPER.

Tom: And the empirical validation supports this; using a random-intercept linear mixed model with Huber loss and Gaussian prior, they show the proposed calibration renders inference essentially invariant to the learning rate eta > Page seven of THIS PAPER.

Meng: Stable coverage and interval width over orders of magnitude in the learning rate sounds like exactly what we need for production deployment where we can't always fine-tune hyperparameters perfectly > Page seven of THIS PAPER.

Lu: They contrast this sharply with uncalibrated posteriors, which show marked undercoverage as eta increases, which highlights how much the calibration helps control that issue > Page seven of THIS PAPER.

Jane: So they’re showing that this location--scale calibration method successfully decouples uncertainty quantification from the choice of learning rate > Page five of THIS PAPER.

Tom: The final result confirms that the limiting calibrated law is learning-rate invariant because the limiting covariance V star target depends only on J star and K star and is independent of eta > Page nine of THIS PAPER.

Lu: This invariance holds for both smooth and non-smooth settings, which means it’s robust across different kinds of penalties we might use, which is a big piece of the puzzle > Page nine of THIS PAPER.

Meng: So as an engineer looking at deploying this, I see a solid theoretical foundation that suggests we can deploy these models with less hyperparameter tuning to get reliable uncertainty estimates > Page seven of THIS PAPER.

Lalam: The paper concludes by showing that the calibrated draws converge to a normal distribution with covariance V star target at any admissible center theta tilde n > Page nine of THIS PAPER.

Title and authors: Jane: So, in short, this work provides a way to get reliable uncertainty quantification for generalized posteriors that doesn't depend on the learning rate you choose > Page five of THIS PAPER.

Tom: And it shows we have a way to handle sample-size dependent priors and non iid observations with this total-variation convergence result > Page five of THIS PAPER.

Lu: It really expands the scope because it extends the open-faced sandwich adjustment into a location–scale calibration that works for any learning rate > Page five of THIS PAPER.

Meng: I'm just wondering what happens if we introduce some very complex, non-smooth penalties; does this framework still hold up under those conditions?

Jane: They did extend the analysis to non-smooth penalties by defining active set penalized curvature J star lambda AA and target covariance V star target AA restricted to the active set A > Page two of THIS PAPER <ref:2511.15320#pg1>.

Tom: And Lemma D.three shows that under assumptions D one and two, the scaled posterior components in the active coordinates converge to a normal distribution with covariance J star lambda AA minus K star AA J star lambda AA minus K star > Page four of THIS PAPER.

Lu: The proofs show that the asymptotic behavior on those active coordinates is equivalent to the smooth case with curvature matrix J star lambda AA and variability K star AA > Page four of THIS PAPER.

Meng: So even when things get messy with non-smooth penalties, the asymptotic math simplifies back down to a standard smooth case, which is good for practical implementation > Page four of THIS PAPER.

Jane: It’s a strong result because it shows that even when the theoretical setting gets complicated, the core uncertainty quantification remains stable across settings > Page four of THIS PAPER.

Tom: So to wrap up this paper, "Location--Scale Calibration for Generalized Posteriors" gives us a method to stabilize posterior uncertainty by aligning draws with their asymptotic target, making it invariant to the learning rate > Page five of THIS PAPER.

Lu: It’s a lot of machinery, but the result is that we get total-variation convergence for generalized posteriors with an effective sample size, allowing for sample-size dependent priors, non iid observations, and convex penalties under model misspecification > Page five of THIS PAPER.

Meng: The practical takeaway is that we have a calibration method where you only need posterior draws and empirical moment estimators to get calibrated draws whose limiting law is invariant to the learning rate > Page nine of THIS PAPER.

Jane: So, this paper really gives us a concrete way to control uncertainty quantification without having to manually tune the learning rate > Page five of THIS PAPER.

The paper's summary: Tom: So, this paper is all about taking these generalized Bayesian updates that use a scaling parameter called eta, which controls how much you weigh your loss function, and they figured out a way to fix the uncertainty problem that comes with it.

Jane: Right. Basically, when you change that learning rate eta, the resulting posterior uncertainty can jump around wildly. This paper introduces a simple post-processing step to align those posterior draws with their true mathematical target, which means the uncertainty estimate stays stable no matter what eta you pick.

Lu: What they are proving is something called total-variation convergence for these generalized posteriors when you account for the effective sample size, and that’s pretty powerful because it lets them handle a lot of messy things.

Meng: Messy how? Like if your prior depends on how many data points you have, or if your observations aren't independent and identically distributed. Can it actually handle those kinds of setups?

Lu: Yes, they prove that this convergence works for sample-size dependent priors and non independent and identically distributed observations, even when you use convex penalties under model misspecification.

Tom: That’s a lot to unpack. So the main result is that this location--scale calibration map aligns the limiting distribution with a target law, and that asymptotic uncertainty quantification doesn't depend on eta at all.

Jane: So, for someone just listening, what does that mean for their models? It means they don't have to spend hours tweaking that learning rate parameter just to get a reasonable measure of how sure the model is.

Lalam: They’re showing that if you use this calibration method, the limiting covariance—that V star target—only depends on the structural properties of your loss function and prior, not on eta.

Tom: Exactly. The paper proves that for both smooth and non-smooth penalties, the final calibrated law is learning rate invariant because that limiting covariance is independent of eta.

Meng: And practically speaking, they show you can just plug in some rough estimates for the target curvature and variability to build an empirical operator called Omega hat, which gives you those calibrated draws.

Jane: So the method is implementable with just posterior samples and some simple moment estimators from the data, rather than needing to solve that whole complex population equation every time.

Lu: They also show that when they test it on real-world data, like a linear mixed model with Huber loss, the calibrated posterior keeps stable coverage and interval width even when you change eta by orders of magnitude.

Tom: That’s the payoff. Uncalibrated posteriors get way worse as eta gets bigger, showing marked undercoverage, but this calibration method tracks frequentist benchmarks across those huge ranges.

Jane: It really decouples the uncertainty quantification from the choice of learning rate, which is a major hurdle in practical AI deployment where you often have to choose a learning rate for speed or stability.

Lu: The core idea they build on is extending an open-faced sandwich adjustment into this location--scale calibration, which works consistently for any learning rate.

Tom: It’s about taking that general framework and showing we can get reliable uncertainty quantification without manually tuning the scaling parameter.

Jane: This sets up a really interesting direction because it moves us away from just picking a good learning rate toward having a method that makes your uncertainty estimate robust regardless of what you choose.

Lalam: It confirms that even when things get messy with non-smooth penalties, the asymptotic behavior simplifies back to the smooth case, which is good for practical implementation.

Tom: So we’ve seen how they handle the theory and how they test it on real models, suggesting this calibration is a solid way forward for deploying these complex Bayesian models reliably.

The paper's improvements: Tom: So, we’re looking at how these authors are taking the original location--scale calibration idea and making it better, because they’re not just doing a quick fix; they’re extending it significantly.

Jane: They build on their initial work to show that this approach works even when things get really messy with non-smooth penalties, which is something we talked about earlier.

Lu: Yes, they define new target quantities for those non-smooth cases, like the active set penalized curvature and the restricted target covariance, and they prove convergence still happens.

Meng: So if we’re working on production models where things are complicated—maybe using certain types of regularization that aren't perfectly smooth—this means we don't have to stop trying to use Bayesian methods.

Tom: Right. They show that the asymptotic behavior on those complicated active coordinates still matches the standard smooth case, which simplifies things for how we think about the math behind it.

Jane: That’s a relief because it means their uncertainty quantification stays stable even when you’re using more complex loss functions or penalty terms than just standard squared errors.

Lalam: They prove that those scaled posterior components in the active coordinates converge to a normal distribution with a specific covariance defined by the active set curvature and variability.

Tom: That's a precise statement, and it shows that even in these complicated scenarios, we still have a clear mathematical target for what the posterior should look like.

Jane: It means they can use this framework to get reliable uncertainty estimates without having to manually simplify their model structure just because of the penalty terms.

Lu: The implication is that you gain robustness across different kinds of penalties, which is crucial when you’re not one hundred percent sure about the exact mathematical formulation you’re using for your loss function.

Meng: For me, that translates to fewer headaches during deployment because I don't have to worry as much about whether my specific penalty term is "too smooth" or "too rough."

Tom: Exactly. They are taking a general theory and hardening it up so it’s more robust against real-world model choices.

Jane: And they still keep that core promise: the uncertainty quantification remains invariant to the learning rate eta, which is still huge for practical use.

Lalam: The final result confirms that this extended framework works well for both smooth and non-smooth settings, validating their approach across a wider range of conditions.

Tom: So what we’re seeing here is a method that doesn't just work in the ideal case, but one that survives when you introduce real-world complexity into your loss functions.

Conclusion: Tom: So, we’re wrapping up on "Location--Scale Calibration for Generalized Posteriors," which is basically showing how you can fix that nasty dependency on your learning rate eta when using generalized Bayesian updating.

Jane: It boils down to this: they’ve developed a location--scale calibration map that aligns the posterior draws with their true mathematical target, making the uncertainty estimate stable regardless of what eta you choose.

Lu: They prove total-variation convergence for these generalized posteriors under certain conditions, which is pretty strong because it lets them handle complex setups like non independent and identically distributed observations.

Meng: For me, that means when I’m deploying models to production where the data isn't perfect, I can trust the uncertainty metrics more without having to spend all day tuning hyperparameters.

Tom: Exactly. They show you how to get stable uncertainty quantification without obsessing over picking the perfect learning rate for your specific application.

Jane: It really decouples that crucial part of model deployment from a hyperparameter you might need to adjust constantly, which is a big win for real-world systems.

Lalam: They confirm that the limiting covariance only depends on the loss function structure and prior, not on eta, which gives us confidence in the final result.

Meng: And they showed this holds even when we use more complicated regularization penalties that aren't perfectly smooth, so it’s more robust for production environments.

Tom: So, "Location--Scale Calibration for Generalized Posteriors" is essentially a way to get reliable uncertainty quantification that doesn't care about the learning rate you pick.

Jane: It provides a solid method for controlling those uncertainty estimates without needing constant manual tuning of the scaling parameter eta.

Lu: This work opens up possibilities for using these generalized posteriors in more complex modeling scenarios where we typically struggle with sensitivity to learning rate changes.

Tom: We’ll be looking at how this calibration can interface with other methods, especially when we start talking about those sequence-derived feature distributions next.

Department of Multidisciplinary Sciences, Graduate School of Arts and Sciences, The University of Tokyo · Department of Epidemiology, National Institute of Infectious Diseases, Japan

stat.ME

Submitted: 2025-11-19

Updated: 2026-11-11

Code: https://github.com/shutech2001/ls-calib-gp

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 91/100

The gist: General Bayesian updating replaces the likelihood with a loss scaled by a learning rate, but posterior uncertainty can depend sharply on that scale <ref:2511.15320#pg2>.

Key concepts

Learning Rate ($\eta$)
This is a single scaling parameter in the Bayesian update that multiplies the loss function. It directly controls how much posterior uncertainty is amplified or suppressed. The paper shows that standard methods are sensitive to this value, but their proposed calibration method makes the final result independent of $\eta$.
Location-Scale Calibration
This is a post-processing technique designed to correct generalized posterior draws. It works by aligning the current posterior distribution with its theoretical asymptotic target. This alignment ensures that uncertainty quantification remains stable and does not depend on the specific learning rate chosen during sampling.
Total-Variation Convergence
This is a mathematical proof showing that as the number of data points ($n$) grows, the generalized posterior draws converge to a specific limiting distribution in terms of total variation distance. This convergence holds even when using complex setups like sample-size-dependent priors or non-i.i.d. observations.
Asymptotic Target ($V^{\star}_{target}$)
This is the theoretical, ideal distribution that the generalized posterior is expected to approach as the sample size becomes very large ($n \to \infty$). The calibration method aims to map the actual posterior draws onto this target distribution, which is mathematically proven to be independent of the learning rate $\eta$.

Terminology

Summary

General Bayesian updating replaces the likelihood with a loss scaled by a learning rate, but posterior uncertainty can depend sharply on that scale <ref:2511. p. 015320#pg2>. We propose a simple post-processing that aligns generalized posterior draws with their asymptotic target, yielding uncertainty quantification that is invariant to the learning rate <ref:2511. p. 015320#pg5>. The paper proves total-variation convergence for generalized posteriors with an effective sample size, allowing for sample-size-dependent priors, non-i. p. 0i.d. observations, and convex penalties under model misspecification <ref:2511. p. 015320#pg5>. Within this framework, the authors justify and extend the open-faced sandwich adjustment into a location–scale calibration whose draws converge in total variation to the target for any learning rate <ref:2511. p. 015320#pg5>.

Problem Formulation

The setup defines a general Bayesian updating framework where a single scaling parameter η, called the learning rate or temperature, multiplies the loss and directly controls posterior dispersion <ref:2511.15320#pg3>. The generalized posterior is given by an expression involving the composite loss Ln(θ) and a baseline prior π˜n(θ) <ref:2511. p. 115320#pg4>. The authors introduce the penalized population equation (1), which involves the empirical criterion Mn(θ), a convex penalty ρ, and defines target quantities like J⋆, K⋆, and V⋆target <ref:2511. p. 115320#pg4>.

Theoretical Guarantees

The main theoretical results establish that the generalized posterior admits a normal limit in total variation under Assumptions 2.1–2.4 <ref:2511. p. 015320#pg10>. The authors construct a location–scale calibration map omega that aligns the limiting distribution with the target law, showing that this asymptotic uncertainty quantification is invariant to the choice of learning rate <ref:2511. p. 115320#pg5>. Theorem 3.2 proves that conditionally on the data, s n(θ(d)calib − θ˜ n) converges in distribution to N(0, V⋆target) as n → ∞ <ref:2511. p. 315320#pg9>.

Practical Calibration Method

The paper presents a fully implementable version of the location–scale calibration that requires only posterior draws and empirical moment estimators <ref:2511.15320#pg5>. This procedure involves estimating the working curvature from the posterior sample and plugging in consistent estimators for the target curvature and variability to define an empirical operator omegaˆ <ref:2511. p. 315320#pg5>. Proposition 4. p. 33 demonstrates that this estimated calibration yields calibrated draws whose limiting law is invariant to the learning rate η <ref:2511. p. 015320#pg9>.

Empirical Validation

The empirical illustration, using a random-intercept linear mixed model with Huber loss and Gaussian prior, shows that the proposed location–scale calibration renders inference essentially invariant to the learning rate η <ref:2511.15320#pg7>. The calibrated posterior maintains stable coverage and interval width over orders of magnitude in the learning rate, whereas uncalibrated posteriors exhibit substantial sensitivity <ref:2511. p. 015320#pg7>. This contrasts sharply with the uncalibrated posterior, which shows marked undercoverage as η increases <ref:2511. p. 515320#pg7>.

Non-smooth Extensions

The analysis extends to non-smooth penalties by defining active set penalized curvature J⋆ λ,AA and target covariance V⋆target AA restricted to the active set A <ref:2511.15320#pg2>. Lemma D.3 shows that under Assumptions D. p. 11 and 2.2, the scaled posterior components in the active coordinates converge to N(0, J⋆ λ,AA−1K⋆ AAJ⋆ λ,AA−1) as n → ∞ <ref:2511. p. 915320#pg4>. The proofs show that the asymptotic behavior on the active coordinates is equivalent to the smooth case with curvature matrix J⋆ λ,AA and variability K⋆ AA <ref:2511.15320#pg4>.

Implementation Details

The implementation details describe an augmentation and Gibbs sampler where latent scales ωij arise from a generalized inverse Gaussian distribution GIG(λ = 1/2, χ = t 2ij, ψ = (ηc) 2) <ref:2511.15320#pg9>. The full conditional of β is multivariate normal with precision and mean Λpost defined by the learning rate η and the penalty terms <ref:2511. p. 115320#pg9>. This structure allows for a single iteration of the Gibbs sampler consisting of three updates: ω → t → β <ref:2511.15320#pg9>. The simulation settings fix G = 100, ni = 5, and p = 1 to focus on the slope β <ref:2511. p. 515320#pg9>.

Conclusion on Invariance

The final result confirms that the limiting calibrated law is learning-rate invariant because the limiting covariance V⋆target depends only on (J⋆ λ, K⋆) and is independent of η <ref:2511.15320#pg9>. This invariance holds for both smooth and non-smooth settings, validating the proposed location–scale calibration method <ref:2511. p. 115320#pg9>. The empirical results confirm that the calibrated posterior tracks frequentist benchmarks across several orders of magnitude in the learning rate <ref:2511. p. 115320#pg7>. The method successfully decouples uncertainty quantification from the choice of learning rate <ref:2511. p. 115320#pg5>. The paper concludes by showing that the calibrated draws converge to N(0, V⋆target) at any admissible center θ˜ n <ref:2511. p. 015320#pg9>.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Location-scale calibration for generalized posteriors allows uncertainty quantification to be invariant to the learning rate by aligning generalized posterior draws with their asymptotic target, yielding uncertainty quantification that is invariant to the learning rate.

  2. The proposed method can handle sample-size-dependent priors, non-i.i.d. observations, and convex penalties under model misspecification through a framework proving total-variation convergence for generalized posteriors with an effective sample size.

  3. The plug-in calibration procedure provides a fully implementable version requiring only posterior draws and empirical moment estimators to estimate the working curvature from the posterior sample and plug in consistent estimators for the target curvature and variability.

  4. The calibrated draws maintain stable coverage, interval width, and bias over orders of magnitude in the learning rate compared to uncalibrated posteriors, which vary markedly.

Related papers