No prior over nuisance parameters is required for credible intervals and Bayes factors
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.
Tom: We're looking at the paper: "No prior over nuisance parameters is required for credible intervals and Bayes factors".
Jane: Here's the gist of the paper. Non-parametric finite-sample credible intervals with one-dimensional priors: a middle ground between Bayesian and frequentist intervals is presented to provide a natural middle ground between Bayesian and frequentist statistical intervals.
Tom: The Proposed Interval Definition A p% credible set for a parameter θ ∈ Θ should be such that, after observing the credible set (but without inspecting the data for ourselves), we should have at least a p% belief that θ lies within it.
Jane: Formally, this requires specifying a prior over the parameter of interest, b(θ) rather than a high-dimensional prior over the full distribution space.
Tom: The validity criterion is defined as b(θ ∈ s Sp = s) ≥ p.
Paper summary: Jane: So, summarizing what we've heard so far, the core thesis of "No prior over nuisance parameters is required for credible intervals and Bayes factors" is that you can define statistical intervals that sit right in the middle between frequentist and Bayesian statistics >
Tom: They claim this method allows you to assign a p percent belief after observing the interval itself, without necessarily having inspected all the raw data, which is different from standard frequentist methods >
Lu: The formal definition they use is a validity criterion where your prior belief about the parameter within that set given you observed that set has to be at least p percent >
Meng: But they also have this soft criterion, which is that your belief in the set before seeing the data, given you already have the interval, should be close to that same p percent belief >
Lalam: They show concrete implementations for two things: first estimating a cumulative distribution function or CDF by looking at how many samples fall below a certain value y >
Jane: And second, estimating the mean of a distribution with bounded support, like when you’re looking at data restricted to an interval from 'a' to 'b' >
Tom: The paper is focused on proving this concept works as a proof of principle, showing that this approach can yield valid intervals across those specific cases where they set up the construction >
Lu: It really shows that you don't need a massive prior over every single parameter in the whole system to get useful results for your target parameter >
Jane: So, what does this mean for people who only look at data and not read heavy math? It suggests we can get interval estimates that respect some Bayesian intuition without having to commit to a huge prior >
Tom: And they are showing that these intervals can be quite narrow in the finite sample case because of that prior information you do feed in >
Conclusion: Jane: So, wrapping up this discussion on "No prior over nuisance parameters is required for credible intervals and Bayes factors," the authors Tim Ritmeester and others are presenting a way to create statistical intervals that bridge the gap between Bayesian and frequentist approaches >
Tom: The key takeaway is that you can achieve a p percent belief in your parameter within an interval after observing the interval, without needing a prior distribution over every single thing in the entire system >
Lu: It’s about relaxing what we used to think was necessary for credible sets, moving away from needing a high dimensional prior on the full distribution space and instead focusing just on the parameter of interest >
Meng: From an engineering standpoint, this is interesting because it makes implementation much more practical; you only need to specify your belief about that specific parameter, b(theta), which is way less demanding than building a massive prior over every possible data configuration >
Lalam: The implication is that this could lead to better tools in many fields where we usually have to choose between a fully Bayesian approach or a purely frequentist one, and this paper proves it’s feasible for finite samples >
Jane: So, for someone just listening, this means there's a new way to think about interval estimation where you can use your knowledge about the parameter you care about without having to define the entire landscape of possibilities >
stat.ME, physics.data-an
Submitted: 2026-01-24
Updated: 2026-11-11
License: http://creativecommons.org/licenses/by-sa/4.0/
Importance score: 76/100
The gist: Non-parametric finite-sample credible intervals with one-dimensional priors: a middle ground between Bayesian and frequentist intervals is presented to provide a natural middle ground between…
Key concepts
- Credible Set Definition
- A p% credible set means that after observing the data, there is at least a p% probability that the parameter lies within it. This is achieved by specifying a prior distribution over the parameter of interest, $b( heta)$, instead of needing a prior over every possible outcome in the full data space.
- Construction Methodology
- The method constructs an interval by accessing data only through a low-dimensional random variable $M$ that depends on the observed data $X$. A specific construction ensures the resulting interval satisfies a mathematical condition related to the ratio of prior densities, making it valid.
- Finite-Sample Credibility
- Unlike some Bayesian methods, this approach provides credible intervals even with finite sample sizes. It is designed to be a middle ground, offering practical results that are not strictly asymptotic but possess credibility properties derived from the specified prior.
Terminology
Summary
Non-parametric finite-sample credible intervals with one-dimensional priors: a middle ground between Bayesian and frequentist intervals is presented to provide a natural middle ground between Bayesian and frequentist statistical intervals. p. 0 Non-parametric finite-sample credible intervals with one-dimensional priors: a middle
ground between Bayesian and frequentist intervals
Tim Ritmeester, PhD
(Dated: May 11, 2026)
We present a method of constructing statistical intervals that obtain a natural middle ground
between Bayesian and frequentist statistical intervals, previously unexplored in literature: To
a p% Bayesian credible interval we should assign a p% belief after observing both the dataset
and the interval, to p% frequentist intervals we can generally only assign a p% belief before observing either the data or the interval, while to the intervals proposed here we can assign
a p% belief after observing the interval, but not necessarily after inspecting the full dataset ourselves. Even in fully non-parametric problems this only requires a prior over the parameter(s) of interest,
not a high-dimensional prior over the full distribution, while maintaining many of the practical
and philosophical advantages of Bayesian methods.Where the paper says it
The Proposed Interval Definition
A p% credible set for a parameter θ ∈ Θ should be such that, after observing the credible set (but without inspecting the data for ourselves), we should have at least a p% belief that θ lies within it. Formally, this requires specifying a prior over the parameter of interest, b(θ) rather than a high-dimensional prior over the full distribution space. p. 0 We
show that this doesn’t require specifying a prior over
the full distribution space, only over the parameter of
interest. They share many of the practical advantages
of Bayesian methods while avoiding the complexity of
assigning high-dimensional priors altogether. Since the data
X are random, the computed credible set Sp is too. Conditional on Sp equaling some set s, we then say that Sp
is a p% credible set if it satisfies the following validity
criterion [7]:
Validity: b(θ ∈ s | Sp = s) ≥ p . Conditional on Sp equaling some set s, we then say that Sp
is a p% credible set if it satisfies the following validity
criterion [7]:
Validity: b(θ ∈ s | Sp = s) ≥ p . (1)
For this to be useful we would also like p to be close to
the belief that we would have assigned to the interval if
we had inspected the data ourselves, giving the following
soft criterion for any realization of Sp:
Precision: b(θ ∈ Sp|X) ≈ p , (2)
which, by the Bernstein-von Mises theorem, also
ensures good frequentist properties [3].Where the paper says it
Where the paper says it
Where the paper says it
Construction Methodology
The method constructs an interval Sp by giving the algorithm access to the data X only through some low-dimensional random variable M (dependent on X). Theorem 1 states that an interval Sp calculated as an intervalvalued function of M that always satisfies R Sp dθ L(θ)b(θ) / R Θ dθ L(θ)b(θ) is a valid credible interval in the sense of Eq. p. 1 A practical way to achieve this is to use the following:
Theorem 1. Given a random variable M (dependent
on the data X) and upper and lower bounds l±(θ|m) on
b(M = m|θ), an interval Sp calculated as an intervalvalued function of M that always satisfies
R
Sp
dθ L(θ)b(θ)
R
Θ
dθ L(θ)b(θ)
(3)
is a valid credible interval in the sense of Eq. IV) to provide bounds on
b(M = m|θ) that are valid for all prior beliefs on X. Thus to construct the credible intervals Sp we only need
access to the low-dimensional prior b(θ) rather than to
the full high-dimensional prior on the distribution of X.Where the paper says it
Where the paper says it
Concrete Implementations
The paper provides two concrete cases for implementation:
-
CDF: Estimating the fraction of a distribution that falls below a certain value y, i. p. 0
e., θ = P(X < y). For this case, m is chosen as the number of samples smaller than y, and l±(θm) is given by Eq. p. 2Where the paper says it
We show that this type of interval can be implemented in practice by giving concrete derivations for two fully non-parametric cases: estimating the fraction of a distribution that falls below a certain value (i. , the CDF at that value) and estimating the mean of a distribution with bounded support, and show the properties that the intervals have for these cases.
13.Where the paper says it
For estimation of the CDF, we choose m to be the number of samples that are smaller than y. Since m is known to be distributed as a binomial random variable with parameter θ, the belief b(m|θ) should be the same for each user, and thus l±(θ) is given by: l±(θ|m) = N m θ m(1 − θ) N−m , (13) and the resulting intervals satisfy the validity criterion of Eq.
-
Mean: Estimating the mean of a distribution with bounded support, i. p. 0
e., θ = R b a dx xP(X = x) for X restricted to [a, b]. p. 2Where the paper says it
We show that this type of interval can be implemented in practice by giving concrete derivations for two fully non-parametric cases: estimating the fraction of a distribution that falls below a certain value (i. , the CDF at that value) and estimating the mean of a distribution with bounded support, and show the properties that the intervals have for these cases.
For this case, m is chosen as the sample mean plus a random variable with distribution univ(−δ, δ), and bounds l±(θm) are generated using Hoeffding’s inequality (Eqs. p. 2Where the paper says it
• Mean: Estimating the mean of a distribution with bounded support, i. , θ = R b a dx xP(X = x) for X restricted to [a, b].
14-15).Where the paper says it
1 with equality. For estimation of the mean, we choose m to equal the sample mean plus a random variable with distribution univ(−δ, δ), where δ ≡ 0.
Comparison and Practicality
The proposed intervals exhibit properties lying between frequentist and Bayesian methods. They require a prior only for the parameter of interest, making them much more practical than fully Bayesian non-parametric approaches. p. 4 Fully Bayesian methods require, for the nonparametric problems described here, a full prior
over all possible probability distributions, making
them rather impractical. The intervals proposed
here require a prior only for the parameter of interest, making them much more practical. As is generally the case for Bayesian methods, for
small samples the intervals produced can be expected to be narrower than the frequentist equivalent. This is because the intervals use prior information. • Narrowness of interval for small/large samples By incorporating prior information, the proposed method and the fully Bayesian approach provide narrow small-sample intervals. Asymptotically, frequentist intervals and intervals of the types
proposed can either be equal in size to the Bayesian
ones or wider, depending on the method used. For the algorithms discussed
in Sec. IV, the second condition is only an issue for large
sample estimation of the mean, and in Sec.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Future Directions
The main direction for future work is to find other applications where such intervals can be constructed. One may engineer methods with different types of properties, such as finite sample bounds, by choosing the statistic m appropriately. p. 4 By choosing the statistic m (as in Sec. III) appropriately, one may engineer methods with different types
of properties (for a given model), something possible in
frequentist statistics but not in fully Bayesian methods. IV for estimation of
the mean may also certainly be improved; for example,
by choosing a different distribution of the noise Z,
or by using other inequalities that incorporate information about the variance of the distribution. This
may reduce a downside it has compared to asymptotic
methods by improving the precision in the sense of Eq. This corresponds to frequentist intervals where observing the interval does not actually change your belief
in the interval [4]. Normally derived for parametric cases,
this may be combined with the proposed methods to provide fully non-parametric intervals without specifying a
subjective prior. The type of statistical interval we have proposed
here, and the principle to derive such intervals, is rather
broad. This work was meant as a proof of principle,
demonstrating that this type of interval is feasible.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
TABLE I: Comparison between the proposed method to generate intervals and to standard Bayesian and frequentist methods. p. 5 6
Proposed method Frequentist Fully Bayesian Asymptotic Bayesian
Finite-sample credible ** * *** *
Asymptotically credible **/*** *** *** ***
Narrow small-sample intervals *** * *** */***
Narrow large-sample intervals * / *** */*** *** ***
Doesn’t require prior ** *** * ***
Flexibility ** * *** ***
TABLE I: Comparison between the proposed method to generate intervals and to standard Bayesian and frequentist
methods. methods (which, in turn, have the relationship to
fully non-parametric Bayesian intervals as discussed in
Sec.Where the paper says it
• Finite-sample/asymptotically credibility All of the methods are asymptotically credible, while the proposed method and the fully Bayesian analysis are also finite-sample credible. p. 4 • Finite-sample/asymptotically credibility
Does the interval actually tell us the degree of faith
we should have in the interval? All of the methods
are asymptotically credible, while the proposed
method and the fully Bayesian analysis are also
finite-sample credible.Where the paper says it
• Narrowness of interval for small/large samples By incorporating prior information, the proposed method and the fully Bayesian approach provide narrow small-sample intervals. p. 4 For the proposed method
this is only true if we have not inspected the data
ourselves. • Narrowness of interval for small/large samples By incorporating prior information, the proposed method and the fully Bayesian approach provide narrow small-sample intervals. • Narrowness of interval for small/large samples By incorporating prior information, the proposed method and the fully Bayesian approach provide narrow small-sample intervals. Asymptotically, frequentist intervals and intervals of the types
proposed can either be equal in size to the Bayesian
ones or wider, depending on the method used.Where the paper says it
Where the paper says it
• Requirement of a prior Frequentist and asymptotic Bayesian methods don’t require specifying a prior, making them objective and off-the-shelf. p. 4 Asymptotically, frequentist intervals and intervals of the types
proposed can either be equal in size to the Bayesian
ones or wider, depending on the method used. • Requirement of a prior Frequentist and asymptotic Bayesian methods don’t require specifying
a prior, making them objective and off-the-shelf.Where the paper says it
• Flexibility Frequentist intervals are rather rigid regarding, e. p. 4 The intervals proposed
here require a prior only for the parameter of interest, making them much more practical. • Flexibility Frequentist intervals are rather rigid
regarding, e. They are furthermore rather rigid in
their application (regarding, e. , sequential and post-hoc
analysis), leading to practical difficulties and common
misuse [1, 5].Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Construct statistical intervals that form a natural middle ground between Bayesian and frequentist intervals by assigning a p percent belief after observing both data and interval, relaxing frequentist rigidity while maintaining Bayesian credibility properties. This allows AI systems to perform decision-making under uncertainty with subjective confidence levels derived from observed results.
-
Implement the proposed method for estimating the fraction of a distribution that falls below a certain value (CDF) by setting m to be the number of samples smaller than y, where l±(thetam) is given by N/m(1-theta). This enables AI to estimate distribution functions with credible intervals that satisfy the validity criterion with equality.
-
Use the construction for estimating the mean of a distribution with bounded support by choosing m equal to the sample mean plus a random variable from univ(-delta, delta), and generating bounds l±(thetam) using Hoeffding's inequality bounds (Eqs. 14-15). This allows AI to estimate means of bounded distributions with intervals that satisfy the validity criterion, even if only asymptotically.
-
Utilize the flexibility of the method to explore different intervals or prior beliefs without affecting the core properties described in Eq. 1 and Eq. 2, which is useful for developing robust decision-making frameworks where prior knowledge over a full distribution space is impractical.
Abstract
This work shows that there are procedures that maintain the subjective Bayesian property of coherently informing decision making under uncertainty (i.e., if they tell you to take a certain bet, you should) without needing to assign a prior distribution over nuisance parameters, only over the parameter(s) of interest. We first carefully define this sense of 'credibility', and show that within this framework one can condition on any statistic of choice rather than only on the full dataset. This relaxation of credibility is a natural middle ground between Bayesian and frequentist statistics in the following sense: To a p% Bayesian credible set we should assign a p% belief after observing both the dataset and the interval, to p% frequentist sets we can generally only assign a p% belief before observing either the data or the interval, while to the statistical sets proposed here we can assign a p% belief after observing the set, but not necessarily after inspecting the full dataset ourselves. We then show that choosing a conditioning statistic whose distribution we know, or can bound, for each value of the parameter of interest leads to prior-free Bayes factors even for composite hypotheses, and to non-parametric credible sets with priors only over the parameter of interest. We derive such procedures for several (groups of) statistical problems. Particularly noteworthy is that any frequentist test gives a Bayes factor of the power over the size of the test, and that anytime-validity of E-process-based tests can be carried over to this Bayes factor.
Related papers
- Learning Robust Treatment Rules for Censored Data
- Neural Estimation of Summary Statistics for Simulation-Based Inference with Non-Asymptotic Guarantees
- Spectral divide-and-conquer MCMC for long stationary time series
- Improving the Efficiency of Subgroup Analysis in Randomized Controlled Trials with TMLE
- Personalized Federated Vector Autoregression with Personalization Diversity
- Doubly Robust and Efficient Calibration of Prediction Sets for Right-Censored Time-to-Event Outcomes