Prior Probabilities for Constrained Hypotheses: Invariance, Neutrality, and Model Search
summary
The gist
When testing hypotheses with equality and inequality constraints using Bayes factors, the resulting posterior probabilities depend on how the analyst divides the parameter space into hypotheses The…
In short
The paper introduces a complexity-induced prior to test hypotheses with equality and inequality constraints using Bayes factors. This prior ensures posterior probabilities are invariant to how the parameter space is divided into hypotheses, providing a neutral framework for analysts who want to avoid pre-specifying constraint plausibility.
Key concepts
- Complexity-Induced Prior
- A two-layer prior derived from a single parameter distribution. The first layer assigns probabilities to equality subspaces based on variable selection priors, and the second layer assigns probabilities to inequality regions based on their relative complexity under the encompassing prior.
- Bayes Factor
- A metric quantifying the evidence in data by balancing how well different hypotheses fit the observed data against their respective levels of complexity. It shows how much more likely one hypothesis is compared to another.
- Invariance Under Refinement
- This property means that if a hypothesis is nested within a larger set of hypotheses, its posterior probability remains unchanged after the alternative space is refined by observing the data. This ensures consistency across different ways of dividing the hypotheses.
- Coherence
- Under this prior, a hypothesis cannot gain more posterior probability than any hypothesis that contains it. This prevents logical fallacies and ensures that testing constrained hypotheses leads to coherent results, avoiding issues like the conjunction fallacy.
Terminology used across episodes
This episode discusses
- Prior Probabilities for Constrained Hypotheses: Invariance, Neutrality, and Model Search · Paper Radio
The paper
Prior Probabilities for Constrained Hypotheses: Invariance, Neutrality, and Model Search · Read on arXiv
Joris Mulder
Tilburg University
When testing hypotheses with equality and inequality constraints using Bayes factors, the hypotheses are commonly assigned equal prior probabilities. The resulting posterior probabilities depend on how the analyst divides the parameter space into hypotheses: splitting one hypothesis into two changes the posterior probabilities of the other hypotheses, and a hypothesis can receive more posterior probability than a hypothesis that contains it. An alternative is considered where the prior probabilities of the hypotheses follow from a single prior distribution on the parameters. Prior probabilities of the equality constrained subspaces are taken from the Bayesian variable selection literature, and prior probabilities of the inequality constrained regions within these subspaces are given by their relative complexities under the encompassing prior. Under this complexity-induced prior, posterior probabilities do not depend on how the hypotheses are divided, are coherent across nested hypotheses, and agree with those from an exploratory model search over the same subspaces. In return, inequality constraints do not increase the posterior probability of a hypothesis relative to the null hypothesis. The prior is therefore suited for analysts who want to test constrained hypotheses without favoring specific constraints a priori, while the potential gain in evidence for the constraints remains quantified by the Bayes factor.
Transcript
Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.
Tom: We're looking at the paper: "Prior Probabilities for Constrained Hypotheses".
Jane: Here's the gist of the paper. When testing hypotheses with equality and inequality constraints using Bayes factors, the resulting posterior probabilities depend on how the analyst divides the parameter space into hypotheses The complexity-induced prior is intended for analysts who want to be neutral about the constraints a priori Pruning redundant reasoning tokens The paper introduces a prior distribution derived from a single prior distribution on the parameters, consisting of two layers: one layer assigns prior probabilities to the equality constrained subspaces, following priors used in Bayesian variable selection The second layer assigns prior probabilities to the inequality constrained regions within these subspaces, given by their relative complexities under the encompassing prior This complexity-induced prior ensures that posterior probabilities do not depend on how the hypotheses are divided, are coherent across nested hypotheses, and agree with those from an exploratory model search over the same subspaces How the training signal is built The analyst's hypothesis space H is defined as a set of T competing hypotheses of the form Ht: R E t θ = r E t & R I t θ > r I t, (1) or unions of such hypotheses Equality constraints define a linear subspace, referred to as its equality structure and indexed by s, Ms: R E s θ = r E s, (2) with subspace Θs and free parameters θs The prior probability of an equality structure Ms is determined by the number of removed parameters rs and the number of equality structures in S that remove r parameters Where the gains come from The Bayes factor quantifies the evidence in the data, balancing relative fit against relative complexity Inequality constraints do not increase the posterior probability of a hypothesis relative to the null hypothesis, while their support remains visible in the Bayes factor The resulting posterior probabilities under this prior are invariant to refinement and coherent across nested hypotheses Implication I: Invariance Under Refinement By Proposition 1(i), a hypothesis that is contained in both H and H∗ has the same posterior probability This means that the alternative may be divided into inequality constrained hypotheses after observing the data without affecting the posterior probability of the null hypothesis Implication II: Coherence, and What Directional Testing Does (Not) Buy Under the complexity-induced prior, a hypothesis cannot receive more posterior probability than a hypothesis that contains it This coherence avoids the conjunction fallacy because inequality constraints do not increase the posterior probability of a hypothesis relative to the null hypothesis Implication III: Confirmatory Testing as a Coarsening of Exploratory Search The confirmatory test and the exploratory search thus give consistent results Every hypothesis in H is a union of models in this space, so that by Proposition 1(ii) its posterior probability under the complexity-induced prior equals the sum of the posterior probabilities of these models Recommendations for Practice The complexity-induced prior is useful when the analyst has formulated a set of hypotheses with equality and inequality constraints based on substantive or theoretical arguments, but does not want to add additional prior beliefs about the hypotheses It is recommended to report these as well because they depend on the equality constraints that are deemed plausible a priori The uniform prior would not be the neutral choice because it assigns the same prior probability to hypotheses that differ in size under the prior on the parameters The posterior probabilities should be reported together with the prior probabilities and the Bayes factors The Bayes factors show the gain in evidence from constrained hypotheses, which would not be visible in the posterior probabilities when using the complexity-induced prior Discussion The complexity-induced prior is intended for analysts who want to test constrained hypotheses using the Bayes factor while being neutral about the plausibility of the constraints a priori The uniform prior would not be the neutral choice because it assigns the same prior probability to hypotheses that differ in size under the prior on the parameters, which corresponds to a prior belief in favor of the more specific hypotheses An analyst who wishes to translate their prior beliefs about the hypotheses, e.g., about the direction of one or more effects, can specify the prior probabilities accordingly The presented results also apply to testing problems beyond regression models Transparency and Openness This study was not preregistered No new data were collected for this study Claude (Anthropic, 2026) was used to assist with editing parts of the text The author takes full responsibility Acknowledgements The author was supported by an ERC Consolidator Grant (101087383) The paper is organized as follows First the formulation of the analyst’s hypotheses is given, the parameter priors, and a running example is introduced Then the complexityinduced prior is introduced Next, the posterior probabilities, an empirical illustration, and three implications are discussed The paper ends with recommendations for practice and a discussion Proofs Proof of Lemma 1 uses (6), ms,j (Y) = m(Y Ms)fs,j (Y)/cs,j, so that mt(Y) = X (s,j)∈Jt P(Ms)cs,j Πt ms,j (Y) = m(Y) Πt(Y), with m(Y) = P s∈S P(Ms)m(Y Ms).
Tom: Bayes theorem with prior probabilities proportional to Πt gives (12), and when the hypotheses are exhaustive, the relative fits of the included regions sum to one within each equality structure, so that P P t Πt(Y) = s P(MsY) = 1 Proof of Proposition 1 shows that because relative fits are additive over disjoint regions, the denominator equals P s∈S P(MsY) Pr(θ ∈ Us Ms, Y), with Us the union of the regions under Ms that are included in the hypotheses, which is the same for both hypothesis spaces B R Code for the Empirical Illustration The code demonstrates how to compute posterior probabilities under both priors using BFpack and BayesFactor The results compare the posterior probabilities under the complexity-induced prior and the uniform prior across two applications Discussion Equal prior probabilities for the hypotheses in the analyst’s hypothesis space make the posterior probabilities depend on how the parameter space is divided The complexityinduced prior instead derives the prior probabilities of the hypotheses from a single prior distribution on the parameters, with prior probabilities of the equality structures from the variable selection literature and relative complexities for the inequality constrained regions The resulting posterior probabilities are invariant to refinement, coherent across nested hypotheses, and consistent with exploratory model search The Bayes factor of the union of the alternatives against the null hypothesis does not depend on the division, so when it is an e-variable, as for a simple null hypothesis, it remains one after refinement Table 1: Posterior Probabilities Under the Complexity-Induced Prior and the Uniform Prior for the Two Applications This table compares posterior probabilities across hypothesis spaces using both methods The results show that under the complexity-induced prior, it mostly goes to the complement, which has a larger prior probability than the order hypothesis Empirical Illustration For both applications, the posterior probabilities under the complexity-induced prior and the uniform prior are compared The priors differ in how the remaining posterior probability is divided Under the complexity-induced prior it mostly goes to the complement, which has a larger prior probability than the order hypothesis (.724 against.
Paper summary: Tom: Hey Jane, so we're looking at this paper "Prior Probabilities for Constrained Hypotheses: Invariance, Neutrality, and Model Search." The main idea here is that when you test different hypotheses using Bayes factors—those numbers that show how much better one model fits the data than another—the resulting posterior probabilities change depending on how you split up the parameter space into those hypotheses.
Jane: Right, so it claims a way to fix that dependency. It says they want a prior setup where the posterior probabilities don't depend on whether you divide your hypotheses in one way or another. It introduces this "complexity-induced prior" which is built from just one initial distribution over all the parameters instead of assigning equal starting probabilities to everything.
Lu: So, if we have these constraints—equality and inequality—the paper suggests that the equality constraints come from Bayesian variable selection literature, and for the inequality regions, they use their relative complexities under that main prior distribution. It sets up a two-layer prior structure <ref:2611.13781#pg3>.
Meng: So what does this mean for us practically? If we're running an AI model, and we've set up hypotheses with these constraints, this method helps us get consistent results even if we are just exploring different ways to group those hypotheses. It’s about being neutral about which constraints you pick initially <ref:2611.13781#pg2>.
Lalam: I can see how that works from the literature perspective. The paper notes that when you have only inequality constraints, the resulting posterior probabilities are linked to a specific prior distribution on the parameter space called Stern's prior <ref:2611.13781#pg3>. So this complexity-induced prior essentially dictates how we weigh those inequality regions based on how complex they are under the main parameter distribution.
Tom: That’s interesting because it means that if I split my model into two different sets of hypotheses, the posterior probability for one of those sets won't suddenly jump just because I moved some boundaries around <ref:2611.13781#pg2>. It keeps things coherent across nested hypotheses.
Jane: And it’s coherent, which is important because it avoids what they call the conjunction fallacy, where you think one hypothesis is more likely than another just because you made it bigger in your test setup <ref:2611.13781#pg2>. It means inequality constraints don't artificially inflate a hypothesis's posterior probability relative to the null hypothesis.
Paper summary: Lu: Exactly. And look at implication two, which is that under this complexity-induced prior, a hypothesis cannot get more posterior probability than one that it contains <ref:2611.13781#pg2>. It keeps the hierarchy of hypotheses intact based on complexity rather than arbitrary splitting.
Meng: So from a deployment standpoint, if we use this for model search, the results should be more stable across different search strategies. We don't have to worry that a small change in how we define our test space completely flips the confidence in our final hypothesis <ref:2611.13781#pg2>.
Lalam: It also connects this to exploratory model searches, meaning the results should match what you'd get from just randomly searching those same subspaces <ref:2611.13781#pg3>. It’s consistent with that kind of search.
Tom: That leads us into implication three: confirmatory testing and exploratory search give the same results under this setup <ref:2611.13781#pg3>. It means the two approaches aren't fighting each other; they just give you different views on the data evidence, which is fine as long as they agree on the overall picture.
Jane: So we start with equal prior probabilities for all hypotheses by default, but this paper gives us a specific way to define those priors based on parameter complexity so that our results stay stable and coherent regardless of how we divide the space <ref:2611.13781#pg2>. It’s about making the posterior probabilities robust.
Lu: The key is that the prior isn't arbitrary; it's derived from a single distribution on the parameters, split into those equality structures and those inequality regions based on their complexity under that encompassing prior <ref:2611.13781#pg3>. It’s a structured way to assign probability where before there was just a simple equal probability assumption.
Meng: I worry about the setup for the initial parameter distribution, though. If that first layer of priors is too restrictive, it might hide real signal or bias us toward certain types of equality constraints <ref:2611.13781#pg3>. We need to be careful about what we assume about the parameters themselves.
Paper summary: Lalam: The paper mentions that you can specify your own prior beliefs about the hypotheses, like the direction of an effect, if you want to move away from this complexity-induced prior <ref:2611.13781#pg3>. It gives us flexibility if we have strong theoretical reasons to believe something specific about the model structure.
Tom: So for someone just looking at a paper without deep prior beliefs, this method lets them test constrained hypotheses using Bayes factors while staying neutral about the constraints themselves <ref:2611.13781#pg1>. It’s a tool for when you want to quantify evidence without pre-committing to model structure.
Jane: That's the core message. It’s not about finding the absolute best model; it’s about getting a more reliable measure of uncertainty across different constrained options <ref:2611.13781#pg2>. It shows how the choice of prior can fundamentally change what we see in the posterior probabilities when we look at inequality constraints.
Lu: The authors emphasize that this complexity-induced prior is useful when you've already formulated your hypotheses based on real arguments, but you don't want to add new beliefs about those constraints <ref:2611.13781#pg3>. It bridges the gap between theoretical hypothesis formulation and empirical testing.
Meng: I think the practical recommendation is to report these prior probabilities alongside your Bayes factors and your initial prior probabilities, because they depend on those equality constraints you chose a priori <ref:2611.13781#pg3>. It’s important context for reproducibility.
Lalam: So, in summary, the paper provides a principled way to handle the ambiguity of dividing parameter space when testing hypotheses with equality and inequality constraints using Bayes factors <ref:2611.13781#pg2>. It shows that you can derive robust posterior probabilities from a single parameter prior structure.
Tom: And for the world, this means when we use Bayesian methods to test complex models, we don't have to get stuck in arguments about how we should partition our search space <ref:2611.13781#pg2>. It gives us a consistent framework for evaluating evidence across different model structures.
Jane: Exactly. We move from subjective decisions about splitting the space to a method where the posterior probabilities are mathematically guaranteed to be coherent and invariant under refinement <ref:2611.13781#pg3>. It’s about building tools that work better when we are testing things with real constraints.
Conclusion: Tom: So we’re wrapping up on "Prior Probabilities for Constrained Hypotheses: Invariance, Neutrality, and Model Search" by those authors on arXiv page eight here. Basically, they’re showing how you can test these complex hypotheses using Bayes factors without getting stuck on how you decide to split your parameter space into those equality and inequality regions.
Jane: Right. So the big takeaway is that if you use their complexity-induced prior, the posterior probabilities for your hypotheses stay consistent no matter how much you refine or rearrange those inequalities afterwards. It’s designed to make your testing more robust.
Lu: I think what they're really showing us is that this approach makes the results coherent across nested hypotheses, which avoids some of those messy situations where you might get conflicting probabilities just by making a small change to your setup.
Meng: From my side, it means when we’re shipping models, we don't have to worry about the exact way we partition the hypothesis space if our underlying parameter distribution is sound because the posterior probabilities won't jump around wildly.
Lalam: I agree with Meng. It takes that guesswork out of how you divide things up and gives you a way to compare different model structures fairly based on how complex they are under a single, unified prior.
Tom: So when we look at the title, "Invariance, Neutrality," it really boils down to this idea: we want our results to be stable—invariant—and unbiased—neutral—about which constraints we pick first.
Jane: And that neutrality comes from using a prior structure derived from just one single distribution over all your parameters instead of setting arbitrary starting probabilities for every possible hypothesis.
Lu: It’s about deriving those probabilities based on the complexity of the regions, which is what you actually get when you have a solid model on your parameter space to start with.
Meng: So it’s less about finding the perfect model and more about getting a reliable measure of evidence that doesn't depend on your arbitrary decision-making process for setting up those constraints.
Lalam: Exactly. It gives you context on how much evidence you actually have when dealing with those tricky equality and inequality constraints in your data analysis.
Tom: And the implication for the world is that this provides a much cleaner way to evaluate competing models when those models have complicated rules or boundaries defined by both strict equalities and fuzzy inequalities.
Jane: It gives researchers a principled tool for making sense of model evidence when they don't want their initial prior beliefs about the constraints to dictate the final posterior probability too strongly.
More episodes
- Daily Summary for 2026-11-19
- 2310.07973-Statistical Performance Guarantee for Subgroup Identification with Generic Machine Learning
- 2408.09155-Learning Robust Treatment Rules for Censored Data
- Daily Summary for 2026-11-19
- 2603.23397-Kinetic Langevin Splitting Schemes for Constrained Sampling
- 2606.09906-An information-geometric framework for mapping maximum potential biodiversity
- 2603.27718-Induced replication and the assessment of models
- 1307.0898-Entropy of a Zipfian Distributed Lexicon
- 2406.19974-Generalizing self-normalized importance sampling with couplings
- 2412.10213-Collaborative Optimal Design of Controlled Experiments with Subject Covariates