Induced replication and the assessment of models
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "Induced replication and the assessment of models".
Paper summary: Tom: So, to summarize this paper on "Induced replication and the assessment of models," Heather Battey and Nancy Reid are looking at assessing highly-parametrised models from a foundational perspective.
Jane: The central thesis is that we can avoid the usual semiparametric considerations that usually require estimating nuisance components through things like kernel smoothing or basis expansion, which introduces problems with tuning parameters.
Lu: Instead, they propose focusing on inducing replication under the postulated model as a key aspect of assessment. This replication can be framed using non-standard inferential separations, specifically Fisherian ancillarity or sufficiency separations.
Meng: So this means instead of estimating all those nuisance parts, we check if the model behaves consistently within the data structure itself when you use these specific ways of separating information.
Tom: Precisely. They claim that this allows replacing out-of-sample prediction error as a criterion for semiparametric assessment with a type of within-sample prediction error instead.
Jane: It matters because it provides a new way to induce replication, and they demonstrate this in several settings, including the proportional hazards model and for time-dependent Poisson processes with semiparametric intensity functions.
Lalam: In the time-dependent Poisson process example, they show that induced replication exists in the conditional distributions across individuals when you look at it through sufficiency separations.
Tom: That’s a concrete finding because they show that under certain conditions, you can check for model adequacy using empirical behavior of statistics like U sub j of psi zero for postulated values psi zero.
Jane: And numerically, simulations confirm the recovery of nominal error rates when the model is correct, but it also shows high sensitivity when you move in directions that are semiparametric alternatives.
Conclusion: Tom: So, looking at "Induced replication and the assessment of models," we see a lot about how to check complex statistical models without getting bogged down in estimating nuisance components manually.
Jane: The paper suggests that by focusing on inducing replication through specific inferential separations—like Fisherian ancillarity or sufficiency separations—we can replace out-of-sample error checks with within-sample ones.
Lu: Essentially, the implication is that if you can correctly specify a model to an adequate order of approximation, this internal replication should be induced, which is then checked using statistics like R.
Meng: For someone in production, that means we might be able to streamline the assessment process by focusing on these statistical separations rather than needing complex estimation routines for every single nuisance parameter.
Tom: It’s a shift from a traditional way of thinking about model checking to one where replication under the postulated model becomes the central test, even if it requires looking at different types of information separation.
Jane: And while they show this works in specific examples, they also lay out that there are still open challenges—namely, finding a general way to induce internal replication without needing those nuisance parameters to be estimated approximately first.
Lalam: So the big takeaway is that these separations give us a new statistical language for checking model adequacy based on how the model behaves internally under specific conditions.
Department of Mathematics, Imperial College London · Department of Statistical Sciences, University of Toronto
stat.ME, math.ST, stat.TH
Submitted: 2026-03-29
Updated: 2026-11-18
License: http://creativecommons.org/licenses/by/4.0/
The gist: We study the assessment of semiparametric and other highly-parametrised models from the perspective of foundational principles of parametric statistical inference In doing so…
Key concepts
- Fisherian ancillarity/co-ancillarity
- This is a specific type of statistical separation used to avoid the usual need for estimating nuisance parameters in semiparametric models. It helps define conditions where replication under the postulated model can be achieved based on internal statistical structures, rather than external data fitting.
- Sufficiency/co-sufficiency separations
- This concept involves looking at how information is structured within a dataset. Using sufficiency and co-sufficiency allows researchers to establish criteria for model assessment by examining internal data relationships, which can induce replication under the assumed model without estimating nuisance components.
- Inducing Replication
- This means showing that if a model is correctly specified to an adequate order of approximation, then the observed data will behave as if they were generated by that postulated model. The paper seeks to define this replication using specific inferential separations as a criterion for assessing model adequacy.
- Within-sample prediction error
- Instead of relying on predictions made on new, unseen data (out-of-sample error), this approach uses errors calculated from the data itself. This internal assessment is used as a powerful tool to determine if the postulated model structure is appropriate for describing the observed phenomena.
Terminology
Summary
We study the assessment of semiparametric and other highly-parametrised models from the perspective of foundational principles of parametric statistical inference In doing so, we highlight the possibility of avoiding the usual semiparametric considerations, which typically require estimation of nuisance components through kernel smoothing or basis expansion, with the associated difficulties of tuning-parameter choice that blur the distinction between estimation and model assessment A key aspect is the inducement of replication under the postulated model This can be cast in terms of some non-standard inferential separations, in the vein of Fisherian ancillarity/co-ancillarity and sufficiency/co-sufficiency separations, allowing the replacement of out-of-sample prediction error as a criterion for semiparametric model assessment by a type of within-sample prediction error Framed in this light are new methodological contributions in multiple example settings, including model assessment for the proportional hazards model, for a time-dependent Poisson process with semiparametric intensity function, and for matched-pair and two-group examples Numerical work confirms recovery of nominal error rates under the postulated model and high sensitivity to departures in the direction of semiparametric alternatives
Foundational Inferential Separations
The paper explores inferential structures beyond standard methods by focusing on separations like Fisherian ancillarity/co-ancillarity and sufficiency/co-sufficiency to avoid the usual semiparametric considerations The idea is to induce replication under the postulated model if and only if it is correctly specified to an adequate order of approximation This involves using non-standard inferential separations as a criterion for semiparametric model assessment by a type of within-sample prediction error
Inducing Replication in Specific Settings
The authors illustrate several routes to inducing internal replication under the postulated model, such as those based on sufficiency/co-sufficiency separations and ancillarity/co-ancillarity separations Examples include:
-
Example 1.1, which shows how to assess matched pair problems by replacing out-of-sample prediction error with an in-sample assessment based on the structure of the postulated model
-
In the semiparametric time-dependent Poisson process, induced replication exists in the conditional distributions across individuals
-
Post-reduction inference approaches are used for constructing confidence sets of sparse regression models
-
For the proportional hazards model, assessment is framed in terms of conditional co-sufficiency using ideas from §3. p. 0
3Where the paper says it
This can be cast in terms of some non-standard inferential separations, in the vein of Fisherian ancillarity/co-ancillarity and sufficiency/co-sufficiency separations, allowing the replacement of out-of-sample prediction error as a criterion for semiparametric model assessment by a type of within-sample prediction error. Framed in this light are new methodological contributions in multiple example settings, including model assessment for the proportional hazards model, for a time-dependent Poisson process with semiparametric intensity function, and for matched-pair and two-group examples.
Assessment via Confidence Sets and Replication Statistics
The paper develops a framework where internal replication is induced if and only if the model is correctly specified to an adequate order of approximation This is formalized through four generic inferential structures (Structures 1–4) based on different types of information separation, which define equivalence classes where the approach has no power to reject the postulated model if the true density function belongs to that class The statistic R = -log Uj(ψˆ) is used for assessment, and its extremity against the χ2 2m distribution casts doubt on model adequacy
Numerical Performance and Sensitivity Analysis
Numerical simulations confirm the recovery of nominal error rates under the postulated model and high sensitivity to departures in the direction of semiparametric alternatives In matched-pair examples, power to detect departures from the model increases with sample size m except when ς = 1, where the true Weibull distribution collapses to the exponential and therefore is closer to the point at which the true and postulated models intersect In time-dependent Poisson process simulations, high power to detect departures from the postulated model is shown even at values of ρ very close to the point of intersection ρ = 0
Open Challenges and Future Directions
The work emphasizes open challenges, noting that a general resolution for inducing internal replication under a postulated model might be necessary An important open question concerns the possibility of eliminating nuisance parameters approximately, suggesting that a notion of approximate exchangeability or collapsing nuisance parameters into a low-dimensional summary is needed There is also an interest in reparametrisation-based assessments of model adequacy, as reparametrisation fails to induce population-level sparsity if the model is violated
Proofs and Derivations
The paper provides formal proofs for key propositions, such as Proposition 2.1 establishing the convergence in distribution of Uj(ψˆ) to Uj(ψ∗0) These derivations include the derivation of equation (7), which bounds the probability pr(R ≥ kα) The proof for Proposition 4. p. 8 β
mi exp(β
Pmi
j=1 tij )
(e
βt0 − 1)mi
, (10)
which is the probability density function of an ordered sample of mi random variables each
with density function
fT (t; β) = βeβt
(e
βt0 − 1), 0 ≤ t ≤ t0, (11)
(Cox and Lewis, 1966, pp. Equation (11) reduces to 1/t0 in the limit as β → 0.Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Ability to perform model assessment without estimating nuisance components by using
non-standard inferential separations, in the vein of Fisherian ancillarity/co-ancillarity and sufficiency/co-sufficiency separations.
This allows for awithin-sample prediction error
criterion instead of out-of-sample prediction error. -
Capability to construct confidence sets for sparse regression models using a
post-reduction inference approach
that avoids discarding information at the model assessment phase, as described in Battey, Rasines & Tang (2025). -
Implementation of inferential structures based on internal replication, specifically Structures 1 through 4, allowing the system to test model adequacy by checking if
Uj (psi0) U(0, 1) if and only if M holds with true parameter value psi = psi0.
-
Functionality to perform local power analyses by assessing sensitivity to departures from the postulated model at points of intersection between models, such as the point where
the true and postulated models intersect (delta star, sigma) = (0, 1).
-
Use of maximum likelihood estimation based on induced random variables Zj = Yj1/Yj0 to estimate parameters like psi star zero in a manner that remains consistent even when the model is misspecified, as shown by the result
psi star zero = argmax psi in R+ log psi - 2 Z infinity 0 log(1 + psi z) (1 + psi star z sigma z sigma-1 dz (1 + psi star z sigma).
Abstract
The paper is a first attempt at a foundation of inference for the assessment of semiparametric and other high-dimensional models, seeking reconciliation with the low-dimensional Fisherian foundations by replacing out-of-sample prediction error as a criterion for semiparametric model assessment by a type of within-sample prediction error. We highlight the possibility, in some contexts, of avoiding the usual semiparametric considerations, including estimation of unparametrised nuisance components and the associated difficulties of tuning-parameter choice that blur the distinction between estimation and model assessment. A key aspect of the work is the inducement of replication under the postulated model, the route to which being context-specific but based on structural properties of the model. The ideas are isolated through examples.
Sources
Related papers
- Learning Robust Treatment Rules for Censored Data
- Neural Estimation of Summary Statistics for Simulation-Based Inference with Non-Asymptotic Guarantees
- Spectral divide-and-conquer MCMC for long stationary time series
- Improving the Efficiency of Subgroup Analysis in Randomized Controlled Trials with TMLE
- Personalized Federated Vector Autoregression with Personalization Diversity
- Doubly Robust and Efficient Calibration of Prediction Sets for Right-Censored Time-to-Event Outcomes