postcard: An R Package for Marginal Effect Estimation with or without Prognostic Score Adjustment

summary

The gist

Postcard introduces an R package that implements Generalized Linear Model (GLM) plug-in estimation with and without prognostic score adjustment to estimate marginal effects and approximate…

In short

Postcard is an R package for estimating marginal effects using Generalized Linear Model (GLM) plug-in estimation, both with and without adjusting for a prognostic score derived from historical data. It provides functions to estimate these effects, their variances, and approximate statistical power robustly, allowing researchers to perform both estimation and planning phases efficiently.

Key concepts

GLM Plug-in Estimation
This method estimates marginal effects by fitting a standard GLM and then using counterfactual predictions derived from that model. These predictions are plugged into the definition of the marginal effect, providing an estimate even when the true underlying data is unknown.
Prognostic Score Construction
A prognostic score is a prediction model built on historical data, often using a Discrete Super Learner. This score captures relevant patient characteristics that can predict future outcomes. It's used as an extra covariate in the main GLM to improve the accuracy of marginal effect estimates.
Prospective Power Approximation
This feature estimates how likely a study is to find a significant marginal effect before conducting it. Crucially, this approximation remains valid even if the initial GLM model used for prediction is slightly incorrect, leading to more realistic sample size planning.

Terminology used across episodes

This episode discusses

The paper

postcard: An R Package for Marginal Effect Estimation with or without Prognostic Score Adjustment · Read on arXiv

MATHIAS LERBECH JEPPESEN, EMILIE HÖJBJERRE-FRANDSEN

Transcript

Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "postcard: An R Package for Marginal Effect Estimation with or without Prognostic Score Adjustment".

Jane: Here's the gist of the paper. Postcard introduces an R package that implements Generalized Linear Model (GLM) plug-in estimation with and without prognostic score adjustment to estimate marginal effects and approximate statistical power.

Tom: Methodological Foundation The package is built upon the GLM plug-in method introduced by Rosenblum and van der Laan

2010: for marginal effect estimation.

Jane: This procedure estimates marginal effects by fitting a GLM and then extracting counterfactual predictions, followed by plugging these into the marginal effect definition.

Tom: The variance is estimated using the variance of the influence function (IF) via a formula derived from the delta method.

Title and authors: Tom: So we're looking at Postcard: An R Package for Marginal Effect Estimation with or without Prognostic Score Adjustment. Jane, what’s the main idea behind this tool?

Jane: Well, it’s about estimating marginal effects using a method called the GLM plug-in procedure, and they built an R package around it that lets you do this both with and without adjusting for a prognostic score. That means you get to see how much a treatment changes an outcome while accounting for some extra information from past data.

Tom: So it’s about getting the marginal effect estimate itself, but also figuring out the variance and even approximating the statistical power of that estimate? I like that part about power approximation; we always worry about having enough participants to actually find a real effect.

Lu: What's interesting here is how they connect these pieces. They take the GLM plug-in method, which is already a way to get marginal effects, and they extend it by adding this prognostic score adjustment, which uses historical data to create a single covariate that summarizes predictive information from previous trials.

Meng: From my side at the startup, I’m always thinking about how much work this adds. The paper mentions using a Discrete Super Learner to build these prognostic scores, which sounds like it needs a lot of training data and careful selection of learners. That’s going to be key for us to actually deploy this in production.

Jane: Exactly, Meng. And the paper points out that while including too many covariates can cause overfitting and bias, using a single prognostic score constructed from historical data can help harness predictive information without introducing new bias into the analysis itself <ref:2510.13347#pg2>.

Tom: So what are they saying about the efficiency gains when you use this prognostic score adjustment compared to just running the GLM plug-in method on its own?

Lu: They show that this combination achieves semi-parametric efficiency under specific assumptions, which means it works well even if we don't know the exact shape of the relationship between treatment and outcome. That broadens what you can apply this to beyond just simple continuous endpoints <ref:2510.13347#pg1>.

Title and authors: Meng: Semi-parametric efficiency is good, but how does that actually translate when you’re trying to run this on a real, messy dataset? I need more concrete numbers on how much precision we gain versus the complexity added by building that prognostic score <ref:2510.13347#pg2>.

Jane: The package provides functions for both scenarios, including rctglm for the basic plug-in and rctglm with prognosticscore which handles the adjustment part, and it lets you see how the power changes when you add that prognostic score <ref:2510.13347#pg3>.

Tom: And what about constructing that prognostic score itself? The paper mentions using a Discrete Super Learner for this, which sounds like a powerful way to build a predictive covariate from baseline variables <ref:2510.13347#pg3>.

Lu: The idea is that you can use the Discrete Super Learner to flexibly construct highly predictive prognostic scores from baseline covariates, which helps you create those single covariates without having to guess which specific features are most important for prediction <ref:2510.13347#pg3>.

Jane: It sounds like this moves us away from just picking a few baseline variables manually and lets the model suggest the best summary of historical information for adjustment <ref:2510.13347#pg2>.

Meng: If we were deploying this, we’d have to worry about how stable that prognostic score is when we move from training data to live traffic; it's a big unknown in production environments <ref:2510.13347#pg2>.

Tom: That leads into the next part: power approximation. The paper introduces a novel procedure for this that’s supposed to be valid even when the underlying GLM model isn't perfectly specified <ref:2510.13347#pg3>.

Jane: That’s a big deal, because usually if your model is wrong, your power calculation is also wrong; this procedure tries to give you a realistic sample size estimate even under those misspecification conditions <ref:2510.13347#pg3>.

Lu: The power approximation functionality relies on the Discrete Super Learner for scenarios involving prognostic adjustment, meaning the method uses that same learning tool to calculate how much data you need based on the expected performance of the model <ref:2510.13347#pg3>.

Meng: But if you can’t perfectly specify your linear model, relying on a complex learned score might just introduce new forms of error into the power calculation itself; that's where I get nervous about it scaling up <ref:2510.13347#pg2>.

Title and authors: Jane: The paper does acknowledge that the package supports power calculations under the assumption of a correctly specified linear model, but they highlight how this applies to GLM plug-in analyses with or without prognostic score adjustment <ref:2510.13347#pg3>.

Tom: So, to wrap up on what it means for us as researchers and practitioners: this package gives us a unified interface for estimation, adjustment via historical data, and power planning <ref:2510.13347#pg2>.

Lu: It gives you a way to harness predictive information from previous trials without introducing bias into the analysis by using prognostic scores derived from historical controls <ref:2510.13347#pg2>.

Meng: For deployment, it means we have a tool that lets us test if adding that prognostic score actually improves precision enough to justify the complexity of building it <ref:2510.13347#pg2>.

Jane: Ultimately, this paper with the title Postcard: An R Package for Marginal Effect Estimation with or without Prognostic Score Adjustment offers a practical toolkit for statisticians by integrating estimation, prognostic adjustment, and power approximation into a single interface <ref:2510.13347#pg2>.

Tom: It’s a solid foundation if you're looking to improve the efficiency of treatment effect estimators using covariate adjustment <ref:2510.13347#pg1>.

Lu: And it shows how using prognostic scores can enhance precision in statistical analysis, especially when we assume additive treatment effects on the link scale <ref:2510.13347#pg1>.

Meng: We have to keep an eye on that overfitting risk when we use the Discrete Super Learner to build those scores because it's a big factor for real-world performance <ref:2510.13347#pg2>.

Jane: And remember, they’ve shown ways to approximate statistical power even when the linear model is not perfectly specified, which helps us plan more robust study designs based on historical data <ref:2510.13347#pg3>.

Tom: So we're looking at a package that lets you estimate marginal effects with or without prognostic score adjustment, and it gives you tools for power approximation that handle model misspecification <ref:2510.13347#pg2>.

Lu: It’s about harnessing predictive information from previous trials to enhance precision in statistical analysis <ref:2510.13347#pg2>.

Jane: That's the gist of Postcard: An R Package for Marginal Effect Estimation with or without Prognostic Score Adjustment, and it provides a practical toolkit for statisticians <ref:2510.13347#pg2>.

The paper's summary: Tom: So we're talking about Postcard now, and it basically puts together an R package that lets you estimate marginal effects using this GLM plug-in method, but you can choose whether or not to add a prognostic score adjustment in there.

Jane: Right. Think of it like this: you want to know the treatment effect on something, but instead of just looking at the raw data for that specific patient, you want to use some historical info—a prognostic score—to make your estimate more precise.

Lu: The core idea they're pushing is that by plugging these prognostic scores into the GLM plug-in procedure, you get semi-parametric efficiency. This means it works better even when you don't know exactly how the treatment and outcome are related on the scale of your model.

Meng: But adding a prognostic score means we need to build this score first, usually from historical data, often using something like a Discrete Super Learner to pick the best predictive features. That sounds like it’s going to take a lot of work just to create that single summary covariate.

Tom: And that's where Postcard comes in—it gives you the functions for both ways: the standard plug-in estimation and then the version with that prognostic score adjustment turned on. It also handles estimating the variance, which is usually a tricky part when you’re doing these kinds of calculations.

Jane: It really changes how we think about planning studies too because they include a function to approximate statistical power. This lets you figure out how many participants you actually need before you start collecting data, and it does this even if the main GLM model isn't perfectly specified.

Lu: That power approximation part is pretty interesting because they’ve made a procedure that works even under model misspecification, which means it gives a more realistic sample size estimate for planning purposes.

Meng: For production, we need to know how much better the adjustment actually makes the estimate compared to just using the standard method without any prognostic score. That's our real question: is building this whole system worth the extra complexity and computation cost?

Tom: That’s exactly what they show in their simulations; they compare models with and without that prognostic score adjustment to see how much precision you gain, which helps you decide if the added complexity is justified by the better estimate.

Jane: It feels like a toolkit for anyone doing causal inference or looking at treatment effects where historical context is important, whether that's in clinical settings or just looking at complex observational data.

Lu: The implication here is that we can leverage predictive information from past trials to improve the accuracy of current effect estimates without having to manually pick every single covariate we think might matter.

Tom: So, Postcard is essentially providing a unified interface for researchers who want to combine estimation, using historical adjustments, and planning their study size all in one place.

Jane: And that moves us toward being able to perform both the estimation phase and the planning phase of research much more efficiently than before.

Meng: It's about making sure the AI systems we build for trial design aren't just guessing based on simple models, but are using learned historical context to make those plans more robust.

Lu: We have a solid foundation here showing that incorporating these learned prognostic scores can enhance precision, especially when you assume additive treatment effects on the link scale.

The paper's improvements: Tom: So we’re looking at what the authors suggest is better for future versions of Postcard, mainly focusing on how they handle the prognostic score adjustment itself.

Jane: They want to make sure this method works well even if you’re dealing with non-continuous outcomes, not just simple continuous numbers. They're aiming for that semi-parametric efficiency we talked about earlier.

Lu: The idea is that by integrating these prognostic scores into the GLM plug-in procedure, they achieve this efficiency under the assumption of additive treatment effects on the link scale. This opens it up for discrete outcomes too, which is a big deal.

Meng: But that requires careful assumptions about how the treatment and outcome interact when you add that historical covariate. We need to see if those assumptions hold up when we move from a simulation to actual, messy production data where things aren't perfectly linear.

Tom: They also highlighted how the method can be used with AI systems to estimate marginal effects with much greater statistical precision when you use historical data for covariate adjustment. That’s a key win for us in building smarter models.

Jane: And they showed that this combination of estimation and adjustment gives researchers a practical toolkit for both the planning phase and the actual estimation phase, which is really efficient.

Lu: Regarding the prognostic score construction, they suggest using tools like the Discrete Super Learner to build these scores from baseline covariates without introducing bias from just picking a few variables manually.

Meng: That's good because it means we don't have to be experts at selecting which historical data points are important; the model can suggest the best summary of that predictive information.

Tom: They also focused on making sure their power approximation procedure is robust, meaning it still gives you a realistic sample size estimate even when the main GLM model isn't perfectly specified. That's a strong feature for study design planning.

Jane: So, the improvements are really about making it more flexible—handling different data types and providing better planning tools that don't break just because the math isn't perfect.

Lu: They’re pushing toward a system where you can build these scores flexibly, and then use that learned structure to enhance both estimation accuracy and sample size planning.

Meng: For deployment, the challenge remains how stable those scores are over time; we still need rigorous testing before we rely on them for live inference.

Conclusion: Tom: So to wrap up, Postcard is an R package that gives you the tools to estimate marginal effects using GLM plug-in methods, with or without adjusting for a prognostic score derived from historical data.

Jane: It really boils down to giving researchers a unified interface for estimation, adjustment via historical data, and power approximation all in one place.

Lu: The main implication is that you can harness predictive information from previous trials to improve the accuracy of current effect estimates without having to manually pick every single covariate we think might matter.

Meng: It means we have a tool that lets us test if adding that prognostic score actually improves precision enough to justify the extra computational cost of building it.

Lalam: I'm confident in this direction because it systematically integrates estimation, adjustment, and planning into a single workflow.

Tom: And they also showed ways to approximate statistical power even when the linear model isn't perfectly specified, which helps us plan more robust study designs based on historical data.

Jane: It’s about making sure we have a realistic sample size estimate for our research, even when our underlying mathematical model is slightly off.

Lu: The method provides a way to leverage prognostic scores to enhance precision, especially when you assume additive treatment effects on the link scale.

Meng: We still need to keep an eye on that risk of overfitting when we use the Discrete Super Learner to build those scores because it's a big factor for real-world performance.

Tom: That’s what Postcard does: it gives you a solid foundation if you're looking to improve the efficiency of treatment effect estimators using covariate adjustment.

Jane: It shows how using prognostic scores can enhance precision in statistical analysis, especially when we assume additive treatment effects on the link scale.

Lu: And it demonstrates how leveraging historical context can enhance precision by incorporating those learned structures into the estimation process.

More episodes

← Home