FDR-Controlled Variable Selection for Generalized Linear Models and Cox Regression with Virtual Dummies
summary
The gist
In genomics, imaging and clinical studies, only a few of many candidate predictors are often nonlinearly associated with a response that may be, e.g., binary, categorical…
In short
The paper introduces a memory-efficient variable selection method called Virtual Dummies (VD) for generalized linear models and Cox regression. It extends virtual dummy construction to score-based forward selection, controlling the false discovery rate (FDR) by making synthetic null variables compete with real predictors.
Key concepts
- Virtual Dummy (VD)
- Instead of storing full vectors for each variable, VD stores only its projections onto an adaptively growing subspace. This saves significant memory while maintaining the same statistical properties as traditional explicit augmentation methods, allowing FDR control to be preserved.
- Score-Based Forward Selection
- This is a selection algorithm that ranks candidate predictors based on how well they fit the model, using a score derived from the residual (e.g., $y - ext{mean}$). The VD framework adapts this ranking by splitting each dummy into revealed and unrevealed parts computable from its projections.
- FDR Control
- FDR control aims to limit the expected proportion of false discoveries among selected variables. The VD method achieves this by letting synthetic null variables compete with real predictors, ensuring that the selection path remains equivalent in distribution to explicit methods like T-Rex.
- Generalized Linear Models (GLMs)
- GLMs are statistical models used when the response variable can be binary, count, or a censored event time. The VD framework is specifically extended to handle these diverse response types by adapting the virtual dummy construction for score ranking.
Terminology used across episodes
This episode discusses
- FDR-Controlled Variable Selection for Generalized Linear Models and Cox Regression with Virtual Dummies · Paper Radio
- Virtual Dummies: Enabling Scalable FDR-Controlled Variable Selection via Sequential Sampling of Null Features
The paper
FDR-Controlled Variable Selection for Generalized Linear Models and Cox Regression with Virtual Dummies · Read on arXiv
TU Darmstadt
Transcript
Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "FDR-Controlled Variable Selection for Generalized Linear Models and Cox Regression with Virtual Dummies".
Jane: Here's the gist of the paper. In genomics, imaging and clinical studies, only a few of many candidate predictors are often nonlinearly associated with a response that may be, e.g., binary, categorical, a count or a censored event time <ref:2609.29335#pg2> The Terminating-Random Experiments (T-Rex) selector is a scalable variable selection method that controls the false discovery rate (FDR) by letting synthetic null variables (dummies) compete with the real predictors <ref:2609.29335#pg2> We propose a memory-efficient selection procedure with FDR control for generalized linear models and Cox regression by extending the recently developed virtual dummy construction to score-based forward selection for Bernoulli, Poisson, multinomial and Cox responses <ref:2609.29335#pg2> The virtual-dummy-based selection path remains equal in distribution to explicit augmentation, so FDR control carries over under the same assumptions <ref:2609.29335#pg2>.
Tom: Variable Selection Context The goal of variable selection in fields like genome-wide association studies and cancer survival analysis is to identify active predictors among a large number of candidates where the response can be binary, categorical, a count or a censored event time <ref:2609.29335#pg2>.
Jane: The main current paradigm involves controlling the false discovery rate (FDR), which is the expected proportion of false discoveries among the selected variables while recovering as many active predictors as possible <ref:2609.29335#pg2>.
Tom: Classical methods like Benjamini-Hochberg and Benjamini-Yekutieli control FDR using p-values, but these are not valid in multivariate models with more predictors than observations <ref:2609.29335#pg2>.
Paper summary: Tom: So we're looking at this paper here, "FDR-Controlled Variable Selection for Generalized Linear Models and Cox Regression with Virtual Dummies". Basically, they're tackling a problem where you have a huge number of candidate predictors, but only a few are actually related to the outcome you care about.
Jane: Right. The core idea seems to be extending some existing selection methods, like the T-Rex selector which controls false discovery rate or FDR by having synthetic null variables compete with real predictors, into generalized linear models and Cox regression.
Lu: What they claim is a memory-efficient way to do this while still controlling that FDR. They're proposing this virtual dummy construction instead of just augmenting the traditional method, which can get really big in terms of storage.
Meng: From an engineering standpoint, memory is everything when you're dealing with high-dimensional data at scale. So if this avoids storing those huge n by L matrices mentioned in page two, that's a win for deployment.
Tom: Exactly. They state that the virtual dummy-based selection path remains equal in distribution to explicit augmentation, which means the FDR control result carries over under the same assumptions as before. That’s a big deal because it validates the method theoretically while making it practical computationally.
Jane: It sounds like they are proving that this new way of selecting variables isn't just a clever trick for memory; it’s mathematically sound in terms of controlling false discoveries when you're dealing with complex model types.
Lalam: I see what they did here, and I can confirm the literature supports this equivalence. The paper states that the equivalence to explicit augmentation of the virtual dummies algorithm rests on two conditions: the dummies are drawn independently of X, y with coefficients from (two), and the selection rule and each new basis vector depend on the dummies only <ref:2609.29335#pg1>.
Tom: That independence is key for them to maintain that FDR control guarantee. But then they extend this framework to handle Bernoulli, Poisson, multinomial, and Cox responses using score-based forward selection.
Lu: The method adapts the virtual dummy construction by splitting each dummy into a revealed and an unrevealed part that can be computed just from the scalars alpha values alone, which is what saves memory.
Jane: So they’re not just applying it to linear models anymore; they’re getting this working for different types of responses, like counts or censored event times in survival analysis. That expands the applicability quite a bit.
Meng: For production, I need to know how this scores-based forward selection actually performs on real traffic. Is it fast enough? The paper mentions a computational complexity that is linear in n and p, which is good, but what about the actual runtime when you run this on a massive set of variables?
Paper summary: Tom: They give us some concrete runtime comparisons. For instance, VD-GLM adds only around two hundred five megabytes for Poisson, Bernoulli and Cox instantiations compared to AD-GLM, and it's two point two to two point five times faster than AD-GLM. The absolute VD-GLM runtimes range from about twenty five seconds to thirty seven seconds against fifty eight seconds to eighty nine seconds for AD-GLM.
Jane: That speed difference is significant when you have high dimensions, so a process that runs in about thirty seven seconds instead of nearly ninety is much more viable in a real-world scenario.
Lu: And they show empirical validation with simulations confirming that the virtual dummy framework is equivalent to explicit augmentation, and correct specification yields higher power than selection under a misspecified linear model.
Tom: They also cite specific power gains for their applications. In genomic settings, VD-Multinomial recovers more than twice as many causal SNPs as any competitor within the target false discovery rate.
Jane: That sounds substantial when you're trying to find true causal signals in a complex dataset like genomics. The Cox instantiation selects eight genes against one for the linear model on log t, which is a pretty tight result for survival data analysis.
Meng: So they’ve shown it works well empirically, but I’m always cautious about benchmarks that don't use real traffic. What if the assumptions they used in those simulations don't hold when we deploy this to live data?
Lu: The paper acknowledges a limitation in the conclusion. They say the loss in true positive rate is most pronounced for multinomial response, while for Bernoulli responses it is negligible.
Jane: That's an important caveat because if you're dealing with a multinomial outcome, you should probably expect a slightly lower true positive rate compared to what you might see with simpler responses like binary ones.
Tom: So, to wrap up on the "FDR-Controlled Variable Selection for Generalized Linear Models and Cox Regression with Virtual Dummies," this paper proposes a memory-efficient method that controls false discovery rate by extending virtual dummy construction to score-based forward selection across four different response types.
Jane: The implication is that researchers in genomics and clinical studies can now use this approach to find active predictors in complex models without getting bogged down by the massive memory requirements of traditional methods.
Lu: It changes how we think about these algorithms because it shows you don't necessarily need the full matrix of all possible variables to control FDR effectively, you just need those projections onto an adaptively growing subspace.
Meng: For deployment, this means a potentially much lighter footprint for model selection components in high-dimensional settings. We'll have to see how robust it is when we move from simulation to real-world data streams.
Tom: It seems like a solid step forward in making these complex selection procedures more scalable and computationally feasible for the kind of massive datasets we deal with daily.
Conclusion: Tom: So we're wrapping up on this paper about "FDR-Controlled Variable Selection for Generalized Linear Models and Cox Regression with Virtual Dummies."
Jane: What they did is take a method that controls false discovery rate, like T-Rex, and extend it to more complex models—generalized linear models and survival analysis.
Lu: Basically, they propose a new way to select predictors called the virtual dummy construction that makes the memory usage much more efficient.
Meng: So instead of storing all those huge data matrices for every possible variable, you only store these smaller projections, which is pretty smart for production systems.
Tom: They prove that this new approach is mathematically sound and gives you the same false discovery rate control as the older explicit augmentation method, which is a big validation point.
Jane: It means researchers can use this to find important predictors in massive datasets without hitting those huge memory walls.
Lu: The extension they made allows them to do this for Bernoulli, Poisson, multinomial responses, and even Cox regression models too.
Meng: But what's the catch? They admit that for some response types, like multinomial data, you might lose a little bit of the true positive rate.
Tom: That’s a necessary caveat because you gotta know where the method stops being perfect.
Jane: It shows that this virtual dummy method can actually give higher power than just selecting variables without controlling anything, even when the underlying linear model is slightly wrong.
Lu: And they show it works well empirically, recovering more causal signals in genomic applications than many other methods we've seen.
Meng: For someone shipping models to production, the runtime comparison shows this method is actually faster than some of the established alternatives we've been testing.
Tom: So it seems like a solid way to handle high-dimensional data selection that balances computational cost with statistical control.
More episodes
- Daily Summary for 2026-11-19
- 2310.07973-Statistical Performance Guarantee for Subgroup Identification with Generic Machine Learning
- 2408.09155-Learning Robust Treatment Rules for Censored Data
- Daily Summary for 2026-11-19
- 2603.23397-Kinetic Langevin Splitting Schemes for Constrained Sampling
- 2606.09906-An information-geometric framework for mapping maximum potential biodiversity
- 2603.27718-Induced replication and the assessment of models
- 1307.0898-Entropy of a Zipfian Distributed Lexicon
- 2406.19974-Generalizing self-normalized importance sampling with couplings
- 2412.10213-Collaborative Optimal Design of Controlled Experiments with Subject Covariates