FDR-Controlled Variable Selection for Generalized Linear Models and Cox Regression with Virtual Dummies
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "FDR-Controlled Variable Selection for Generalized Linear Models and Cox Regression with Virtual Dummies".
Jane: Here's the gist of the paper. In genomics, imaging and clinical studies, only a few of many candidate predictors are often nonlinearly associated with a response that may be, e.g., binary, categorical, a count or a censored event time <ref:2609.29335#pg2> The Terminating-Random Experiments (T-Rex) selector is a scalable variable selection method that controls the false discovery rate (FDR) by letting synthetic null variables (dummies) compete with the real predictors <ref:2609.29335#pg2> We propose a memory-efficient selection procedure with FDR control for generalized linear models and Cox regression by extending the recently developed virtual dummy construction to score-based forward selection for Bernoulli, Poisson, multinomial and Cox responses <ref:2609.29335#pg2> The virtual-dummy-based selection path remains equal in distribution to explicit augmentation, so FDR control carries over under the same assumptions <ref:2609.29335#pg2>.
Tom: Variable Selection Context The goal of variable selection in fields like genome-wide association studies and cancer survival analysis is to identify active predictors among a large number of candidates where the response can be binary, categorical, a count or a censored event time <ref:2609.29335#pg2>.
Jane: The main current paradigm involves controlling the false discovery rate (FDR), which is the expected proportion of false discoveries among the selected variables while recovering as many active predictors as possible <ref:2609.29335#pg2>.
Tom: Classical methods like Benjamini-Hochberg and Benjamini-Yekutieli control FDR using p-values, but these are not valid in multivariate models with more predictors than observations <ref:2609.29335#pg2>.
Paper summary: Tom: So we're looking at this paper here, "FDR-Controlled Variable Selection for Generalized Linear Models and Cox Regression with Virtual Dummies". Basically, they're tackling a problem where you have a huge number of candidate predictors, but only a few are actually related to the outcome you care about.
Jane: Right. The core idea seems to be extending some existing selection methods, like the T-Rex selector which controls false discovery rate or FDR by having synthetic null variables compete with real predictors, into generalized linear models and Cox regression.
Lu: What they claim is a memory-efficient way to do this while still controlling that FDR. They're proposing this virtual dummy construction instead of just augmenting the traditional method, which can get really big in terms of storage.
Meng: From an engineering standpoint, memory is everything when you're dealing with high-dimensional data at scale. So if this avoids storing those huge n by L matrices mentioned in page two, that's a win for deployment.
Tom: Exactly. They state that the virtual dummy-based selection path remains equal in distribution to explicit augmentation, which means the FDR control result carries over under the same assumptions as before. That’s a big deal because it validates the method theoretically while making it practical computationally.
Jane: It sounds like they are proving that this new way of selecting variables isn't just a clever trick for memory; it’s mathematically sound in terms of controlling false discoveries when you're dealing with complex model types.
Lalam: I see what they did here, and I can confirm the literature supports this equivalence. The paper states that the equivalence to explicit augmentation of the virtual dummies algorithm rests on two conditions: the dummies are drawn independently of X, y with coefficients from (two), and the selection rule and each new basis vector depend on the dummies only <ref:2609.29335#pg1>.
Tom: That independence is key for them to maintain that FDR control guarantee. But then they extend this framework to handle Bernoulli, Poisson, multinomial, and Cox responses using score-based forward selection.
Lu: The method adapts the virtual dummy construction by splitting each dummy into a revealed and an unrevealed part that can be computed just from the scalars alpha values alone, which is what saves memory.
Jane: So they’re not just applying it to linear models anymore; they’re getting this working for different types of responses, like counts or censored event times in survival analysis. That expands the applicability quite a bit.
Meng: For production, I need to know how this scores-based forward selection actually performs on real traffic. Is it fast enough? The paper mentions a computational complexity that is linear in n and p, which is good, but what about the actual runtime when you run this on a massive set of variables?
Paper summary: Tom: They give us some concrete runtime comparisons. For instance, VD-GLM adds only around two hundred five megabytes for Poisson, Bernoulli and Cox instantiations compared to AD-GLM, and it's two point two to two point five times faster than AD-GLM. The absolute VD-GLM runtimes range from about twenty five seconds to thirty seven seconds against fifty eight seconds to eighty nine seconds for AD-GLM.
Jane: That speed difference is significant when you have high dimensions, so a process that runs in about thirty seven seconds instead of nearly ninety is much more viable in a real-world scenario.
Lu: And they show empirical validation with simulations confirming that the virtual dummy framework is equivalent to explicit augmentation, and correct specification yields higher power than selection under a misspecified linear model.
Tom: They also cite specific power gains for their applications. In genomic settings, VD-Multinomial recovers more than twice as many causal SNPs as any competitor within the target false discovery rate.
Jane: That sounds substantial when you're trying to find true causal signals in a complex dataset like genomics. The Cox instantiation selects eight genes against one for the linear model on log t, which is a pretty tight result for survival data analysis.
Meng: So they’ve shown it works well empirically, but I’m always cautious about benchmarks that don't use real traffic. What if the assumptions they used in those simulations don't hold when we deploy this to live data?
Lu: The paper acknowledges a limitation in the conclusion. They say the loss in true positive rate is most pronounced for multinomial response, while for Bernoulli responses it is negligible.
Jane: That's an important caveat because if you're dealing with a multinomial outcome, you should probably expect a slightly lower true positive rate compared to what you might see with simpler responses like binary ones.
Tom: So, to wrap up on the "FDR-Controlled Variable Selection for Generalized Linear Models and Cox Regression with Virtual Dummies," this paper proposes a memory-efficient method that controls false discovery rate by extending virtual dummy construction to score-based forward selection across four different response types.
Jane: The implication is that researchers in genomics and clinical studies can now use this approach to find active predictors in complex models without getting bogged down by the massive memory requirements of traditional methods.
Lu: It changes how we think about these algorithms because it shows you don't necessarily need the full matrix of all possible variables to control FDR effectively, you just need those projections onto an adaptively growing subspace.
Meng: For deployment, this means a potentially much lighter footprint for model selection components in high-dimensional settings. We'll have to see how robust it is when we move from simulation to real-world data streams.
Tom: It seems like a solid step forward in making these complex selection procedures more scalable and computationally feasible for the kind of massive datasets we deal with daily.
Conclusion: Tom: So we're wrapping up on this paper about "FDR-Controlled Variable Selection for Generalized Linear Models and Cox Regression with Virtual Dummies."
Jane: What they did is take a method that controls false discovery rate, like T-Rex, and extend it to more complex models—generalized linear models and survival analysis.
Lu: Basically, they propose a new way to select predictors called the virtual dummy construction that makes the memory usage much more efficient.
Meng: So instead of storing all those huge data matrices for every possible variable, you only store these smaller projections, which is pretty smart for production systems.
Tom: They prove that this new approach is mathematically sound and gives you the same false discovery rate control as the older explicit augmentation method, which is a big validation point.
Jane: It means researchers can use this to find important predictors in massive datasets without hitting those huge memory walls.
Lu: The extension they made allows them to do this for Bernoulli, Poisson, multinomial responses, and even Cox regression models too.
Meng: But what's the catch? They admit that for some response types, like multinomial data, you might lose a little bit of the true positive rate.
Tom: That’s a necessary caveat because you gotta know where the method stops being perfect.
Jane: It shows that this virtual dummy method can actually give higher power than just selecting variables without controlling anything, even when the underlying linear model is slightly wrong.
Lu: And they show it works well empirically, recovering more causal signals in genomic applications than many other methods we've seen.
Meng: For someone shipping models to production, the runtime comparison shows this method is actually faster than some of the established alternatives we've been testing.
Tom: So it seems like a solid way to handle high-dimensional data selection that balances computational cost with statistical control.
TU Darmstadt
stat.ME, eess.SP
Submitted: 2026-09-24
Updated: 2026-11-10
Code: https://github.com/helenamehler/virtual-dummies-glm
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 89/100
The gist: In genomics, imaging and clinical studies, only a few of many candidate predictors are often nonlinearly associated with a response that may be, e.g., binary, categorical…
Key concepts
- Virtual Dummy (VD)
- Instead of storing full vectors for each variable, VD stores only its projections onto an adaptively growing subspace. This saves significant memory while maintaining the same statistical properties as traditional explicit augmentation methods, allowing FDR control to be preserved.
- Score-Based Forward Selection
- This is a selection algorithm that ranks candidate predictors based on how well they fit the model, using a score derived from the residual (e.g., $y - ext{mean}$). The VD framework adapts this ranking by splitting each dummy into revealed and unrevealed parts computable from its projections.
- FDR Control
- FDR control aims to limit the expected proportion of false discoveries among selected variables. The VD method achieves this by letting synthetic null variables compete with real predictors, ensuring that the selection path remains equivalent in distribution to explicit methods like T-Rex.
- Generalized Linear Models (GLMs)
- GLMs are statistical models used when the response variable can be binary, count, or a censored event time. The VD framework is specifically extended to handle these diverse response types by adapting the virtual dummy construction for score ranking.
Terminology
Summary
In genomics, imaging and clinical studies, only a few of many candidate predictors are often nonlinearly associated with a response that may be, e. p. 0g., binary, categorical, a count or a censored event time <ref:2609. p. 029335#pg2> The Terminating-Random Experiments (T-Rex) selector is a scalable variable selection method that controls the false discovery rate (FDR) by letting synthetic null variables (dummies) compete with the real predictors <ref:2609. p. 029335#pg2> We propose a memory-efficient selection procedure with FDR control for generalized linear models and Cox regression by extending the recently developed virtual dummy construction to score-based forward selection for Bernoulli, Poisson, multinomial and Cox responses <ref:2609. p. 029335#pg2> The virtual-dummy-based selection path remains equal in distribution to explicit augmentation, so FDR control carries over under the same assumptions <ref:2609. p. 029335#pg2>.
Variable Selection Context
The goal of variable selection in fields like genome-wide association studies and cancer survival analysis is to identify active predictors among a large number of candidates where the response can be binary, categorical, a count or a censored event time <ref:2609.29335#pg2>. The main current paradigm involves controlling the false discovery rate (FDR), which is the expected proportion of false discoveries among the selected variables while recovering as many active predictors as possible <ref:2609. p. 029335#pg2>. Classical methods like Benjamini-Hochberg and Benjamini-Yekutieli control FDR using p-values, but these are not valid in multivariate models with more predictors than observations <ref:2609. p. 029335#pg2>. ModelX knockoffs control the FDR by creating knockoff copies for each variable from an estimated model of the predictor distribution, but this framework becomes prohibitively computationally demanding beyond a few tens of thousands of variables <ref:2609. p. 029335#pg2>.
Virtual Dummy Mechanism
The virtual dummy (VD) framework removes the memory bottleneck by leveraging the fact that many forward selection procedures do not require the n coordinates of a dummy, but only its projections onto an adaptively growing subspace <ref:2609.29335#pg2>. Specifically, each dummy dl is stored as the growing collection of scalars αil:= ⟨dl, ei⟩ instead of an n-vector <ref:2609. p. 129335#pg2>. This construction ensures that the virtual and the augmented selection path are identical in distribution, so FDR control carries over to the virtual dummy framework <ref:2609. p. 129335#pg2>.
Extension to Generalized Models
The paper extends this framework to score-based forward selection for Bernoulli, Poisson, multinomial and Cox responses by adapting the virtual dummy construction <ref:2609.29335#pg2>. The forward selection algorithm ranks candidates by the score of a maximum-likelihood (ML) fit, where the residual is replaced by the score contribution sk = y−µk, µk = g−1(ηk), ηk = b01 + XAbβ(k) <ref:2609. p. 129335#pg2>. The virtual dummy construction adapts to this score ranking by splitting each dummy into a revealed and an unrevealed part, which is computable from the αil alone <ref:2609. p. 129335#pg2>.
Empirical Validation and Performance
Simulations confirm the equivalence of the virtual dummy framework with explicit augmentation, showing that correct specification yields higher power than selection under a misspecified linear model <ref:2609.29335#pg2>. In genomic applications, VD-Multinomial recovers more than twice as many causal SNPs as any competitor within the target FDR <ref:2609. p. 329335#pg2>. The Cox instantiation selects eight genes against one for the linear model on log t <ref:2609. p. 329335#pg2>. VD-GLM attains the highest power among methods keeping the FDR below the target in disease classification tasks <ref:2609. p. 329335#pg2>.
Resource Efficiency
The memory footprint and runtime comparisons show that VD-GLM adds only around 205 MB for Poisson, Bernoulli and Cox instantiations, and is 2.2 to 2.5× faster than AD-GLM <ref:2609. p. 229335#pg2>. The absolute VD-GLM runtimes range from about 25 s to 37 s, against 58 s to 89 s for AD-GLM <ref:2609. p. 229335#pg2>. This efficiency demonstrates real-world applicability in high-dimensional settings <ref:2609. p. 029335#pg2>.
Conclusion
The proposed selector satisfies the conditions of the equivalence result, and its selection path is equal in distribution to explicit augmentation, thus inheriting the FDR-control result of the T-Rex selector <ref:2609.29335#pg2>. The loss in TPR is most pronounced for multinomial response, while for Bernoulli responses it is negligible <ref:2609. p. 329335#pg2>. Future work could include robust forward selectors such as the LAD or Huber, or other path algorithms <ref:2609. p. 329335#pg2>.
REFERENCES
[1] Abdel Abdellaoui, Loic Yengo, Karin J. H. Verweij, and Peter M. p. 4 Visscher, “15 years of GWAS discovery: Realizing the promise,” The American Journal of Human Genetics, vol. p. 4 110, no. 2, pp. 179–194, 2023
[5] Yoav Benjamini and Yosef Hochberg, “Controlling the false discovery rate: A practical and powerful approach to multiple testing,” Journal of the Royal Statistical Society: Series B (Methodological), vol. 57, no. 1, pp. 289–300, 1995
[7] Rina Foygel Barber and Emmanuel J. Candes, “Controlling the false discovery rate via knockoffs,” The Annals of Statistics, vol. p. 4 43, no. 5, pp. 2055 – 2085, 2015
[8] Emmanuel Candes, Yingying Fan, Lucas Janson, and Jinchi Lv, “Panning for gold: ‘model-X’ knockoffs for high dimensional controlled variable selection,” Journal of the Royal Statistical Society Series B: Statistical Methodology, vol. 80, no. 3, pp. 551–577, 2018
[9] Jasin Machkour, Michael Muma, and Daniel P. Palomar, “The terminating-random experiments selector: Fast highdimensional variable selection with false discovery rate control,” Signal Processing, vol. p. 4 231, pp. 109894, 2025
[10] Taulant Koka, Jasin Machkour, Daniel P. Palomar, and Michael Muma, “Virtual dummies: Enabling scalable FDR-controlled variable selection via sequential sampling of null features,” arXiv preprint arXiv:2604. p. 407464, 2026 <ref:2609. p. 429335#pg2>
[11] Yagyensh Chandra Pati, Ramin Rezaiifar, and Perinkulam Sambamurthy Krishnaprasad, “Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition,” in Proceedings of 27th Asilomar conference on signals, systems and computers. IEEE, 1993, pp. p. 4 40–44 <ref:2609. p. 029335#pg2>
[12] Ivy Zhang and Robert Tibshirani, “Adaptive forward stepwise: A method for high sparsity regression,” Journal of Machine Learning Research, vol. 27, no. 35, pp. 1–24, 2026 <ref:2609. p. 029335#pg2>
[18] Daoji Li, Jinzhao Yu, and Hui Zhao, “Coxknockoff: Controlled feature selection for the cox model using knockoffs,” Stat, vol. 12, no. 1, pp. e607, 2023 <ref:2609. p. 429335#pg2>
[19] Noah Simon, Jerome H. Friedman, Trevor Hastie, and Robert Tibshirani, “Regularization paths for Cox’s proportional hazards model via coordinate descent,” Journal of Statistical Software, vol. p. 4 39, no. 5, pp. 1–13, 2011 <ref:2609. p. 029335#pg2>
[20] Niklas Kluemper, Iulia Blajan, Doris Schmidt, Glen Kristiansen, Marieta Toma, Michael Hoelzel, Manuel Ritter, and Joerg Ellinger, “Downstream neighbor of SON (DONSON) is associated with unfavorable survival across diverse cancers with oncogenic properties in clear cell renal cell carcinoma,” Translational Oncology, vol. 13, no. 11, pp. 100844, 2020 <ref:2609. p. 429335#pg2>
[25] Fabian Scheidt and Michael Muma, “Terminating Greedy Pursuits for High-Dimensional False Discovery Rate Control,” in Proc. 34th Eur. Signal Process. p. 4 Conf. (EUSIPCO), 2026, pp. p. 4 2511–2515 <ref:2609. p. 429335#pg2>
[13] Sophie Wharrie, Zhiyu Yang, Vishnu Raj, Remo Monti, Rahul Gupta, Ying Wang, Alicia Martin, Luke J O’Connor, Samuel Kaski, Pekka Marttinen, et al., “Hapnest: efficient, large-scale generation and evaluation of synthetic datasets for genotypes and phenotypes,” Bioinformatics vol. p. 4 39 no. 9 pp. btad535,2023 <ref:2609. p. 429335#pg2>
[16] The Cancer Genome Atlas Research Network, “Comprehensive molecular characterization of clear cell renal cell carcinoma,” Nature vol. 499 no. 7456 pp. 43–49, 2013 <ref:2609. p. 029335#pg2>
[17] Mary J. Goldman, Brian Craft, Mim Hastie, Kristupas Repecka, Fran McDade, Akhil Kamath, Ayan Banerjee, Yun-hai Luo, Dave Rogers, Angela N. p. 4 Brooks, et al. p. 4, “Visualizing and interpreting cancer genomics data via the Xena platform,” Nature Biotechnology vol. p. 4 38 no. 6 pp. 675–678, 2020 <ref:2609. p. 429335#pg2>
[14] William G Cochran, “Some methods for strengthening the common χ2 tests,” Biometrics vol. 10 no. 4 pp. 417–451, 1954
[15] Peter Armitage, “Tests for linear trends in proportions and frequencies,” Biometrics vol. 11 no. 3 pp. 375–386, 1955
[4] Kenneth A Norman, Sean M Polyn, Greg J Detre, and James V Haxby, “Beyond mind-reading: Multi-voxel pattern analysis of fMRI data,” Trends in Cognitive Sciences vol. 10 no. 9 pp. 424–430, 2006 <ref:2609. p. 4
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Virtual dummy-based selection for generalized linear models and Cox regression allows for memory-efficient FDR control by storing only projections of synthetic null variables instead of full n-dimensional matrices, enabling scalable variable selection in high dimensions.
-
The proposed selector can recover more active predictors than competitors within the target false discovery rate, as shown in simulations where
VD-Multinomial recovers more than twice as many causal SNPs as any competitor within the target FDR.
-
The system can perform real-world variable selection on genomic data, such as disease subtype classification and cancer survival analysis, achieving superior performance compared to classical methods like ModelX knockoffs and marginal Cox score tests.
Sources
Related papers
- Learning Robust Treatment Rules for Censored Data
- Neural Estimation of Summary Statistics for Simulation-Based Inference with Non-Asymptotic Guarantees
- Spectral divide-and-conquer MCMC for long stationary time series
- Improving the Efficiency of Subgroup Analysis in Randomized Controlled Trials with TMLE
- Personalized Federated Vector Autoregression with Personalization Diversity
- Doubly Robust and Efficient Calibration of Prediction Sets for Right-Censored Time-to-Event Outcomes