Selective Inference for Survey-Weighted Generalized Linear Models: Growing Dimension and Average Marginal Effects
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.
Ruth: We're looking at the paper: "Selective Inference for Survey-Weighted Generalized Linear Models".
Warren: Here's the gist of the paper. This paper extends three Lasso inferential methods to a survey environment and establishes their asymptotic validity in generalized linear models with survey weights and/or heteroskedasticity <ref:2304.07855#pg2>.
Ruth: Moreover, it generalizes these methods to inference on nonlinear parameter functions such as the average marginal effect in survey logit models <ref:2304.07855#pg2>.
Warren: Model Framework and Extensions The framework considers a GLM specifying the conditional density of a scalar outcome variable given covariates with survey weights <ref:2304.07855#pg3>.
Ruth: The weighted log-likelihood function is defined as L(θ) ≡ −n−1 Xn i=1 wig(yi, x0iθ) <ref:2304.07855#pg4>.
Title and authors: Iris: So, to summarize this paper, the authors are extending these three Lasso inferential methods to handle survey weights and heteroskedasticity while also generalizing them to inference on average marginal effects in logit models.
Ruth: They’re essentially taking tools that work for simpler linear models and adapting them so they can handle the messier reality of survey data, which includes those survey weights and the varying variance across groups.
Warren: What's important is that they establish asymptotic validity for these procedures in generalized linear models with survey weights or heteroskedasticity, and they prove it even when you’re looking at non-linear things like average marginal effects.
Iris: That means we’re not just talking about simple coefficients anymore; we are now looking at how a change in one variable actually changes the effect on the outcome, which is what economists care about in many fields.
Ruth: They lay out the whole mathematical setup first, defining that weighted log-likelihood function and then building that selection event based on minimizing the likelihood penalized by those slope parameters.
Warren: So, this selection event is where selective inference comes in; it defines exactly when we can actually make an inference, meaning when the selection event happens and the sign of those selected coefficients is fixed.
Iris: The simulation evidence shows they hold up reasonably well with moderate numbers of variables, but they admit that if you have a p divided by n ratio of point five, the C alpha test can show some size distortion.
Ruth: And as that ratio gets bigger, the tsvy test deteriorates too. So while these methods are robust, there are clear limits on how big the model can get relative to your sample size before things start getting distorted.
Warren: It means we still have a way to reliably estimate average marginal effects in survey models without getting lost in the complexity of traditional methods, even with those weights or non-linear relationships.
Iris: So they’re providing a way to do this reliably, which is important because it bridges the gap where most high-dimensional tools were focused on simpler linear setups.
Ruth: That bridge is useful for policy evaluation where understanding how employment status affects online service use matters, for instance.
The paper's summary: Warren: Now let's talk about what they suggest as improvements in this paper, because the authors aren't just presenting a finished product; they are showing how to tighten up the constraints on things like the loss function.
Iris: They’re focusing on making sure that when we move to those more complex survey sampling schemes, like stratified sampling or exogenous stratification, the methods still maintain their accuracy under those conditions.
Ruth: They are explicitly addressing how the debiased Lasso estimator behaves when you introduce those extra complexities in the sampling design. They show that its convergence rate, that big O notation, is still behaving as expected even with these survey adjustments.
Warren: For me, it’s about how the computational cost scales. They’re showing that for a growing number of regressors, like when p divided by n increases, the overall strategy stays stable and doesn't explode computationally.
Iris: What I find interesting is how they build the selective inference test statistic itself. They construct a specific test statistic that works conditionally on both the selection event and the sign of those selected coefficients, which is very specific to dealing with survey weights and heteroskedasticity in standard models.
Ruth: That level of specificity is what makes it useful for economists because it’s not just giving you a general result; it’s telling you exactly how to test a specific effect based on the data structure you actually have.
Warren: It means when we are running these strategies, we can be more confident in our results because the test statistic is tailored precisely to the noise structure of the survey data. That reduces what I would call estimation risk.
Iris: So, in short, they’re improving it by showing exactly how to verify those specific identification conditions and by providing a precise test statistic for those tricky heteroskedastic settings.
Ruth: And this leads us to the next part of the story: they did show some limitations too. They admit that the C alpha test can get distorted in certain situations, like when p divided by n is point five, and that’s something we need to keep in mind when applying it.
Warren: So it’s not perfect for every single setup, but these improvements show they are robust enough for most things we encounter with survey data. It's about finding the sweet spot between a strong statistical tool and real-world applicability.
The paper's improvements: Iris: So to wrap up this paper, the main point is that these extended inference methods can reliably handle survey weights and heteroskedasticity while still giving us reliable estimates for non-linear effects like average marginal effects in logit models.
Ruth: Exactly. We’re talking about making sure that when we use these high-dimensional methods, they actually work on real survey data, not just perfectly randomized data.
Warren: It's about getting valid inferences even when you have imperfect sampling information or when the variance of the outcomes changes across different groups.
Iris: And it’s important to remember that they aren't just looking at simple linear coefficients anymore; they are generalizing these methods to average marginal effects, which is what we actually care about in social science applications.
Ruth: Right, so this bridges the gap where most high-dimensional tools were focused on simpler linear setups. They show these methods still hold up when you introduce survey complexities and those more complex non-linear effects we see in logit models.
Warren: The paper lays out the whole setup first, then shows how they extend it to solve those problems using the weighted log-likelihood function and defining that selection event based on minimizing the likelihood penalized by the slope parameters.
Iris: And that selection event is what makes selective inference possible; it defines exactly when we can make an inference, meaning when the selection event happens and the sign of those selected coefficients is fixed.
Ruth: The simulation evidence shows they hold up reasonably well when you have a moderate number of variables, but they do flag a problem: if you have more predictors than observations, say p divided by n is point five, the C alpha test can show some size distortion.
Warren: And as the number of predictors grows larger, when p divided by n gets bigger, they noted that the tsvy test deteriorates too. So these methods are robust but have limits on how big the model can get relative to your sample size before things get distorted.
Iris: That means we still have a way to reliably estimate average marginal effects in survey models without getting lost in the complexity of traditional methods, even with those weights or non-linear relationships.
Ruth: It gives us a concrete tool for policy evaluation where understanding how employment status affects online service use matters.
Ruth: So, we can now get reliable estimates of average marginal effects in survey data under much more realistic conditions than before.
Conclusion: Ruth: So we’ve looked at how they're taking three high-dimensional tools—Debiased Lasso, C alpha, and Selective Inference—and making them work with survey data and non-linear effects like average marginal effects.
Warren: They showed these methods are valid even when you have those survey weights or when the data has heteroskedasticity. It’s about getting a solid statistical inference for those coefficients in generalized linear models and also for non-linear things like average marginal effects in logit models.
Iris: The key part is generalizing it to those non-linear functions, the average marginal effects. That means they aren't just looking at simple linear relationships anymore; they're checking how a change in a variable actually changes that effect on the outcome.
Ruth: Exactly, and what’s interesting is how they prove this works under specific assumptions about the loss function. They use things like sub-Gaussian norms and concentration inequalities to back up their claims about asymptotic validity.
Warren: The methodology involves defining the weighted log-likelihood function, L of theta, using those survey weights. Then they define a score function and sample information matrices built using those weights too. It’s a whole pipeline for setting up the selection event based on minimizing that likelihood penalized by the slope parameters.
Iris: That selection event is crucial because it defines when M hat equals M and s hat equals s M. They test the asymptotic distribution of those average marginal effects conditional on that specific event happening.
Ruth: The simulation evidence shows they hold up reasonably well when the number of variables is moderate. But they flag a problem: if you have more predictors than observations, say p divided by n is point five, the C alpha test shows some size distortion.
Warren: And as p divided by n gets larger, that tsvy test deteriorates too. So while these methods are robust, there are clear limits on how big the model can get relative to your sample size before things start getting distorted.
Iris: So what this means for us is that we have a way to reliably estimate average marginal effects in survey models without getting bogged down by the complexities of traditional methods, even when you’re dealing with survey weights or non-linear relationships.
Ruth: It gives us a concrete tool for policy evaluation and social science research where understanding those complex effects matters, like looking at how employment status affects online service use.
Warren: And the empirical test on Canadian Internet Use Survey twenty twenty data showed that variables with the largest average marginal effects were things like whether someone was employed or whether they had a university degree.
Iris: That’s a measurable finding, showing exactly which factors have the biggest impact on that outcome in real data. It moves this from just theory to something applicable.
Ruth: So, the big picture here is that we can now get these reliable estimates of average marginal effects in survey data under much more realistic conditions than before.
Warren: This sets a new standard for how quantitative methods should approach complex social science data with sampling noise; it's about building tools that don't break when the real world gets messy.
Iris: It certainly opens the door for more rigorous analysis of complex survey designs, moving beyond simpler assumptions about the data structure.
Ruth: That’s what we have with "Selective Inference for Survey-Weighted Generalized Linear Models: Growing Dimension and Average Marginal Effects." We’ll be keeping an eye on how these tools get applied in the next few years.
Warren: And if you want to see how these tools interact with other ideas, we're looking at a paper that talks about learning from the size of surprise—how investors learn about earnings surprises from just the size of the surprise itself.
Joann Jasiak, Purevdorj Tuvaandorj
York University
econ.EM, math.ST, stat.ML, stat.TH
Submitted: 2023-04-16
Updated: 2026-11-15
Comments: 54 pages, 4 tables
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 83/100
The gist: This paper extends three Lasso inferential methods to a survey environment and establishes their asymptotic validity in generalized linear models with survey weights and/or heteroskedasticity…
Key concepts
- Survey-Weighted GLM
- This is a statistical model used when data comes from a survey, where each respondent has a specific weight reflecting their sampling probability. The model describes the relationship between outcomes and predictors using generalized linear models, accounting for these weights to ensure accurate estimates.
- Selective Inference (SI)
- SI is an inference method designed to test hypotheses about specific parameters after a variable selection event occurs. In this context, it helps determine the asymptotic distribution of an estimator conditional on which variables were selected in the model.
- Average Marginal Effect (AME)
- The AME measures how much a specific predictor affects the outcome across different levels of another predictor. For example, it tells you how much using government online services changes for employed people versus unemployed people, providing insight into causal effects.
- Asymptotic Validity
- This means that as the sample size gets very large, the statistical methods used (like C(α) or DB Lasso) will converge to their theoretical correct distributions. The paper establishes this validity under specific assumptions related to the loss function and concentration inequalities.
Terminology
Summary
This paper extends three Lasso inferential methods to a survey environment and establishes their asymptotic validity in generalized linear models with survey weights and/or heteroskedasticity <ref:2304. p. 007855#pg2>. Moreover, it generalizes these methods to inference on nonlinear parameter functions such as the average marginal effect in survey logit models <ref:2304. p. 0 p. 0 p. 007855#pg2>.
Model Framework and Extensions
The framework considers a GLM specifying the conditional density of a scalar outcome variable given covariates with survey weights <ref:2304.07855#pg3>. The weighted log-likelihood function is defined as L(θ) ≡ −n−1 Xn i=1 wig(yi, x0iθ) <ref:2304. p. 307855#pg4>. The model involves a survey-weighted Lasso where the negative of the weighted log-likelihood function is minimized subject to a penalty on the slope parameters <ref:2304. p. 507855#pg7>. The selection event Mˆ = M, sˆMˆ = sM holds if there exist random vectors ˆθM ∈ RM and u ∈ R p+1−M in the Karush-Kuhn-Tucker condition for the problem <ref:2304. p. 607855#pg8>.
Inference Methods and Validity
The paper extends C(α), Debiased Lasso (DB), and Selective Inference (SI) methods to accommodate survey weights and/or heteroskedasticity <ref:2304.07855#pg9>. The asymptotic validity of these three methods is established for inference on nonlinear parameter functions like average marginal effects in a survey logit model <ref:2304. p. 207855#pg10>. Specifically, the SI method involves a one-step estimator ˜θM ≡ ˆθM + HˆM(ˆθM)−1S(ˆθM) and tests the asymptotic distribution of β˜M conditional on the selection event Mˆ = M and ˆsM̂ = sM. p. 8
Simulation Evidence
The effectiveness of the proposed approach is illustrated through simulation evidence and an empirical application to Canadian Internet Use Survey 2020 data <ref:2304.07855#pg10>. In simulations, tests show reasonable size control when the number of regressors is moderate, but the C(α) test exhibits a substantial size distortion when p/n = 0. p. 25 and the tsvy test deteriorates as p/n grows. p. 8
Empirical Application Results
The empirical application to CIUS 2020 data confirms the relevance of the proposed approach, showing that variables with the largest (in absolute value) AMEs on whether a person uses government online services are whether or not a person is employed, whether or not a person is single, and if their educational attainment was a High school or less or a University degree. The results indicate that the employed are about 8-11 percentage points more likely to use the government online service than those not employed. p. 6
Conclusion
The paper concludes that the proposed extended inference methods are reliable when p/n increases, as illustrated by simulation and empirical studies <ref:2304.07855#pg10>. The results confirm the relevance of this approach in analyzing survey data, such as the CIUS 2020 data. p. 0 The proposed extended inference methods were applied to the logit model and remain reliable when p/n increases as illustrated in a simulation study with standard stratified sampling and exogenous stratification <ref:2304. p. 207855#pg10>. The empirical illustration based on the CIUS 2020 data also confirms the relevance of the proposed approach. p. 2 The paper has provided two main results. p. 1
Pruning redundant reasoning tokens
The authors extend three Lasso inferential methods to a survey environment and establish their asymptotic validity in generalized linear models with survey weights and/or heteroskedasticity <ref:2304.07855#pg2>. Moreover, it generalizes these methods to inference on nonlinear parameter functions such as the average marginal effect in survey logit models <ref:2304.07855#pg2>.
How the training signal is built
The framework involves defining the weighted log-likelihood function L(θ) ≡ −n−1 Xn i=1 wig(yi, x0iθ) <ref:2304.07855#pg4>. The score function S(θ) ≡ ∂L(θ)/∂θ is defined as −n−1 Xn i=1 wixig˙(yi, x0iθ) <ref:2304. p. 507855#pg7>. The sample information and negative Hessian matrices are defined as ˆI(θ) ≡ n−1 Xn i=1 w 2i xix0i g˙(yi, x0iθ) squared <ref:2304. p. 407855#pg8> and Hˆ (θ) ≡ −∂2L(θ)/∂θ∂θ0 = n−1 Xn i=1 wixix0i g¨(yi, x0iθ) <ref:2304. p. 507855#pg8>.
Where the gains come from
The asymptotic validity of the inference procedures is established under Assumption 1, which imposes conditions on the loss function g(y, t). The proof relies on verifying assumptions related to sub-Gaussian norms and concentration inequalities. p. 2 The asymptotic validity of the SI procedure is established by verifying conditions given in Kuchibhotla et al. p. 4 (2022). The asymptotic validity of the DB method relies on Lemma 3. p. 05 which shows that the rate of convergence of the Hessian and information matrices is Op(m0λ). p. 4
The paper extends three Lasso inferential methods to a survey environment and establishes their asymptotic validity in generalized linear models with survey weights and/or heteroskedasticity <ref:2304.07855#pg2>. Moreover, it generalizes these methods to inference on nonlinear parameter functions such as the average marginal effect in survey logit models <ref:2304.07855#pg2>.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Extended Lasso inference to survey data with weights or heteroskedasticity allows for valid inference on model coefficients in generalized linear models and nonlinear parameter functions like average marginal effects in survey logit models.
-
The proposed methods provide asymptotic validity for debiased Lasso, C(alpha), and selective inference procedures under assumptions that accommodate the complexities of survey sampling schemes like standard stratified sampling and exogenous stratification.
-
A specific selective inference test statistic can be constructed conditional on the selection event M hat equals M and the sign of the selected coefficients being s hat equals s M, addressing challenges arising from survey weights and heteroskedasticity in homoskedastic GLMs.
-
The debiased Lasso estimator for average marginal effects has been shown to have an asymptotic normal distribution with a specific covariance matrix, enabling the construction of confidence intervals for AMEs in survey logit models.
-
The C(alpha) test statistic can be used to test the hypothesis that a specific average marginal effect is zero by solving for an auxiliary estimate that satisfies the constraint AME j equals 0, leading to an asymptotic chi-squared distribution.
Sources
- A Short Note on Concentration Inequalities for Random Vectors with SubGaussian Norm
- Unifying approach to selective inference with applications to cross-validation
- Introduction to the non-asymptotic analysis of random matrices
Related papers
- Stationarity with Occasionally Binding Constraints
- Backtesting High-Dimensional Expected Shortfall
- Higher-Order Neyman Orthogonality in Moment-Condition Models
- Online Bayesian Inference for a Dirichlet Process Mixture of Binary Logit Models
- Debiased Conditional Independence Tests for Causal Effects
- Sharp Testable Implications of Encouragement Designs