Choosing the Dictionary and Penalty for IV-LASSO
summary
The gist
Estimating the first stage of an instrumental variables (IV) model with the least absolute shrinkage and selection operator (LASSO) requires choosing a dictionary of technical instruments and a…
In short
The paper addresses how to choose the best first-stage implementation for an instrumental variables LASSO estimator when many instruments are available, as standard theory offers no guidance. It develops a principled method using second-order asymptotic theory and an Approximate Mean Squared Error (AMSE) criterion to rank candidate first-stage fits. This criterion selects the fit that minimizes AMSE, balancing bias and variance based on the endogeneity of the regressor.
Key concepts
- Instrumental Variables (IV) LASSO
- This is a statistical method used to estimate structural parameters when there is an endogenous variable. It combines instrumental variables with a LASSO penalty to select relevant predictors, aiming for a more accurate parameter estimate than standard methods.
- Approximate Mean Squared Error (AMSE)
- AMSE is a metric used to compare different first-stage models. It estimates the expected squared error of the final IV-LASSO estimator. Minimizing AMSE helps select a first-stage fit that provides the best trade-off between fitting the data well and remaining consistent with structural errors.
- Endogeneity
- Endogeneity refers to a situation where an explanatory variable in a regression model is correlated with the error term. The paper argues that this level of endogeneity significantly impacts which first-stage implementation is optimal, as more endogeneity increases the complexity cost of a first-stage fit.
- Second-Order Asymptotic Theory
- This advanced mathematical framework is used to derive a criterion for selecting the best first-stage model. It goes beyond simple asymptotic theory by analyzing second-order terms in the mean squared error expansion, allowing for a more precise ranking of candidate models.
Terminology used across episodes
This episode discusses
The paper
Choosing the Dictionary and Penalty for IV-LASSO · Read on arXiv
University of Rochester · Texas A&M University · University of California, Los Angeles
Estimating an instrumental variables (IV) first stage with the least absolute shrinkage and selection operator (LASSO) requires choosing a dictionary and penalty. Existing rules target first-stage objectives, although these choices affect structural estimation. We derive a feasible approximate mean squared error (AMSE) criterion for full-sample IV-LASSO with one endogenous regressor and homoskedastic Gaussian errors. For a prespecified list of dictionary-penalty pairs, the criterion trades off approximation of the optimal instrument against a complexity cost that depends on the first two moments of the LASSO effective dimension and on the degree of endogeneity. The selected pair attains the smallest candidate-specific AMSE component up to vanishing relative error. In simulations calibrated to Paravisini et al. (2015), the criterion improves substantially on the plug-in rule at low endogeneity and on cross-validation at high endogeneity, with nearly identical results under non-Gaussian errors. A supplementary appendix formally extends analysis to a class of non-Gaussian errors.
Transcript
Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.
Ruth: I'm Ruth.
Warren: And I'm Warren, here with Iris.
Ruth: We're looking at the paper: "Choosing the Dictionary and Penalty for IV-LASSO".
Warren: Here's the gist of the paper. Estimating the first stage of an instrumental variables (IV) model with the least absolute shrinkage and selection operator (LASSO) requires choosing a dictionary of technical instruments and a penalty level.
Ruth: First-order asymptotic theory offers no guidance on these choices, as any consistent implementation yields a structural parameter estimator with the same limiting distribution.
Warren: In finite samples, however, these choices can have a substantial impact on the resulting structural parameter estimate.
Ruth: Working in a model with a single endogenous regressor and homoskedastic Gaussian errors, we use first- and secondorder Stein identities to derive the approximate mean squared error (AMSE) of the instrumental-variables LASSO (IV-LASSO) estimator.
Paper summary: Ruth: We talked about how standard theory fails us here, and this paper proposes using second-order asymptotic theory to find a way around that limitation >
Warren: They focus on the leading case of a single endogenous regressor and work specifically in a homoskedastic Gaussian model when deriving the approximate mean squared error, AMSE >
Iris: They introduce this criterion by decomposing the candidate specific part of the AMSE into two components: a first-stage approximation cost and a complexity cost >
Ruth: That complexity cost is measured by a collinearity-robust analogue of the number of instruments selected by the LASSO, and it grows with the squared covariance between those first stage and structural errors >
Warren: They then characterize this as a feasible criterion that you can actually construct from each candidate's fitted first stage >
Iris: Minimizing this specific criterion selects an estimator whose AMSE is as small as that of the best candidate in your list, up to vanishing relative error, which is what they want >
Ruth: It’s about ranking a finite list of dictionary penalty candidates by their expected performance before you even run the main estimation >
Warren: And they show this works when you estimate those structural error moments, sigma two epsilon and sigma two ev, using residuals from a pilot candidate > <ref:2609.07033#pg2>
Iris: The theoretical guarantee is that this feasible criterion can be consistently estimated, and using it lets you pick the optimal first-stage implementation for your IV-LASSO estimator >
Ruth: It’s about moving from ad hoc tuning rules to a method where you systematically rank implementations based on how well they control the bias versus the variance trade-off >
Warren: So, before we move on, this all hinges on their derivation of that AMSE criterion through first- and second-order Stein identities >
Iris: That’s what makes it rigorous. It's not just a guess; it's derived from higher-order theory to give you a concrete way to choose between the candidates >
Ruth: Moving into the conclusion, we need to see what this all translates into for applied work and how it fits into the broader landscape of IV estimation >
Conclusion: Ruth: So, looking at "Choosing the Dictionary and Penalty for IV-LASSO," the authors are essentially saying that while first-order theory doesn't guide you on which technical instrument setup to use, they developed a principled method based on AMSE ranking >
Warren: The core implication is that for finite samples, this AMSE criterion can actually help you select an estimator whose performance is close to the best one in your tested list >
Iris: It means you stop guessing and start using a systematic way to choose between first-stage implementations by analyzing how they handle the bias versus variance trade-off based on endogeneity >
Ruth: The title itself suggests that they are providing a method for choosing both the dictionary and the penalty simultaneously, which is what their method does by minimizing that combined criterion >
Warren: In simple terms, this means if you have many options, you don't have to test them all at once; you can use this criterion to narrow down your choices based on theoretical performance metrics >
Iris: The limitation they state is that the derivation of the AMSE ranking relies heavily on conditional Gaussianity and homoskedasticity assumptions for those results >
Ruth: So, while it's a powerful tool, you have to be careful about applying it outside of those specific model assumptions because the AMSE ranking is derived under those conditions >
Warren: It’s a useful way to get a good result when you are working in that Gaussian setting, but if your errors aren't Gaussian, you need to check if this method still gives you the same performance gains >
More episodes
- Daily Summary for 2026-11-19
- 2608.21274-Recommendation Quality and the Concentration of Consumption: Experimental Evidence from Netflix
- 2408.02757-A nonparametric test for diurnal variation in spot correlation processes
- 2306.16393-High-Dimensional Canonical Correlation Analysis
- 2605.17299-Geometric Brownian motion with intermittent entries and exits
- 2607.04753-Fooling Yourself: how narratives shape beliefs
- 2408.14671-Inference on Treatment Effects with Repeatedly Mismeasured High-Dimensional Controls
- 2201.06898-Difference-in-Differences Estimators for Treatments Continuously Distributed at Every Period
- 2501.15307-Influence Functions: A Unified Differentiation-Based Framework
- The day in conversation for 2026-11-19