Choosing the Dictionary and Penalty for IV-LASSO

arXiv:2609.07033 · econ.EM, stat.AP · Submitted 2026-09-07 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.

Ruth: I'm Ruth.

Warren: And I'm Warren, here with Iris.

Ruth: We're looking at the paper: "Choosing the Dictionary and Penalty for IV-LASSO".

Warren: Here's the gist of the paper. Estimating the first stage of an instrumental variables (IV) model with the least absolute shrinkage and selection operator (LASSO) requires choosing a dictionary of technical instruments and a penalty level.

Ruth: First-order asymptotic theory offers no guidance on these choices, as any consistent implementation yields a structural parameter estimator with the same limiting distribution.

Warren: In finite samples, however, these choices can have a substantial impact on the resulting structural parameter estimate.

Ruth: Working in a model with a single endogenous regressor and homoskedastic Gaussian errors, we use first- and secondorder Stein identities to derive the approximate mean squared error (AMSE) of the instrumental-variables LASSO (IV-LASSO) estimator.

Paper summary: Ruth: We talked about how standard theory fails us here, and this paper proposes using second-order asymptotic theory to find a way around that limitation >

Warren: They focus on the leading case of a single endogenous regressor and work specifically in a homoskedastic Gaussian model when deriving the approximate mean squared error, AMSE >

Iris: They introduce this criterion by decomposing the candidate specific part of the AMSE into two components: a first-stage approximation cost and a complexity cost >

Ruth: That complexity cost is measured by a collinearity-robust analogue of the number of instruments selected by the LASSO, and it grows with the squared covariance between those first stage and structural errors >

Warren: They then characterize this as a feasible criterion that you can actually construct from each candidate's fitted first stage >

Iris: Minimizing this specific criterion selects an estimator whose AMSE is as small as that of the best candidate in your list, up to vanishing relative error, which is what they want >

Ruth: It’s about ranking a finite list of dictionary penalty candidates by their expected performance before you even run the main estimation >

Warren: And they show this works when you estimate those structural error moments, sigma two epsilon and sigma two ev, using residuals from a pilot candidate > <ref:2609.07033#pg2>

Iris: The theoretical guarantee is that this feasible criterion can be consistently estimated, and using it lets you pick the optimal first-stage implementation for your IV-LASSO estimator >

Ruth: It’s about moving from ad hoc tuning rules to a method where you systematically rank implementations based on how well they control the bias versus the variance trade-off >

Warren: So, before we move on, this all hinges on their derivation of that AMSE criterion through first- and second-order Stein identities >

Iris: That’s what makes it rigorous. It's not just a guess; it's derived from higher-order theory to give you a concrete way to choose between the candidates >

Ruth: Moving into the conclusion, we need to see what this all translates into for applied work and how it fits into the broader landscape of IV estimation >

Conclusion: Ruth: So, looking at "Choosing the Dictionary and Penalty for IV-LASSO," the authors are essentially saying that while first-order theory doesn't guide you on which technical instrument setup to use, they developed a principled method based on AMSE ranking >

Warren: The core implication is that for finite samples, this AMSE criterion can actually help you select an estimator whose performance is close to the best one in your tested list >

Iris: It means you stop guessing and start using a systematic way to choose between first-stage implementations by analyzing how they handle the bias versus variance trade-off based on endogeneity >

Ruth: The title itself suggests that they are providing a method for choosing both the dictionary and the penalty simultaneously, which is what their method does by minimizing that combined criterion >

Warren: In simple terms, this means if you have many options, you don't have to test them all at once; you can use this criterion to narrow down your choices based on theoretical performance metrics >

Iris: The limitation they state is that the derivation of the AMSE ranking relies heavily on conditional Gaussianity and homoskedasticity assumptions for those results >

Ruth: So, while it's a powerful tool, you have to be careful about applying it outside of those specific model assumptions because the AMSE ranking is derived under those conditions >

Warren: It’s a useful way to get a good result when you are working in that Gaussian setting, but if your errors aren't Gaussian, you need to check if this method still gives you the same performance gains >

University of Rochester · Texas A&M University · University of California, Los Angeles

econ.EM, stat.AP

Submitted: 2026-09-07

Updated: 2026-11-09

Comments: 83 pages, 2 figures; revised exposition and simulations; includes full proofs, a formal treatment of included controls, and a supplemental appendix on non-Gaussian errors

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 91/100

The gist: Estimating the first stage of an instrumental variables (IV) model with the least absolute shrinkage and selection operator (LASSO) requires choosing a dictionary of technical instruments and a…

Key concepts

Instrumental Variables (IV) LASSO
This is a statistical method used to estimate structural parameters when there is an endogenous variable. It combines instrumental variables with a LASSO penalty to select relevant predictors, aiming for a more accurate parameter estimate than standard methods.
Approximate Mean Squared Error (AMSE)
AMSE is a metric used to compare different first-stage models. It estimates the expected squared error of the final IV-LASSO estimator. Minimizing AMSE helps select a first-stage fit that provides the best trade-off between fitting the data well and remaining consistent with structural errors.
Endogeneity
Endogeneity refers to a situation where an explanatory variable in a regression model is correlated with the error term. The paper argues that this level of endogeneity significantly impacts which first-stage implementation is optimal, as more endogeneity increases the complexity cost of a first-stage fit.
Second-Order Asymptotic Theory
This advanced mathematical framework is used to derive a criterion for selecting the best first-stage model. It goes beyond simple asymptotic theory by analyzing second-order terms in the mean squared error expansion, allowing for a more precise ranking of candidate models.

Terminology

Summary

Estimating the first stage of an instrumental variables (IV) model with the least absolute shrinkage and selection operator (LASSO) requires choosing a dictionary of technical instruments and a penalty level. p. 0

Where the paper says it

edu Bohdan Salahub University of California, Los Angeles bsalahub@g. edu September 9, 2026 Abstract Estimating the first stage of an instrumental variables (IV) model with the least absolute shrinkage and selection operator (LASSO) requires choosing a dictionary of technical instruments and a penalty level.

First-order asymptotic theory offers no guidance on these choices, as any consistent implementation yields a structural parameter estimator with the same limiting distribution. p. 0
Where the paper says it

edu September 9, 2026 Abstract Estimating the first stage of an instrumental variables (IV) model with the least absolute shrinkage and selection operator (LASSO) requires choosing a dictionary of technical instruments and a penalty level. First-order asymptotic theory offers no guidance on these choices, as any consistent implementation yields a structural parameter estimator with the same limiting distribution.

In finite samples, however, these choices can have a substantial impact on the resulting structural parameter estimate. p. 0
Where the paper says it

First-order asymptotic theory offers no guidance on these choices, as any consistent implementation yields a structural parameter estimator with the same limiting distribution. In finite samples, however, these choices can have a substantial impact on the resulting structural parameter estimate.

Working in a model with a single endogenous regressor and homoskedastic Gaussian errors, we use first- and secondorder Stein identities to derive the approximate mean squared error (AMSE) of the instrumental-variables LASSO (IV-LASSO) estimator. p. 0
Where the paper says it

In finite samples, however, these choices can have a substantial impact on the resulting structural parameter estimate. Working in a model with a single endogenous regressor and homoskedastic Gaussian errors, we use first- and secondorder Stein identities to derive the approximate mean squared error (AMSE) of the instrumental-variables LASSO (IV-LASSO) estimator, which can be consistently estimated and used to rank a prespecified list of dictionary–penalty candidates.

The AMSE reveals a bias–variance trade-off: more complex first-stage fits better approximate the conditional mean of the endogenous variable but are also more correlated with the structural errors, with complexity measured by the degrees of freedom of the LASSO fit. p. 0
Where the paper says it

Working in a model with a single endogenous regressor and homoskedastic Gaussian errors, we use first- and secondorder Stein identities to derive the approximate mean squared error (AMSE) of the instrumental-variables LASSO (IV-LASSO) estimator, which can be consistently estimated and used to rank a prespecified list of dictionary–penalty candidates. The AMSE reveals a bias–variance trade-off: more complex first-stage fits better approximate the conditional mean of the endogenous variable but are also more correlated with the structural errors, with complexity measured by the degrees of freedom of the LASSO fit.

The weight on this bias rises with the endogeneity of the regressor, a quantity that neither plug-in nor crossvalidation penalty rules take into account. p. 0
Where the paper says it

The AMSE reveals a bias–variance trade-off: more complex first-stage fits better approximate the conditional mean of the endogenous variable but are also more correlated with the structural errors, with complexity measured by the degrees of freedom of the LASSO fit. The weight on this bias rises with the endogeneity of the regressor, a quantity that neither plug-in nor crossvalidation penalty rules take into account.

Despite the AMSE being derived in a Gaussian model, penalty selection by minimizing the feasible AMSE criterion delivers up to a one-third lower mean squared error compared to cross-validation and plug-in penalty rules in Gaussian and non-Gaussian simulation designs calibrated to the data of Gilchrist and Sands (2016). p. 0
Where the paper says it

The weight on this bias rises with the endogeneity of the regressor, a quantity that neither plug-in nor crossvalidation penalty rules take into account. Despite the AMSE being derived in a Gaussian model, penalty selection by minimizing the feasible AMSE criterion delivers up to a one-third lower mean squared error compared to cross-validation and plug-in penalty rules in Gaussian and non-Gaussian simulation designs calibrated to the data of Gilchrist and Sands (2016).

Introduction to the Problem

The paper addresses the difficulty applied researchers face when choosing among possible first-stage implementations for IV estimators when a large set of technical instruments is available. Standard asymptotic theory therefore does not rank admissible implementations, even though their finite-sample performance can differ. p. 1

Where the paper says it

Introduction Page 2 accurate LASSO fits yield IV estimators with the same limiting distribution. Standard asymptotic theory therefore does not rank admissible implementations, even though their finite-sample performance can differ.

Figure 1 illustrates this issue in the eminent-domain application of Belloni et al. p. 1
Where the paper says it

Standard asymptotic theory therefore does not rank admissible implementations, even though their finite-sample performance can differ. Figure 1 illustrates this issue in the eminent-domain application of Belloni et al.

(2012) Existing first-stage tuning rules are not designed to maximize the precision of resulting structural parameter estimates. p. 1
Where the paper says it

The panels report the two outcomes for which that interval excludes zero (183 Case–Shiller and 110 non-metro circuit-year observations). Existing first-stage tuning rules are not designed to maximize the precision of resulting structural parameter estimates.

The optimal first-stage implementation therefore depends not only on goodness of first-stage fit and instrument strength, but also on the level of endogeneity. p. 1
Where the paper says it

When the first-stage and structural errors are correlated, this in turn introduces bias into the structural estimate. The optimal firststage implementation therefore depends not only on goodness of first-stage fit and instrument strength, but also on the level of endogeneity.

Developing a Principled Criterion

This paper develops a principled method for choosing the first-stage implementation by developing second-order asymptotic theory for the full-sample IV-LASSO estimator. We focus on the leading case of a single endogenous regressor and work in a homoskedastic Gaussian model. p. 2

Where the paper says it

Introduction Page 3 order asymptotic theory for the full-sample IV-LASSO estimator. We focus on the leading case of a single endogenous regressor and work in a homoskedastic Gaussian model.

Following the higher-order tradition of Nagar (1959) and Donald and Newey (2001), we prespecify a finite list of first-stage LASSO dictionary–penalty candidates and rank them by the approximate mean squared error (AMSE) of their resulting IV-LASSO estimators. p. 2
Where the paper says it

We focus on the leading case of a single endogenous regressor and work in a homoskedastic Gaussian model. Following the higher-order tradition of Nagar (1959) and Donald and Newey (2001), we prespecify a finite list of first-stage LASSO dictionary–penalty candidates and rank them by the approximate mean squared error (AMSE) of their resulting IV-LASSO estimators.

The size of this complexity cost rises with the squared covariance between the first-stage and structural errors, reflecting the discussion above. p. 2
Where the paper says it

The candidate specific part of the AMSE can be decomposed into a first-stage approximation cost and a complexity cost, with complexity measured by a collinearity-robust analogue of the number of instruments selected by the LASSO. The size of this complexity cost rises with the squared covariance between the first-stage and structural errors, reflecting the discussion above.

The AMSE Criterion

We define a criterion that can be constructed from each candidate’s fitted first-stage. This criterion is designed to rank these candidates by analyzing second-order terms in a mean-squared error expansion of the resulting second-stage estimator. p. 4

Where the paper says it

, the choice of instrument basis, and a penalty parameter. The criterion developed in Section 3 ranks these candidate dictionary–penalty pairs by analyzing second-order terms in a mean-squared-error expansion of the resulting second-stage estimator.

Minimizing this criterion selects an estimator whose AMSE is as small as that of the best candidate in the list, up to vanishing relative error. p. 2
Where the paper says it

Our characterization yields a feasible criterion that can be constructed from each candidate’s fitted first-stage. Minimizing this criterion selects an estimator whose AMSE is as small as that of the best candidate in the list, up to vanishing relative error.

Feasible Selection

We construct an observable criterion that estimates it up to a candidate-independent centering term and therefore preserves its ranking asymptotically. The only additional inputs not obtained from the candidate fits are the structural error moments σ2ε and σev. p. 0

Where the paper says it

Working in a model with a single endogenous regressor and homoskedastic Gaussian errors, we use first- and secondorder Stein identities to derive the approximate mean squared error (AMSE) of the instrumental-variables LASSO (IV-LASSO) estimator, which can be consistently estimated and used to rank a prespecified list of dictionary–penalty candidates. The AMSE reveals a bias–variance trade-off: more complex first-stage fits better approximate the conditional mean of the endogenous variable but are also more correlated with the structural errors, with complexity measured by the degrees of freedom of the LASSO fit.

We estimate these moments using residuals from a pilot candidate. p. 9
Where the paper says it

The ratio Ac/h2 c appearing in the infeasible criterion below is invariant to rescaling of the fitted instrument. The closed-form approximation to the expectation of the candidate-specific term combines these two quantities with certain error moments and moments of the effective dimension of the LASSO fit defined in Section 2.

Theoretical Guarantees

The paper establishes that this feasible criterion, and thus the expectation of the candidate-specific term itself, can be consistently estimated and use it to select the optimal first-stage implementation.

Simulation Results

The simulation study shows that the proposed criterion adjusts the first-stage complexity based on the level of endogeneity, consistent with the increasing complexity cost seen in (23)

Robustness to Non-Gaussian Errors

The similarity between results in Gaussian and non-Gaussian regimes is echoed in Table 2 which shows that the feasible criterion again has the smallest average and maximum relative risk at both sample sizes.

Conclusion

The conditional Gaussianity and homoskedasticity restrictions in Assumption 1 are used to derive the AMSE ranking.

Simulation Details and Complete Results

We use the residualized movie sales and weather data for the 1,671 opening-weekend observations in Gilchrist and Sands (2016)

Table 3: Relative risk by simulation design under Gaussian errors

The table shows that the proposed criterion adjusts the first-stage complexity based on the level of endogeneity, consistent with the increasing complexity cost seen in (23)

Table 4: Relative risk by simulation design under Laplace errors

The table shows that the proposed criterion adjusts the first-stage complexity based on the level of endogeneity, consistent with the increasing complexity cost seen in (23)

Table 5: Mean effective dimension under Gaussian errors

The table shows that the proposed criterion adjusts the first-stage complexity based on the level of endogeneity, consistent with the increasing complexity cost seen in (23)

Table 6: Mean effective dimension under Laplace errors

The table shows that the proposed criterion adjusts the first-stage complexity based on the level of endogeneity, consistent with the increasing complexity cost seen in (23)

Table 7: Risk relative to the oracle under Gaussian errors

The table shows that the proposed criterion adjusts the first-stage complexity based on the level of endogeneity, consistent with the increasing complexity cost seen in (23)

Table 8: Risk relative to the oracle under Laplace errors

The table shows that the proposed criterion adjusts the first-stage complexity based on the level of endogeneity, consistent with the increasing complexity cost seen in (23)

Table 9: Mean effective dimension under Gaussian errors

The table shows that the proposed criterion adjusts the first-stage complexity based on the level of endogeneity, consistent with the increasing complexity cost seen in (23) The table shows that the proposed criterion adjusts the first-stage complexity based on the level of endogeneity, consistent with the increasing complexity cost seen in (2609.0703315>.

Table 1: Relative risk by simulation design under Gaussian errors

The table shows that the proposed criterion adjusts the first-stage complexity based on the level of endogeneity, consistent with the increasing complexity cost seen in (23) The table shows that the proposed criterion adjusts the first-stage complexity based on the level of endogeneity, consistent with the increasing complexity cost seen in (2609.0703315>.

Table 2: Relative risk by simulation design under Laplace errors

The table shows that the proposed criterion adjusts the first-stage complexity based on the level of endogeneity, consistent with the increasing complexity cost seen in (23) The table shows that the proposed criterion adjusts the first-stage complexity based on the level of endogeneity, consistent with the increasing complexity cost seen in (2609.0703315>.

The table shows that the proposed criterion adjusts the first-stage complexity based on the level of endogeneity, consistent with the increasing complexity cost seen in (23) The table shows that the proposed criterion adjusts the first-stage complexity based on the level of endogeneity, consistent with the increasing complexity cost seen in (2609.0703315>.

The table shows that the proposed criterion adjusts the first-stage complexity based on the level of endogeneity, consistent with the increasing complexity cost seen in (23) The table shows that the proposed criterion adjusts the first-stage complexity based on the level of endogeneity, consistent with the increasing complexity cost seen in (2609.0703315>.

The table shows that the proposed criterion adjusts the first-stage complexity based on the level of endogeneity, consistent with the increasing complexity cost seen in (23) The table shows that the proposed criterion adjusts the first-stage complexity based on the level of endogeneity, consistent with the increasing complexity cost seen in (2609.0703315>.

The table shows that the proposed criterion adjusts the first-stage complexity based on the level of endogeneity, consistent with the increasing complexity cost seen in (23) The table shows that the proposed criterion adjusts the first-stage complexity based on the level of endogeneity, consistent with the increasing complexity cost seen in (2609.0703315>.

The table shows that the proposed criterion adjusts the first-stage complexity based on the level of endogeneity, consistent with the increasing complexity cost seen in (23) The table shows that the proposed criterion adjusts the first-stage complexity based on the level of endogeneity, consistent with the increasing complexity cost seen in (2609.0703315>.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Develop an automated first-stage dictionary selection system that ranks candidate dictionaries by minimizing a feasible criterion Sbc, which is constructed to be consistent for selecting optimal implementations according to Theorem 2.

  2. Create an AI system capable of adjusting the complexity cost in a LASSO first stage based on the level of endogeneity, as the AMSE criterion reveals the weight on this bias rises with the endogeneity of the regressor.

  3. Build a model selection tool that uses second-order asymptotic theory to rank first-stage implementations, specifically by minimizing Sc, which is asymptotically equivalent to minimizing Q bar c, thereby selecting estimators with up to a third lower mean squared error compared to cross-validation and plug-in penalty rules in Gaussian and non-Gaussian simulation designs.

  4. Implement a system that utilizes the observed effective dimension dc as an observable criterion for candidate selection, as this is used in the feasible criterion Sbc to rank candidates.

  5. Design a robust evaluation module that estimates necessary population quantities like sigma squared epsilon and sigma squared sev using residuals from a pilot candidate to construct the feasible criterion Sbc, ensuring that the selected candidate attains the smallest candidate-specific AMSE component in the candidate list up to vanishing relative error.

Abstract

Estimating an instrumental variables (IV) first stage with the least absolute shrinkage and selection operator (LASSO) requires choosing a dictionary and penalty. Existing rules target first-stage objectives, although these choices affect structural estimation. We derive a feasible approximate mean squared error (AMSE) criterion for full-sample IV-LASSO with one endogenous regressor and homoskedastic Gaussian errors. For a prespecified list of dictionary-penalty pairs, the criterion trades off approximation of the optimal instrument against a complexity cost that depends on the first two moments of the LASSO effective dimension and on the degree of endogeneity. The selected pair attains the smallest candidate-specific AMSE component up to vanishing relative error. In simulations calibrated to Paravisini et al. (2015), the criterion improves substantially on the plug-in rule at low endogeneity and on cross-validation at high endogeneity, with nearly identical results under non-Gaussian errors. A supplementary appendix formally extends analysis to a class of non-Gaussian errors.

Related papers