Welfare at Risk: Distributionally Robust Bounds for Policy Evaluation

This paper develops a distributional welfare framework that accounts for the heterogeneity in welfare generated by policy interventions by deriving quantile-based bounds that allow analysts to learn about the distribution of individual welfare changes even when this distribution is unobserved.

Distributional Welfare Primitives

The approach builds on the perturbed utility model (PUM) of Allen and Rehbeck [3] and McFadden and Fosgerau [3], which is a latent choice model that accommodates general additive unobserved heterogeneity. The PUM encompasses a broad class of latent utility models with additively separable unobserved heterogeneity, including additive random utility models (ARUMs), bundling models, and matching models. The conditional average welfare W(x) is defined as E max y∈B U(y; x, ε) X = x.

Quantile-Based Bounds

The paper introduces superquantiles to generalize quantiles by averaging over the distribution’s tail beyond a given quantile—for instance, the 20% superquantile reports the average welfare among the bottom 20%. The distributional welfare Wβ(x) is defined as Sβ max y∈B U(y; x, ε) X = x. Theorem 1 shows that the superquantile of individual welfare changes—unobservable to the analyst—is bounded above by the superquantile of conditional average welfare changes, which is identified.

Robustness and Ambiguity

The framework extends to allow for ambiguity in the analyst’s knowledge of the distribution of observables X, bounding welfare changes under a worst case over a ϕ-divergence neighborhood where the center is given by the nominal covariate distribution. The robust bound S Rβ(Z) is defined as inf G∈Uϕ(F,ρ) S Gβ(Z). Theorem 3 shows that for all β ∈ (0, 1] and any ϕ ∈ Φ and ρ > 0, the upper bound on the superquantile of individual welfare changes continues to hold when the marginal distribution of X is itself ambiguous, with the extent of this ambiguity controlled by ρ.

Application to Treatment Effects

The framework is applied to generalized Roy models with subjective participation costs, deriving bounds for treatment on the treated (TT) that are new to this literature. Theorem 5(i) shows that Sβ(WTT) ≤ Sβ(WTT(X, Z)). Corollary 4 provides a lower bound on the buffered probability of welfare loss among the treated, which is computable from the identified function WT T (X, Z) alone.

Instrument Sensitivity Immunity

The results are immune to instrument-sensitivity because they take the infimum over an ambiguity set rather than over instrument choices, meaning the robust welfare risk is a valid bound regardless of which instrument is used to estimate the transition-specific propensity scores. The price of robustness is quantified by Inf(ρ∗), which measures the distance above the trivial floor relative to the non-robust bound, and this quantity can be computed directly from estimated propensity scores.

Conclusion

This paper develops a distributional framework for analyzing welfare heterogeneity across individuals by combining superquantiles with observable-group average welfare effects to construct informative bounds on individual welfare changes without relying on strong structural assumptions or detailed micro-level data. The results provide distributional bounds for the treated that are new to this literature. The core results established in Section 4 extend naturally to this setting, even though the relevant welfare concepts and identification strategies differ. The framework extends naturally to the richer multi-sector environments studied by HUV, providing distributional welfare guarantees that are unavailable from standard IV methods alone.

References

Nikhil Agarwal and Paulo J Somaini. Demand analysis under latent choice constraints. Technical report, National Bureau of Economic Research, 2022.

Roy Allen and John Rehbeck. Identification with additively separable heterogeneity. Econometrica, 87(3):1021–1054, 2019.

Steven Berry, James Levinsohn, and Ariel Pakes. Automobile prices in market equilibrium. Econometrica, 63(4):841–890, 1995.

Steven T. Berry. Estimating discrete-choice models of product differentiation. The RAND Journal of Economics, 25(2):242–262, 1994.

Steven T. Berry and Philip A. Haile. Identification in differentiated products markets using market level data. Econometrica, 82(5):1749–1797, 2014.

Debopam Bhattacharya. Nonparametric welfare analysis for discrete choice. Econometrica, 83(2):617–649, 2015.

Debopam Bhattacharya. Nonparametric approaches to empirical welfare analysis. Journal of Economic Literature, 62(2):554–593, 2024.

Anders Bj¨orklund and Robert Moffitt. The estimation of wage gains and welfare gains in self-selection models. The Review of Economics and Statistics, 69(1):42–49, 1987.

Donald Brown and Caterina Calsamiglia. The strong law of demand. Available at SSRN 379520, 2003.

I. Csiszar. Information-type measures of difference of probability distributions and indirect observation. Studia Scientiarum Mathematicarum Hungarica, 2:229–318, 1967.

Philipp Eisenhauer, James J. Heckman, and Edward Vytlacil. The generalized roy model and the cost-benefit analysis of social programs. Journal of Political Economy, 123(2):413–443, 2015.

Jeremy T Fox and Natalia Lazzati. A note on identification of discrete choice models for bundles and binary games. Quantitative Economics, 8(3):1021–1036, 2017.

Drew Fudenberg, Ryota Iijima, and Tomasz Strzalecki. Stochastic choice and revealed perturbed utility. Econometrica, 83(6):2371–2409, 2015.

Alfred Galichon and Bernard Salani´e. Cupid’s Invisible Hand: Social Surplus and Identification in Matching Models. The Review of Economic Studies, 89(5):2600–2629, 12 2021.

Matthew Gentzkow. Valuing new goods in a model with complementarity: Online newspapers. American Economic Review, 97(3):713–744, June 2007.

W. Hanemann. Welfare Analysis with Discrete Choice Models. CUDARE Working Papers 198646, University of California, Berkeley, Department of Agricultural and Resource Economics, May 1996.

Lars Peter Hansen and Thomas J. Sargent. Robust control and model uncertainty. The American Economic Review, 91(2):60–66, 2001.

Lars Peter Hansen and Thomas J. Sargent. Robustness. Princeton University Press, student edition edition, 2008.

Benjamin H´ebert. Moral hazard and the optimality of debt. The Review of Economic Studies, 85(4):2214–2252, 2018.

James J. Heckman and Edward Vytlacil. Structural equations, treatment effects, and econometric policy evaluation. Econometrica, 73(3):669–738, 2005.

James J. Heckman and Edward J. Vytlacil. Local instrumental variables and latent variable models for identifying and bounding treatment effects. Proceedings of the National Academy of Sciences, 96(8):4730–4734, 1999.

James J. Heckman, Sergio Urzua, and Edward Vytlacil. Understanding instrumental variables in models with essential heterogeneity. The Review of Economics and Statistics, 88(3):389–432, 2006.

Yu-Wei Hsieh, Xiaoxia Shi, and Matthew Shum. Inference on estimators defined by mathematical programming. Journal of Econometrics, 226(2):248–268, None 2022.

Alessandro Iaria and Ao Wang.

1. Distributional Welfare Bounds for Policy Evaluation: The improved system can now learn how gains and losses are spread across a population by using identified conditional average welfare changes to bound superquantiles of individual welfare changes. This allows AI to assess fairness and identify subpopulations most affected by interventions, moving beyond simple average gains or losses.

2. Robustness Against Distributional Ambiguity: The system can perform policy evaluation even when the distribution of observable characteristics is uncertain by using a distributionally robust framework based on phi-divergence neighborhoods. This ensures that welfare bounds remain valid even if the analyst's knowledge of the population shifts, preventing systematic overoptimism seen in non-robust bounds.

3. Treatment on Treated Distributional Analysis: The system can derive distributional results for treatment on the treated (TREATMENT ON THE TREATED) welfare among participants in social programs using generalized Roy models with subjective participation costs. This provides novel insights into the welfare of those who actually participate, which is a result new to this literature.

4. Robust Instrument-Sensitivity Immunity: The system can provide welfare guarantees immune to the instrument-sensitivity problem found in multi-outcome IV frameworks by integrating distributional robust optimization (DRO) bounds over ambiguity sets rather than instrument choices. This means the welfare risk assessment remains valid regardless of which specific instrument is used to estimate propensity scores.

5. Actionable Loss Threshold Identification: The system can invert the bounds to bound the fraction of the population that falls below a specific policy-relevant loss threshold, such as a poverty line or compensation trigger, directly from identified aggregate data. This offers a more direct policy test than fixing a tail fraction in advance.

This paper develops a framework to study how the welfare effects of policy interventions are distributed across individuals when those effects are not directly observed. We bound the superquantile of individual welfare changes by the superquantile of conditional average welfare changes, which is typically identified, revealing how gains and losses are spread across the population. We also bound the share of the population whose welfare loss from the policy exceeds any given threshold --- a distribution-free counterpart to average cost--benefit analysis, closer to a poverty-rate measure of policy harm than to a mean welfare effect. We further extend both bounds to allow for ambiguity in the analyst's knowledge of the distribution of observable characteristics: a non-robust bound estimated on one population can be systematically overoptimistic when applied to a different target population, while the robust version, with its ambiguity radius chosen commensurate with the shift, restores a valid guarantee. We illustrate the method in generalized Roy models with subjective participation costs, bounding not only the distribution of welfare changes in the overall population but also, in a result new to this literature, the entire distribution of welfare (and of participation costs) among those who select into treatment --- the treatment-on-the-treated distribution, not merely its mean. We connect these results to the multi-outcome IV framework of, showing our robust bounds are immune to the instrument-sensitivity problem they document