Supercompliers
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.
Ruth: We're looking at the paper: "Supercompliers".
Warren: Here's the gist of the paper. In this paper, we develop methods to characterize the supercomplier subpopulation in a canonical instrumental variable framework with a binary instrument and treatment.
Ruth: Supercompliers are defined as those whose treatment take-up positively responds to eligibility and whose outcome improves following treatment take-up, making them important for policy.
Warren: Defining the Supercomplier Population Supercompliers are defined as the subpopulation whose treatment take-up positively responds to eligibility and whose outcome positively responds to take-up.
Ruth: They are characterized by being those who benefit from gaining treatment eligibility and whose outcome improves following treatment take-up, which is the subset of compliers for whom treatment improves the outcome.
Paper summary: Ruth: So we're looking at this paper by Comey and Eng and Leung, titled "Supercompliers." They’re setting up a framework with a binary instrument and treatment to figure out a specific group of people called supercompliers.
Warren: Right, so what exactly are these supercompliers? The summary says they're the subpopulation whose treatment take-up positively responds to eligibility and whose outcome also improves after taking it up.
Ruth: Exactly, so they benefit from getting treatment eligibility and their actual outcome gets better once they start taking the treatment. This group is important for policy because of how much it drives the local average treatment effect we talk about in this context.
Iris: It sounds like they are defining a subset of compliers who actually see a positive change in their results, which is key for understanding why we get certain causal estimates <ref:2212.14105#pg3>.
Ruth: Right, so the core idea here is that this supercomplier population is a building block for the local average treatment effect we already know. They provide that essential piece of the puzzle.
Warren: The paper says they give us tools to characterize these groups under standard local average treatment effect assumptions plus an outcome monotonicity assumption. That's a big chunk of their methodology <ref:2212.14105#pg3>.
Iris: Outcome monotonicity is an extra assumption they need, which essentially means that receiving treatment either doesn't change the outcome or it changes it in just one direction, so some people get better and others get worse <ref:2212.14105#pg3>.
Ruth: That’s a crucial condition because if that monotonicity doesn't hold, the paper suggests the characteristics of supercompliers we try to estimate might be biased unless those two groups share the same average characteristics <ref:2212.14105#pg3>.
Warren: And they also point out that this bias grows linearly with how much of that complier population is affected by the treatment <ref:2212.14105#pg3>.
Iris: So, to summarize their identification tools, they claim the share of supercompliers can be identified using the intent-to-treat effect itself <ref:2212.14105#pg3>.
Ruth: And since a local average treatment effect is just the ratio of that intent-to-treat effect and the first stage effects, this means it’s equivalent to the supercomplier population share as a fraction of the total complier population share <ref:2212.14105#pg3>.
Warren: They show that their characteristics are identified with expressions very similar to how we characterize compliers, which is good because it means we can use existing methods <ref:2212.14105#pg3>.
Iris: They provide estimators for the average and distributional characteristics of supercompliers that you can implement using standard instrumental variable regressions <ref:2212.14105#pg3>.
Ruth: That’s practical stuff, so it means these tools aren't some abstract theory; they are things we can actually run in Stata or similar software right now <ref:2212.14105#pg3>.
Warren: They also developed sharp joint tests to check the validity of their assumptions, including that outcome monotonicity <ref:2212.14105#pg3>.
Iris: And one of the applications they show is in welfare analysis, where you can use these supercomplier characteristics to add social weights into a marginal value of public funds analysis <ref:2212.14105#pg3>.
Paper summary: Ruth: That’s interesting because they suggest that the weighted willingness to pay for the complier group equals the unweighted local average treatment effect multiplied by these identified supercomplier characteristics <ref:2212.14105#pg3>.
Warren: That means readers can compute their own customized weighted willingness to pay and marginal value of public funds even without having microdata, as long as those social weights are a linear function of the characteristics they report <ref:2212.14105#pg3>.
Iris: To see this in action, they use data from two job-training experiments, the National Job Corps Study and the National Job Training Partnership Act Study <ref:2212.14105#pg3>.
Ruth: Their empirical findings show some interesting differences between the two studies. In the Job Corps experiment, participants whose labor market outcomes were improved by that program had relatively higher baseline incomes <ref:2212.14105#pg3>.
Warren: But in the JTPA study, they found that adult women who benefited from that program had relatively lower baseline incomes <ref:2212.14105#pg3>.
Iris: So it shows that the characteristics of these supercompliers can vary significantly depending on the specific training or treatment being studied <ref:2212.14105#pg3>.
Ruth: The paper concludes by summarizing that they developed methods to characterize supercompliers in a binary instrument setting <ref:2212.14105#pg3>.
Warren: They confirm that under the standard local average treatment effect assumptions plus outcome monotonicity, we can identify the characteristics of these supercompliers <ref:2212.14105#pg3>.
Iris: And they show that these identification results lead to natural estimators that are easy to implement using instrumental variable regressions <ref:2212.14105#pg3>.
Ruth: And finally, they show we can use supercomplier characteristics to incorporate social weights into a welfare analysis of the marginal value of public funds <ref:2212.14105#pg3>.
Warren: The main implication for me is that this gives researchers a way to actually enrich their welfare analyses by using these identified characteristics <ref:2212.14105#pg3>.
Iris: It means you can compute a customized weighted willingness to pay and marginal value of public funds even without having the raw microdata, provided you have those linear social weights <ref:2212.14105#pg3>.
Ruth: So, in simple terms, this paper gives us a concrete way to define who the supercompliers are and how they relate to the treatment effect we measure <ref:2212.14105#pg3>.
Warren: It shows that this framework is useful for policy because it helps you understand the specific subset of people who actually benefit from the eligibility criteria <ref:2212.14105#pg1>.
Iris: The paper also clearly states a limitation, and it’s that if outcome monotonicity doesn't hold, the estimator for supercomplier characteristics will be biased unless those two groups have the same average characteristics <ref:2212.14105#pg3>.
Ruth: So we need to be careful when applying this; if we assume the outcome isn't always moving in one direction, our estimates get skewed unless we make that assumption about the baseline characteristics <ref:2212.14105#pg3>.
Warren: The overall picture is that they developed methods to characterize supercompliers in a binary treatment setting with clear identification results for their use in welfare analysis and empirical tests like job training studies <ref:2212.14105#pg3>.
Conclusion: Ruth: So we're wrapping up the discussion on this paper called "Supercompliers."
Warren: Basically, they developed these methods to identify this specific subpopulation of people in a study with a binary instrument and treatment.
Iris: They are focusing on those individuals whose treatment take-up responds positively to eligibility and whose outcome actually improves after they start taking it up.
Ruth: That's the supercomplier group, which is crucial for understanding the local average treatment effect we're trying to measure.
Warren: The authors show you can characterize this group under standard local average treatment effect assumptions plus an outcome monotonicity assumption.
Iris: Outcome monotonicity is a key thing here because it means the outcome either doesn't change or it only moves in one direction after the treatment.
Ruth: And they give us estimators, which are basically tools you can plug into existing instrumental variable regressions to figure out the characteristics of these supercompliers.
Warren: They also show that if you drop that monotonicity assumption, your estimates for supercomplier characteristics will be biased unless those two groups share the same average characteristics.
Iris: That bias gets bigger as the share of compliers changes, so it’s a real warning about when you can trust these results without making strong assumptions.
Ruth: The paper also shows how these supercomplier traits can be used in welfare analysis to figure out customized social weights for public funds.
Warren: So, this isn't just theoretical; it suggests researchers can use these identified traits to build more realistic models of how public spending actually works.
Iris: It opens up a way to compute a weighted willingness to pay that incorporates these specific supercomplier characteristics, even if you don't have all the raw microdata.
Ruth: It gives us a tangible way to enrich those welfare analyses we've been talking about, moving beyond just the main treatment effect.
Warren: And it shows that empirical findings from job training experiments can be used to test these ideas with real data, like the job corps and jtpa studies.
Iris: The two studies actually showed different things about baseline incomes for different groups of participants.
Ruth: So, to summarize, this paper provides a framework for precisely defining supercompliers in a binary instrument setting and shows how those traits matter for policy analysis and welfare modeling.
Warren: It gives us the tools to move from just measuring an average effect to understanding the specific people who make that effect happen.
Iris: The next thing we'll look at is how they actually test if those identification assumptions hold up in practice.
Joint Committee on Taxation · Internal Revenue Service · Cornell University
econ.EM
Submitted: 2022-12-28
Updated: 2024-12-20
Comments: This version substantially revises v2. Pauline Leung has made significant contributions and is now a coauthor. We expand the non-binary outcome case, essential in the new connection to MVPF (Section 3). We replace the original empirical application with two job training experiments (Section 4), add new theoretical results in Remark 5, Appendix A.3, and A.7. References are updated
DOI: 10.1016/j.jpubeco.2026.105660
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 79/100
The gist: In this paper, we develop methods to characterize the supercomplier subpopulation in a canonical instrumental variable framework with a binary instrument and treatment.
Key concepts
- Supercompliers
- This is a specific subpopulation where receiving treatment positively affects both eligibility and the final outcome. They benefit from gaining eligibility, and their results improve after they receive the treatment. They are crucial for understanding policy impact.
- LATE (Local Average Treatment Effect)
- LATE refers to the effect of a treatment on a specific group of people—the compliers. The paper shows that the share of supercompliers is equivalent to the LATE when compared to the complier population share, helping define this key policy measure.
- Outcome Monotonicity
- This assumption requires that if receiving treatment improves eligibility, it must also improve the outcome. The authors test this assumption and find that if it fails, their estimates for supercomplier characteristics become biased unless the groups have identical average characteristics.
Terminology
Summary
In this paper, we develop methods to characterize the supercomplier subpopulation in a canonical instrumental variable framework with a binary instrument and treatment. p. 1 Supercompliers are defined as those whose treatment take-up positively responds to eligibility and whose outcome improves following treatment take-up, making them important for policy. p. 1
Defining the Supercomplier Population
Supercompliers are defined as the subpopulation whose treatment take-up positively responds to eligibility and whose outcome positively responds to take-up. They are characterized by being those who benefit from gaining treatment eligibility and whose outcome improves following treatment take-up, which is the subset of compliers for whom treatment improves the outcome. p. 1 This subpopulation is a key building block of a LATE, and its prevalence drives the LATE as we know it. p. 1
Identification and Estimation Tools
The authors provide tools to characterize supercompliers under standard LATE assumptions plus an outcome monotonicity assumption. They show that the supercomplier population share is identified by the intent-to-treat effect, and since a LATE is the ratio of intent-to-treat and first stage effects, it is equivalent to the supercomplier population share as a fraction of the complier population share. p. 1 They show that their characteristics are identified with expressions analogous to those of complier characteristics, and they provide estimators for supercomplier average and distributional characteristics that can be easily implemented with standard instrumental variable regressions. p. 2
Testing Assumptions and Robustness
The authors develop sharp joint tests of the validity of their identifying assumptions, including outcome monotonicity. They show that when outcome monotonicity does not hold, the estimand for supercomplier characteristics will be biased unless the supercompliers and the compfiers have the same average characteristics, and this bias increases linearly with the compfier share. p. 2 They also show that their framework can be applied to quasi-experimental research designs by making appropriate adjustments to the underlying assumptions. p. 9
Application in Welfare Analysis
The supercomplier characteristics can be used to enrich welfare analyses by incorporating social weights into a marginal value of public funds (MVPF) analysis. Specifically, the weighted willingness to pay for the complier group is equal to the unweighted local average treatment effect multiplied by our identified supercomplier characteristics. p. 0 This allows readers to compute a customized weighted WTP and MVPF themselves, even without microdata, provided social weights are a linear function of those reported characteristics. p. 3
Empirical Findings
The authors illustrate the utility of their tools using data from two job-training experiments, the National Job Corps Study and the National Job Training Partnership Act Study. They find that participants whose labor market outcomes were improved by Job Corps had relatively higher baseline incomes, while the adult women who benefited from JTPA had relatively lower baseline incomes. p. 4
Conclusion
In this paper, we develop methods to characterize the supercomplier subpopulation in a canonical instrumental variable framework with a binary instrument and treatment. We show that under the standard LATE assumptions plus outcome monotonicity, we can identify the characteristics of supercompliers. p. 0 Our identification results lead to natural estimators, which can be easily implemented using instrumental variable regressions via existing Stata commands. p. 2 Finally, we show that supercomplier characteristics can be used to incorporate social weights in a welfare analysis of the marginal value of public funds. p. 0
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Improve treatment effect estimation by characterizing supercompliers instead of just compliers, which
are only subpopulation to benefit from treatment eligibility and hence, are important for policy.
This allows for targeting treatment eligibility to a population similar to supercompliers if their characteristics differ from compliers. -
Enhance marginal value of public funds analysis by incorporating social welfare weights using reported supercomplier characteristics, as the paper shows that
when social welfare weights depend linearly on observed characteristics, a weighted MVPF can be computed using only information on supercomplier characteristics and the reported LATE.
-
Develop a robust joint test for identifying the assumptions of the instrumental variable framework by extending Proposition 10 to accommodate non-binary outcome variables, allowing researchers to test
outcome monotonicity
without imposing restrictions on Y. -
Implement natural estimators for supercomplier characteristics using standard instrumental variable regressions via existing Stata commands, as shown in Proposition 7 and Appendix A.4, which are efficient and avoid approximation issues related to machine learning methods.
-
Facilitate policy comparison by calculating the weighted marginal value of public funds (MVPF) that reflects a social planner's redistributive preferences, as the framework allows for computing a customized MVPF
using only reported supercomplier characteristics.
Sources
Related papers
- Stationarity with Occasionally Binding Constraints
- Backtesting High-Dimensional Expected Shortfall
- Higher-Order Neyman Orthogonality in Moment-Condition Models
- Online Bayesian Inference for a Dirichlet Process Mixture of Binary Logit Models
- Debiased Conditional Independence Tests for Causal Effects
- Sharp Testable Implications of Encouragement Designs