Empirical Bayes mirror testing for persistent widening of achievement gaps in U.S. school districts
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "Empirical Bayes mirror testing for persistent widening of achievement gaps in U.S. school districts".
Jane: Here's the gist of the paper. Unequal recovery from pandemic learning losses raises concerns about widening achievement gaps between economically disadvantaged students and their peers The study seeks to identify districts whose mathematics gaps widened in every annual interval from 2022 to 2025 <ref:2611.15667#pg5> This analysis uses estimates from the Stanford Education Data Archive (SEDA) for 5,600 U.S.
Tom: school districts <ref:2611.15667#pg2> Identifying such districts requires accounting for correlated errors and differing precision in the estimated annual changes, as well as multiplicity <ref:2611.15667#pg3> The true annual gap changes in each district form an effect vector, and persistent widening means that every component is positive <ref:2611.15667#pg4>.
Jane: Pruning redundant reasoning tokens The central question addressed by this research is which districts experienced a widening mathematics gap between economically disadvantaged students and their peers in every annual interval <ref:2611.15667#pg3>.
Tom: Recovery patterns vary substantially across districts, meaning a larger gap at the end of the period does not by itself establish an increase in every year <ref:2611.15667#pg2>.
Paper summary: Tom: To wrap up, this paper, "Empirical Bayes mirror testing for persistent widening of achievement gaps in U.S. school districts," addresses the problem of identifying which districts are seeing a sustained widening of the mathematics gap between disadvantaged students and their peers over several years.
Jane: The main contribution is proposing an empirical Bayes mirror procedure that uses an estimated distribution of effect vectors to rank units, and uses covariance-adjusted mirror observations to calibrate selections.
Lu: It’s about moving beyond just looking at the final gap size, because a bigger gap at the end doesn't prove it grew every year.
Meng: Essentially, it provides a way to select districts where the effect vector has all positive components across those years, which is more specific than standard multiple testing approaches.
Jane: And they show that for their SEDA data, this method successfully selected eight districts at a two point five level of significance.
Tom: The implication is that we can use this technique to pinpoint communities with evidence of sustained divergence during recovery periods.
Lu: It’s a way to identify those communities with evidence of sustained divergence between economically disadvantaged students and their peers across the time interval analyzed.
Jane: They've shown this procedure controls the false discovery rate under specific assumptions about the observation model, which is important for making sure our findings are reliable.
Conclusion: Tom: So, we've been diving into this paper about empirical Bayes mirror testing for persistent widening of achievement gaps in U.S. school districts across years twenty twenty two to twenty twenty five, and what it actually means for those communities.
Jane: It boils down to finding districts where the gap between disadvantaged students and their peers got bigger every single year, not just at the end of the study period. So, we’re looking for sustained divergence over time.
Lu: The authors are using this empirical Bayes mirror procedure to select these specific districts based on their effect vectors all being positive across those annual changes, which is a pretty specific way to filter the data.
Meng: From an engineering side, it’s interesting how they handle the correlations between different parts of the data when selecting these units; that part about adjusting coordinates based on error covariance sounds like it’s a lot of bookkeeping to get right at scale.
Tom: It sounds like they're trying to make sure we don't just pick districts with a big final gap, but ones that have actually been consistently widening, which is the core issue they’re tackling.
Jane: Right. And the authors found that this method successfully identified eight districts using their SEDA data at a two point five level of significance when they ran it at q equals zero point two five.
Lu: That selection process shows how pooling information across different units with this procedure allows them to find these patterns, and it’s a step toward making more robust selections than just running standard tests on each district individually.
Meng: But what does this mean for someone actually working in education or policy? Does finding these eight districts give them a clear path forward for intervention, or is it just another statistical artifact?
Tom: It suggests that when we look at the long-term recovery from things like the pandemic, we need to focus on these specific communities where the mathematical gap isn't just fluctuating, but actively growing year after year.
Jane: Exactly. This work gives us a tool to pinpoint exactly where sustained divergence is happening, moving beyond just a single snapshot of achievement differences.
Lu: And it opens up avenues for future research into modeling those common trajectory patterns, which is the next logical step after they’ve established this selection framework.
Bowen Gang, Yin Xia
Department of Mathematical and Statistical Sciences, University of Alberta · Department of Statistics and Data Science, Fudan University
stat.AP, stat.ME
Submitted: 2026-11-11
Updated: 2026-11-11
Comments: 24 pages, 4 figures
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
Importance score: 88/100
The gist: Unequal recovery from pandemic learning losses raises concerns about widening achievement gaps between economically disadvantaged students and their peers The study seeks to identify districts whose…
Key concepts
- Widening Mathematics Gap
- This refers to a situation where the difference in math achievement between students from economically disadvantaged backgrounds and their peers increases consistently every year over a period. The research specifically looks for districts where this trend of increasing inequality persisted annually.
- Empirical Bayes Mirror Procedure
- This is a statistical tool designed to select data points (school districts) based on patterns in their yearly changes. It learns the distribution of these change patterns and uses this knowledge to rank candidates, aiming to find units with consistently positive trends across all measured years.
- Finite-Sample FDR Control
- False Discovery Rate (FDR) control is a method used when testing many hypotheses simultaneously to manage the risk of making false discoveries. This procedure ensures that among the districts selected as having widening gaps, only a small, controlled proportion are likely false positives, even with limited data.
Terminology
Summary
Unequal recovery from pandemic learning losses raises concerns about widening achievement gaps between economically disadvantaged students and their peers The study seeks to identify districts whose mathematics gaps widened in every annual interval from 2022 to 2025 <ref:2611. p. 0 cn
Abstract
Unequal recovery from pandemic learning losses raises concerns about widening achievement
gaps between economically disadvantaged students and their peers. This study seeks to identify
districts whose mathematics gaps widened in every annual interval from 2022 to 2025. This study seeks to identify
districts whose mathematics gaps widened in every annual interval from 2022 to 2025. The
analysis uses estimates from the Stanford Education Data Archive (SEDA) for 5,600 U. Since academic achievement is linked to college attendance and adult earnings
(Chetty, Friedman and Rockoff, 2014), identifying these districts can help locate communities where
the achievement gap between disadvantaged students and their peers continues to widen. 1
arXiv:2611. school
districts. Identifying such districts requires accounting for correlated errors and differing precision
in the estimated annual changes, as well as multiplicity. Identifying such districts requires accounting for correlated errors and differing precision
in the estimated annual changes, as well as multiplicity. The true annual gap changes in each
district form an effect vector, and persistent widening means that every component is positive.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Pruning redundant reasoning tokens
The central question addressed by this research is which districts experienced a widening mathematics gap between economically disadvantaged students and their peers in every annual interval <ref:2611.15667#pg3>. Recovery patterns vary substantially across districts, meaning a larger gap at the end of the period does not by itself establish an increase in every year <ref:2611. p. 1 Recovery patterns vary substantially across districts (Dewey et al. Within a district, a
larger gap at the end of the period does not by itself establish an increase in every year. Within a district, a
larger gap at the end of the period does not by itself establish an increase in every year. Assessing
such persistence requires accounting for differences in the precision of annual estimates across
districts and for correlated errors in adjacent estimated changes. These correlations arise even when
annual estimation errors are independent, because a shared annual estimate enters the two differences
with opposite signs. With thousands of districts under consideration, multiple comparisons must
also be addressed. We therefore seek to identify persistent increases while controlling the false
discovery rate (FDR; Benjamini and Hochberg, 1995), the expected proportion of selected districts
for which the claim of persistent widening is false. A standard approach takes the largest one-sided p-value across the annual changes for each
district (Benjamini and Heller, 2008) and applies the Benjamini–Hochberg procedure (Benjamini
and Hochberg, 1995). Although this approach controls FDR under independence across districts, its
ranking depends only on the weakest standardized change. It therefore leaves information beyond
the weakest change unused and does not adapt the ranking to the distribution of change vectors
across districts.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
How the training signal is built
The empirical Bayes mirror procedure is developed to select units whose mean vectors have all positive components <ref:2611.15667#pg4>. The procedure learns the distribution of effect vectors across units and uses estimated posterior null probabilities to rank candidate discoveries <ref:2611. p. 2 In this paper, we develop an empirical Bayes mirror procedure for selecting units whose mean
vectors have all positive components. The procedure learns the distribution of effect vectors across
units and uses estimated posterior null probabilities to rank candidate discoveries. The procedure learns the distribution of effect vectors across
units and uses estimated posterior null probabilities to rank candidate discoveries. To calibrate
selection, we construct reflected versions of each observation using its error covariance. To calibrate
selection, we construct reflected versions of each observation using its error covariance. The resulting
comparisons classify observations as candidate discoveries or controls, with a null observation at
least as likely to be a control as a candidate discovery. It uses an estimated distribution of effect vectors to guide ranking and covariance-adjusted mirror
observations to calibrate selection. The procedure achieves finite-sample false discovery rate
(FDR) control under a Gaussian observation model, without requiring accurate estimation of
the posterior null probabilities used for ranking.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the gains come from
The core mechanism involves constructing mirror transformations that account for correlations among coordinates and support conservative FDR calibration <ref:2611.15667#pg6>. The map changes xj to −xj and adjusts the other coordinates according to their error covariances with coordinate j <ref:2611. p. 5 4)
where Σ
(i)
·j
is the jth column of Σ
(i)
. The map changes xj to −xj and adjusts the other coordinates
according to their error covariances with coordinate j. A district is eligible if its observed vector or at least one of its d reflections lies in
C. We define the eligible set by
I = {i : X(i) ∈ C} ∪ [
d
j=1
{i : R
(i)[j]X(i) ∈ C}. Since academic achievement is linked to college attendance and adult earnings
(Chetty, Friedman and Rockoff, 2014), identifying these districts can help locate communities where
the achievement gap between disadvantaged students and their peers continues to widen. 1
arXiv:2611. It uses an estimated distribution of effect vectors to guide ranking and covariance-adjusted mirror
observations to calibrate selection. The procedure achieves finite-sample false discovery rate
(FDR) control under a Gaussian observation model, without requiring accurate estimation of
the posterior null probabilities used for ranking. A district has a persistently increasing gap if every coordinate of ∆(i)
is positive. This leads to
the simultaneous tests
H
(i)
0
: min
1≤j≤d
∆
(i)
j ≤ 0 versus H
(i)
1
: ∆(i)
j > 0 for all j. The control count then determines where
to cut off the ranking. We establish finite-sample FDR control for independent Gaussian observation vectors with
known covariance matrices having nonpositive off-diagonal entries.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Application to the SEDA data
The study applies the proposed procedure and comparison methods to the SEDA sample described in Section 2.1 <ref:2611.15667#pg2>. The results show that at q = 0. p. 1 Symmetry and exchangeability provide a basis for such calibration. Knockoff procedures (Barber
and Candès, 2015; Ren and Candès, 2023) and masking methods (Lei and Fithian, 2018; Leung and
Sun, 2022) show how learned rankings can be combined with finite-sample FDR control. At a target FDR of 25%, the
SEDA analysis identifies eight districts in five states with evidence of widening gaps in every
annual interval. Keywords: Achievement gaps; Empirical Bayes; False discovery rate; Mirror statistics; Multiple
testing. Simulations demonstrate power gains from
incorporating information about the distribution of effect vectors. At a target FDR of 25%, the
SEDA analysis identifies eight districts in five states with evidence of widening gaps in every
annual interval. states, form our analysis sample. The annual standard errors range from 0. 6) as the observation reconstructs the same list. We assign a common
score a
(i)
to this list, so a candidate discovery and its possible controls are compared at the same
threshold. Since academic achievement is linked to college attendance and adult earnings
(Chetty, Friedman and Rockoff, 2014), identifying these districts can help locate communities where
the achievement gap between disadvantaged students and their peers continues to widen. 1
arXiv:2611.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Discussion
The empirical Bayes mirror procedure allows information to be pooled across units while retaining finite-sample FDR control under the observation model in Section 2.2 <ref:2611. p. 0 Since academic achievement is linked to college attendance and adult earnings
(Chetty, Friedman and Rockoff, 2014), identifying these districts can help locate communities where
the achievement gap between disadvantaged students and their peers continues to widen. 1
arXiv:2611. Since academic achievement is linked to college attendance and adult earnings
(Chetty, Friedman and Rockoff, 2014), identifying these districts can help locate communities where
the achievement gap between disadvantaged students and their peers continues to widen. 1
arXiv:2611. At a target FDR of 25%, the
SEDA analysis identifies eight districts in five states with evidence of widening gaps in every
annual interval. Keywords: Achievement gaps; Empirical Bayes; False discovery rate; Mirror statistics; Multiple
testing. cn
Abstract
Unequal recovery from pandemic learning losses raises concerns about widening achievement
gaps between economically disadvantaged students and their peers. This study seeks to identify
districts whose mathematics gaps widened in every annual interval from 2022 to 2025.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
References
Barber, R. F. and Candès, E. p. 1 Symmetry and exchangeability provide a basis for such calibration. Knockoff procedures (Barber
and Candès, 2015; Ren and Candès, 2023) and masking methods (Lei and Fithian, 2018; Leung and
Sun, 2022) show how learned rankings can be combined with finite-sample FDR control. Symmetry and exchangeability provide a basis for such calibration. Knockoff procedures (Barber
and Candès, 2015; Ren and Candès, 2023) and masking methods (Lei and Fithian, 2018; Leung and
Sun, 2022) show how learned rankings can be combined with finite-sample FDR control. With thousands of districts under consideration, multiple comparisons must
also be addressed. We therefore seek to identify persistent increases while controlling the false
discovery rate (FDR; Benjamini and Hochberg, 1995), the expected proportion of selected districts
for which the claim of persistent widening is false.Where the paper says it
Where the paper says it
Where the paper says it
Benjamini, Y. and Hochberg, Y. p. 1 With thousands of districts under consideration, multiple comparisons must
also be addressed. We therefore seek to identify persistent increases while controlling the false
discovery rate (FDR; Benjamini and Hochberg, 1995), the expected proportion of selected districts
for which the claim of persistent widening is false. We therefore seek to identify persistent increases while controlling the false
discovery rate (FDR; Benjamini and Hochberg, 1995), the expected proportion of selected districts
for which the claim of persistent widening is false. A standard approach takes the largest one-sided p-value across the annual changes for each
district (Benjamini and Heller, 2008) and applies the Benjamini–Hochberg procedure (Benjamini
and Hochberg, 1995).Where the paper says it
Where the paper says it
Dai, R. and Zheng, C. p. 2 The resulting comparisons must
remain valid when the same data are used to learn the ranking. Related work on union null hypotheses includes simultaneous knockoffs (Dai and Zheng, 2023)
and the joint mirror procedure (Deng, He and Zhang, 2024). Empirical Bayes methods are widely used in large-scale multiple testing. A key idea is to use
information across units to rank hypotheses by the local false discovery rate (lfdr), the posterior
probability that the null hypothesis is true (Efron et al.Where the paper says it
Where the paper says it
Dewey, D. C., Fahle, E. p. 2 We use the released unshrunk gap estimates and their standard errors, with
economically disadvantaged status defined by each state’s classification. The gaps are on SEDA’s
cohort-standardized scale, in student standard-deviation units relative to national reference cohorts
(Fahle et al. We use the released unshrunk gap estimates and their standard errors, with
economically disadvantaged status defined by each state’s classification. The gaps are on SEDA’s
cohort-standardized scale, in student standard-deviation units relative to national reference cohorts
(Fahle et al. We analyze the Stanford Education Data Archive (SEDA) 2025. 2 (Reardon et al. ca xiayin@fudan. cn
Abstract
Unequal recovery from pandemic learning losses raises concerns about widening achievement
gaps between economically disadvantaged students and their peers.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Fu, L., Gang, B., James, G. M., and Sun, W. p. 1 A key idea is to use
information across units to rank hypotheses by the local false discovery rate (lfdr), the posterior
probability that the null hypothesis is true (Efron et al. , 2001; Sun and Cai, 2007). A key idea is to use
information across units to rank hypotheses by the local false discovery rate (lfdr), the posterior
probability that the null hypothesis is true (Efron et al. , 2001; Sun and Cai, 2007). A key idea is to use
information across units to rank hypotheses by the local false discovery rate (lfdr), the posterior
probability that the null hypothesis is true (Efron et al. , 2001; Sun and Cai, 2007). A key idea is to use
information across units to rank hypotheses by the local false discovery rate (lfdr), the posterior
probability that the null hypothesis is true (Efron et al. , 2001; Sun and Cai, 2007). These
problems share a union null hypothesis that allows at least one nonpositive effect, with the remaining
effects unrestricted. Empirical Bayes methods are widely used in large-scale multiple testing.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Gang, B. and Banerjee, T. p. 7 Since the reflection maps have
absolute Jacobian one, the masked log-likelihood is
ℓmask(G) = X
i∈I
log(X
d
r=0
f
(i)
G (x
(i)[r]
)
)
+
X
i /∈I
log f
(i)
G (X(i)
). 14)
We approximate G by a mixture of point masses on a finite grid, as in Jiang and Zhang (2009) and
Gang and Banerjee (2025). Empirical Bayes methods are widely used in large-scale multiple testing. A key idea is to use
information across units to rank hypotheses by the local false discovery rate (lfdr), the posterior
probability that the null hypothesis is true (Efron et al.Where the paper says it
Where the paper says it
Heller, R. and Rosset, S. p. 7 Expectations are taken over both the random effect vectors and their
observations, with the covariance matrices held fixed. The next theorem specializes the ordering
argument in Heller and Rosset (2021, Theorem 2. 11)
independently across districts, where G is a probability distribution on R
d
. This induces a two-group
model according to whether ∆(i) ∈ C.Where the paper says it
Where the paper says it
Jiang, W. and Zhang, C. p. 2 The resulting comparisons must
remain valid when the same data are used to learn the ranking. Related work on union null hypotheses includes simultaneous knockoffs (Dai and Zheng, 2023)
and the joint mirror procedure (Deng, He and Zhang, 2024). This motivates scores that approximate
the lfdr ordering while satisfying the masking requirement in Section 3. To estimate the lfdr, we fit G by maximum likelihood using only the information in M, following
the multivariate empirical Bayes approach of Soloff, Guntuboyina and Sen (2025).Where the paper says it
Where the paper says it
Lei, L. and Fithian, W. p. 1 Symmetry and exchangeability provide a basis for such calibration. Knockoff procedures (Barber
and Candès, 2015; Ren and Candès, 2023) and masking methods (Lei and Fithian, 2018; Leung and
Sun, 2022) show how learned rankings can be combined with finite-sample FDR control. It therefore leaves information beyond
the weakest change unused and does not adapt the ranking to the distribution of change vectors
across districts. The same testing problem arises beyond achievement gaps whenever the goal is to identify
units whose mean vectors have all positive components.Where the paper says it
Where the paper says it
Leung, D. and Sun, W. (2022). ZAP: Z-value adaptive procedures for false discovery rate control with side information <ref:2611. p. 0 The procedure achieves finite-sample false discovery rate
(FDR) control under a Gaussian observation model, without requiring accurate estimation of
the posterior null probabilities used for ranking. Simulations demonstrate power gains from
incorporating information about the distribution of effect vectors.Where the paper says it
Reardon, S. F., Fahle, E. M., Ho, A. D., Shear, B. R., Saliba, J., Min, J., Shim, J. and Kalogrides, D. (2026). Stanford Education Data Archive Technical Documentation: SEDA 2025. p. 0 This study seeks to identify
districts whose mathematics gaps widened in every annual interval from 2022 to 2025. The
analysis uses estimates from the Stanford Education Data Archive (SEDA) for 5,600 U.Where the paper says it
Ren, Z. and Candès, E. (2023). Knockoffs with side information <ref:2611. p. 0 Since academic achievement is linked to college attendance and adult earnings
(Chetty, Friedman and Rockoff, 2014), identifying these districts can help locate communities where
the achievement gap between disadvantaged students and their peers continues to widen. 1
arXiv:2611.Where the paper says it
Soloff, J. A., Guntuboyina, A. p. 7 This motivates scores that approximate
the lfdr ordering while satisfying the masking requirement in Section 3. To estimate the lfdr, we fit G by maximum likelihood using only the information in M, following
the multivariate empirical Bayes approach of Soloff, Guntuboyina and Sen (2025). This motivates scores that approximate
the lfdr ordering while satisfying the masking requirement in Section 3. To estimate the lfdr, we fit G by maximum likelihood using only the information in M, following
the multivariate empirical Bayes approach of Soloff, Guntuboyina and Sen (2025). This motivates scores that approximate
the lfdr ordering while satisfying the masking requirement in Section 3. To estimate the lfdr, we fit G by maximum likelihood using only the information in M, following
the multivariate empirical Bayes approach of Soloff, Guntuboyina and Sen (2025).Where the paper says it
Where the paper says it
Where the paper says it
Subramanian, A., Narayan, R., Corsello, S. M. et al. (2017). A next generation connectivity map: L1000 platform and the first 1,000,000 profiles <ref:2611. p. 1 In dose-response screening, one may seek genes whose mean
expression increases at every dose increment for a fixed drug, cell type, and exposure time. Large
drug-perturbation resources such as LINCS L1000 provide expression profiles for investigating such
questions (Subramanian et al.Where the paper says it
Sun, W. and Cai, T. p. 1 A key idea is to use
information across units to rank hypotheses by the local false discovery rate (lfdr), the posterior
probability that the null hypothesis is true (Efron et al. , 2001; Sun and Cai, 2007). It uses an estimated distribution of effect vectors to guide ranking and covariance-adjusted mirror
observations to calibrate selection. The procedure achieves finite-sample false discovery rate
(FDR) control under a Gaussian observation model, without requiring accurate estimation of
the posterior null probabilities used for ranking.Where the paper says it
Where the paper says it
U.S. Census Bureau (2024). p. 0 Pandemic learning losses were larger among children
from lower socioeconomic backgrounds (Betthäuser et al. , 2023), and the average within-district
socioeconomic mathematics gap in the United States increased from 2022 to 2024 (Dewey et al. Since academic achievement is linked to college attendance and adult earnings
(Chetty, Friedman and Rockoff, 2014), identifying these districts can help locate communities where
the achievement gap between disadvantaged students and their peers continues to widen. 1
arXiv:2611.Where the paper says it
Where the paper says it
Zhao, Z. and Sun, W. (2025a). A conformalized empirical Bayes method for multiple testing with side information <ref:2611. p. 1 Empirical Bayes methods are widely used in large-scale multiple testing. A key idea is to use
information across units to rank hypotheses by the local false discovery rate (lfdr), the posterior
probability that the null hypothesis is true (Efron et al.Where the paper says it
Zhao, Z. and Sun, W. (2025b). False discovery rate control for structured multiple testing: Asymmetric rules and conformal Q-values <ref:2611. p. 0 At a target FDR of 25%, the
SEDA analysis identifies eight districts in five states with evidence of widening gaps in every
annual interval. Keywords: Achievement gaps; Empirical Bayes; False discovery rate; Mirror statistics; Multiple
testing.Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Empirical Bayes mirror procedure for selecting units with all positive components can be used to identify persistent widening of achievement gaps in U.S. school districts, achieving a finite sample false discovery rate FDR control without requiring accurate estimation of posterior null probabilities used for ranking.
-
The system can rank candidate discoveries using an empirical Bayes score based on the posterior probability of the union null, which is estimated by fitting a mixture distribution on a finite grid to masked data from eligible districts.
-
The AI system can perform selection by setting a threshold T based on the comparison of candidate discoveries and controls, ensuring that
a null observation at least as likely to be a control as a candidate discovery
holds without knowing which mean component is nonpositive. -
The system can estimate the mixing distribution G by maximizing a masked log-likelihood using only information in M, and then construct an empirical Bayes score where
Larger scores correspond to smaller lfdr values at the all-positive vectors x(i)[0].
-
The AI system can compare different selection methods, such as EB mirror, fixed mirror, and full-data NPMLE, to determine which method provides better power when learning the distribution of effect vectors is incorporated into the ranking.
Abstract
Unequal recovery from pandemic learning losses raises concerns about widening achievement gaps between economically disadvantaged students and their peers. This study seeks to identify districts whose mathematics gaps widened in every annual interval from 2022 to 2025. The analysis uses estimates from the Stanford Education Data Archive (SEDA) for 5,600 U.S. school districts. Identifying such districts requires accounting for correlated errors and differing precision in the estimated annual changes, as well as multiplicity. The true annual gap changes in each district form an effect vector, and persistent widening means that every component is positive. To address this class of multiple testing problems, we propose an empirical Bayes mirror procedure. It uses an estimated distribution of effect vectors to guide ranking and covariance-adjusted mirror observations to calibrate selection. The procedure achieves finite-sample false discovery rate (FDR) control under a Gaussian observation model, without requiring accurate estimation of the posterior null probabilities used for ranking. Simulations demonstrate power gains from incorporating information about the distribution of effect vectors. At a target FDR of 25%, the SEDA analysis identifies eight districts in five states with evidence of widening gaps in every annual interval.
Sources
Related papers
- Unraveling time-varying causal effects of multiple exposures: integrating Functional Data Analysis with Multivariable Mendelian Randomization
- Strategic Causal Policy Learning with Manipulable Eligibility
- BayesJudge: Uncertainty-Aware Bayesian Meta-Evaluation of Human and LLM Judgments
- Automating structural reliability analysis with a multi-agent large language model framework
- Improving Minority Population Sampling with BISG Probabilities: Evidence from a Survey of Jewish Americans
- When Do Sports Leagues Stop Being Competitive? Bayesian Detection of Competitive-Balance Eras from Match-Level Data