Empirical Bayes mirror testing for persistent widening of achievement gaps in U.S. school districts

arXiv:2611.15667 · stat.AP, stat.ME · Submitted 2026-11-11 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Empirical Bayes mirror testing for persistent widening of achievement gaps in U.S. school districts".

Jane: Here's the gist of the paper. Unequal recovery from pandemic learning losses raises concerns about widening achievement gaps between economically disadvantaged students and their peers The study seeks to identify districts whose mathematics gaps widened in every annual interval from 2022 to 2025 <ref:2611.15667#pg5> This analysis uses estimates from the Stanford Education Data Archive (SEDA) for 5,600 U.S.

Tom: school districts <ref:2611.15667#pg2> Identifying such districts requires accounting for correlated errors and differing precision in the estimated annual changes, as well as multiplicity <ref:2611.15667#pg3> The true annual gap changes in each district form an effect vector, and persistent widening means that every component is positive <ref:2611.15667#pg4>.

Jane: Pruning redundant reasoning tokens The central question addressed by this research is which districts experienced a widening mathematics gap between economically disadvantaged students and their peers in every annual interval <ref:2611.15667#pg3>.

Tom: Recovery patterns vary substantially across districts, meaning a larger gap at the end of the period does not by itself establish an increase in every year <ref:2611.15667#pg2>.

Paper summary: Tom: To wrap up, this paper, "Empirical Bayes mirror testing for persistent widening of achievement gaps in U.S. school districts," addresses the problem of identifying which districts are seeing a sustained widening of the mathematics gap between disadvantaged students and their peers over several years.

Jane: The main contribution is proposing an empirical Bayes mirror procedure that uses an estimated distribution of effect vectors to rank units, and uses covariance-adjusted mirror observations to calibrate selections.

Lu: It’s about moving beyond just looking at the final gap size, because a bigger gap at the end doesn't prove it grew every year.

Meng: Essentially, it provides a way to select districts where the effect vector has all positive components across those years, which is more specific than standard multiple testing approaches.

Jane: And they show that for their SEDA data, this method successfully selected eight districts at a two point five level of significance.

Tom: The implication is that we can use this technique to pinpoint communities with evidence of sustained divergence during recovery periods.

Lu: It’s a way to identify those communities with evidence of sustained divergence between economically disadvantaged students and their peers across the time interval analyzed.

Jane: They've shown this procedure controls the false discovery rate under specific assumptions about the observation model, which is important for making sure our findings are reliable.

Conclusion: Tom: So, we've been diving into this paper about empirical Bayes mirror testing for persistent widening of achievement gaps in U.S. school districts across years twenty twenty two to twenty twenty five, and what it actually means for those communities.

Jane: It boils down to finding districts where the gap between disadvantaged students and their peers got bigger every single year, not just at the end of the study period. So, we’re looking for sustained divergence over time.

Lu: The authors are using this empirical Bayes mirror procedure to select these specific districts based on their effect vectors all being positive across those annual changes, which is a pretty specific way to filter the data.

Meng: From an engineering side, it’s interesting how they handle the correlations between different parts of the data when selecting these units; that part about adjusting coordinates based on error covariance sounds like it’s a lot of bookkeeping to get right at scale.

Tom: It sounds like they're trying to make sure we don't just pick districts with a big final gap, but ones that have actually been consistently widening, which is the core issue they’re tackling.

Jane: Right. And the authors found that this method successfully identified eight districts using their SEDA data at a two point five level of significance when they ran it at q equals zero point two five.

Lu: That selection process shows how pooling information across different units with this procedure allows them to find these patterns, and it’s a step toward making more robust selections than just running standard tests on each district individually.

Meng: But what does this mean for someone actually working in education or policy? Does finding these eight districts give them a clear path forward for intervention, or is it just another statistical artifact?

Tom: It suggests that when we look at the long-term recovery from things like the pandemic, we need to focus on these specific communities where the mathematical gap isn't just fluctuating, but actively growing year after year.

Jane: Exactly. This work gives us a tool to pinpoint exactly where sustained divergence is happening, moving beyond just a single snapshot of achievement differences.

Lu: And it opens up avenues for future research into modeling those common trajectory patterns, which is the next logical step after they’ve established this selection framework.

Bowen Gang, Yin Xia

Department of Mathematical and Statistical Sciences, University of Alberta · Department of Statistics and Data Science, Fudan University

stat.AP, stat.ME

Submitted: 2026-11-11

Updated: 2026-11-11

Comments: 24 pages, 4 figures

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

Importance score: 88/100

The gist: Unequal recovery from pandemic learning losses raises concerns about widening achievement gaps between economically disadvantaged students and their peers The study seeks to identify districts whose…

Key concepts

Widening Mathematics Gap
This refers to a situation where the difference in math achievement between students from economically disadvantaged backgrounds and their peers increases consistently every year over a period. The research specifically looks for districts where this trend of increasing inequality persisted annually.
Empirical Bayes Mirror Procedure
This is a statistical tool designed to select data points (school districts) based on patterns in their yearly changes. It learns the distribution of these change patterns and uses this knowledge to rank candidates, aiming to find units with consistently positive trends across all measured years.
Finite-Sample FDR Control
False Discovery Rate (FDR) control is a method used when testing many hypotheses simultaneously to manage the risk of making false discoveries. This procedure ensures that among the districts selected as having widening gaps, only a small, controlled proportion are likely false positives, even with limited data.

Terminology

Summary

Unequal recovery from pandemic learning losses raises concerns about widening achievement gaps between economically disadvantaged students and their peers The study seeks to identify districts whose mathematics gaps widened in every annual interval from 2022 to 2025 <ref:2611. p. 0

Where the paper says it

cn Abstract Unequal recovery from pandemic learning losses raises concerns about widening achievement gaps between economically disadvantaged students and their peers. This study seeks to identify districts whose mathematics gaps widened in every annual interval from 2022 to 2025.

15667#pg5> This analysis uses estimates from the Stanford Education Data Archive (SEDA) for 5,600 U. p. 0
Where the paper says it

This study seeks to identify districts whose mathematics gaps widened in every annual interval from 2022 to 2025. The analysis uses estimates from the Stanford Education Data Archive (SEDA) for 5,600 U.

S. school districts <ref:2611. p. 0
Where the paper says it

Since academic achievement is linked to college attendance and adult earnings (Chetty, Friedman and Rockoff, 2014), identifying these districts can help locate communities where the achievement gap between disadvantaged students and their peers continues to widen. 1 arXiv:2611.

15667#pg2> Identifying such districts requires accounting for correlated errors and differing precision in the estimated annual changes, as well as multiplicity <ref:2611. p. 0
Where the paper says it

school districts. Identifying such districts requires accounting for correlated errors and differing precision in the estimated annual changes, as well as multiplicity.

15667#pg3> The true annual gap changes in each district form an effect vector, and persistent widening means that every component is positive <ref:2611. p. 0
Where the paper says it

Identifying such districts requires accounting for correlated errors and differing precision in the estimated annual changes, as well as multiplicity. The true annual gap changes in each district form an effect vector, and persistent widening means that every component is positive.

15667#pg4>.

Pruning redundant reasoning tokens

The central question addressed by this research is which districts experienced a widening mathematics gap between economically disadvantaged students and their peers in every annual interval <ref:2611.15667#pg3>. Recovery patterns vary substantially across districts, meaning a larger gap at the end of the period does not by itself establish an increase in every year <ref:2611. p. 1

Where the paper says it

Recovery patterns vary substantially across districts (Dewey et al. Within a district, a larger gap at the end of the period does not by itself establish an increase in every year.

15667#pg2>. Assessing such persistence requires accounting for differences in the precision of annual estimates across districts and for correlated errors in adjacent estimated changes <ref:2611. p. 1
Where the paper says it

Within a district, a larger gap at the end of the period does not by itself establish an increase in every year. Assessing such persistence requires accounting for differences in the precision of annual estimates across districts and for correlated errors in adjacent estimated changes.

15667#pg2>. With thousands of districts under consideration, multiple comparisons must also be addressed <ref:2611. p. 1
Where the paper says it

These correlations arise even when annual estimation errors are independent, because a shared annual estimate enters the two differences with opposite signs. With thousands of districts under consideration, multiple comparisons must also be addressed.

15667#pg2>. A standard approach takes the largest one-sided p-value across the annual changes for each district and applies the Benjamini–Hochberg procedure <ref:2611. p. 1
Where the paper says it

We therefore seek to identify persistent increases while controlling the false discovery rate (FDR; Benjamini and Hochberg, 1995), the expected proportion of selected districts for which the claim of persistent widening is false. A standard approach takes the largest one-sided p-value across the annual changes for each district (Benjamini and Heller, 2008) and applies the Benjamini–Hochberg procedure (Benjamini and Hochberg, 1995).

15667#pg2>. This approach leaves information beyond the weakest change unused and does not adapt the ranking to the distribution of change vectors across districts <ref:2611. p. 1
Where the paper says it

Although this approach controls FDR under independence across districts, its ranking depends only on the weakest standardized change. It therefore leaves information beyond the weakest change unused and does not adapt the ranking to the distribution of change vectors across districts.

15667#pg2>.

How the training signal is built

The empirical Bayes mirror procedure is developed to select units whose mean vectors have all positive components <ref:2611.15667#pg4>. The procedure learns the distribution of effect vectors across units and uses estimated posterior null probabilities to rank candidate discoveries <ref:2611. p. 2

Where the paper says it

In this paper, we develop an empirical Bayes mirror procedure for selecting units whose mean vectors have all positive components. The procedure learns the distribution of effect vectors across units and uses estimated posterior null probabilities to rank candidate discoveries.

15667#pg5>. To calibrate selection, reflected versions of each observation are constructed using its error covariance <ref:2611. p. 2
Where the paper says it

The procedure learns the distribution of effect vectors across units and uses estimated posterior null probabilities to rank candidate discoveries. To calibrate selection, we construct reflected versions of each observation using its error covariance.

15667#pg5>. These comparisons classify observations as candidate discoveries or controls, with a null observation at least as likely to be a control as a candidate discovery <ref:2611. p. 2
Where the paper says it

To calibrate selection, we construct reflected versions of each observation using its error covariance. The resulting comparisons classify observations as candidate discoveries or controls, with a null observation at least as likely to be a control as a candidate discovery.

15667#pg5>. The procedure achieves finite-sample FDR control under a Gaussian observation model without requiring accurate estimation of the posterior null probabilities used for ranking <ref:2611. p. 0
Where the paper says it

It uses an estimated distribution of effect vectors to guide ranking and covariance-adjusted mirror observations to calibrate selection. The procedure achieves finite-sample false discovery rate (FDR) control under a Gaussian observation model, without requiring accurate estimation of the posterior null probabilities used for ranking.

15667#pg5>.

Where the gains come from

The core mechanism involves constructing mirror transformations that account for correlations among coordinates and support conservative FDR calibration <ref:2611.15667#pg6>. The map changes xj to −xj and adjusts the other coordinates according to their error covariances with coordinate j <ref:2611. p. 5

Where the paper says it

4) where Σ (i) ·j is the jth column of Σ (i) . The map changes xj to −xj and adjusts the other coordinates according to their error covariances with coordinate j.

15667#pg7>. A district is eligible if its observed vector or at least one of its d reflections lies in C, where C is the set of vectors with every coordinate positive <ref:2611. p. 5
Where the paper says it

A district is eligible if its observed vector or at least one of its d reflections lies in C. We define the eligible set by I = {i : X(i) ∈ C} ∪ [ d j=1 {i : R (i)[j]X(i) ∈ C}.

15667#pg6>. The selection rule uses a threshold T derived from the estimated false discovery proportion FDP([t) = 1 + N−(t) / N+(t) ∨ 1 <ref:2611. p. 0
Where the paper says it

Since academic achievement is linked to college attendance and adult earnings (Chetty, Friedman and Rockoff, 2014), identifying these districts can help locate communities where the achievement gap between disadvantaged students and their peers continues to widen. 1 arXiv:2611.

p. 0
Where the paper says it

It uses an estimated distribution of effect vectors to guide ranking and covariance-adjusted mirror observations to calibrate selection. The procedure achieves finite-sample false discovery rate (FDR) control under a Gaussian observation model, without requiring accurate estimation of the posterior null probabilities used for ranking.

15667#pg8>. This leads to Theorem 3. p. 3
Where the paper says it

A district has a persistently increasing gap if every coordinate of ∆(i) is positive. This leads to the simultaneous tests H (i) 0 : min 1≤j≤d ∆ (i) j ≤ 0 versus H (i) 1 : ∆(i) j > 0 for all j.

1, which establishes finite-sample FDR control for independent Gaussian observation vectors with known, positive-definite covariance matrices satisfying Σ(i)jk ≤ 0 for j ≠ k <ref:2611. p. 2
Where the paper says it

The control count then determines where to cut off the ranking. We establish finite-sample FDR control for independent Gaussian observation vectors with known covariance matrices having nonpositive off-diagonal entries.

15667#pg5>.

Application to the SEDA data

The study applies the proposed procedure and comparison methods to the SEDA sample described in Section 2.1 <ref:2611.15667#pg2>. The results show that at q = 0. p. 1

Where the paper says it

Symmetry and exchangeability provide a basis for such calibration. Knockoff procedures (Barber and Candès, 2015; Ren and Candès, 2023) and masking methods (Lei and Fithian, 2018; Leung and Sun, 2022) show how learned rankings can be combined with finite-sample FDR control.

25, EB mirror selects eight districts and fixed mirror selects four, while BH still selects none <ref:2611. p. 0
Where the paper says it

At a target FDR of 25%, the SEDA analysis identifies eight districts in five states with evidence of widening gaps in every annual interval. Keywords: Achievement gaps; Empirical Bayes; False discovery rate; Mirror statistics; Multiple testing.

15667#pg2>. The EB mirror selections include all four fixed-mirror selections, plus Long Beach Unified, Washoe County Schools, Wake County Schools, and Cypress-Fairbanks ISD <ref:2611.15667#pg3>. These selected districts span five states <ref:2611. p. 0
Where the paper says it

Simulations demonstrate power gains from incorporating information about the distribution of effect vectors. At a target FDR of 25%, the SEDA analysis identifies eight districts in five states with evidence of widening gaps in every annual interval.

15667#pg3>. The annual median standard errors among the selected districts range from 0. p. 3
Where the paper says it

states, form our analysis sample. The annual standard errors range from 0.

014 to 0.016, compared with 0. p. 5
Where the paper says it

6) as the observation reconstructs the same list. We assign a common score a (i) to this list, so a candidate discovery and its possible controls are compared at the same threshold.

092 to 0.096 in the full sample <ref:2611. p. 0
Where the paper says it

Since academic achievement is linked to college attendance and adult earnings (Chetty, Friedman and Rockoff, 2014), identifying these districts can help locate communities where the achievement gap between disadvantaged students and their peers continues to widen. 1 arXiv:2611.

15667#pg3>.

Discussion

The empirical Bayes mirror procedure allows information to be pooled across units while retaining finite-sample FDR control under the observation model in Section 2.2 <ref:2611. p. 0

Where the paper says it

Since academic achievement is linked to college attendance and adult earnings (Chetty, Friedman and Rockoff, 2014), identifying these districts can help locate communities where the achievement gap between disadvantaged students and their peers continues to widen. 1 arXiv:2611.

p. 0
Where the paper says it

Since academic achievement is linked to college attendance and adult earnings (Chetty, Friedman and Rockoff, 2014), identifying these districts can help locate communities where the achievement gap between disadvantaged students and their peers continues to widen. 1 arXiv:2611.

15667#pg2>. The SEDA analysis illustrates the importance of studying how achievement gaps change over successive years <ref:2611. p. 0
Where the paper says it

At a target FDR of 25%, the SEDA analysis identifies eight districts in five states with evidence of widening gaps in every annual interval. Keywords: Achievement gaps; Empirical Bayes; False discovery rate; Mirror statistics; Multiple testing.

15667#pg2>. The resulting selections identify communities with evidence of sustained divergence between economically disadvantaged students and their peers during the recovery period <ref:2611. p. 0
Where the paper says it

cn Abstract Unequal recovery from pandemic learning losses raises concerns about widening achievement gaps between economically disadvantaged students and their peers. This study seeks to identify districts whose mathematics gaps widened in every annual interval from 2022 to 2025.

15667#pg4>. Further research directions include modeling common trajectory patterns for efficiency and quantifying the effect of covariance estimation on comparisons <ref:2611.15667#pg4>. Extending calibration to accommodate dependence across units or non-Gaussian errors could broaden its applicability <ref:2611.15667#pg4>.

References

Barber, R. F. and Candès, E. p. 1

Where the paper says it

Symmetry and exchangeability provide a basis for such calibration. Knockoff procedures (Barber and Candès, 2015; Ren and Candès, 2023) and masking methods (Lei and Fithian, 2018; Leung and Sun, 2022) show how learned rankings can be combined with finite-sample FDR control.

p. 1
Where the paper says it

Symmetry and exchangeability provide a basis for such calibration. Knockoff procedures (Barber and Candès, 2015; Ren and Candès, 2023) and masking methods (Lei and Fithian, 2018; Leung and Sun, 2022) show how learned rankings can be combined with finite-sample FDR control.

J. (2015). Controlling the false discovery rate via knockoffs <ref:2611. p. 1
Where the paper says it

With thousands of districts under consideration, multiple comparisons must also be addressed. We therefore seek to identify persistent increases while controlling the false discovery rate (FDR; Benjamini and Hochberg, 1995), the expected proportion of selected districts for which the claim of persistent widening is false.

15667#pg19>

Benjamini, Y. and Hochberg, Y. p. 1

Where the paper says it

With thousands of districts under consideration, multiple comparisons must also be addressed. We therefore seek to identify persistent increases while controlling the false discovery rate (FDR; Benjamini and Hochberg, 1995), the expected proportion of selected districts for which the claim of persistent widening is false.

(1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing <ref:2611. p. 1
Where the paper says it

We therefore seek to identify persistent increases while controlling the false discovery rate (FDR; Benjamini and Hochberg, 1995), the expected proportion of selected districts for which the claim of persistent widening is false. A standard approach takes the largest one-sided p-value across the annual changes for each district (Benjamini and Heller, 2008) and applies the Benjamini–Hochberg procedure (Benjamini and Hochberg, 1995).

15667#pg4>

Dai, R. and Zheng, C. p. 2

Where the paper says it

The resulting comparisons must remain valid when the same data are used to learn the ranking. Related work on union null hypotheses includes simultaneous knockoffs (Dai and Zheng, 2023) and the joint mirror procedure (Deng, He and Zhang, 2024).

(2023). False discovery rate-controlled multiple testing for union null hypotheses: A knockoff-based approach <ref:2611. p. 1
Where the paper says it

Empirical Bayes methods are widely used in large-scale multiple testing. A key idea is to use information across units to rank hypotheses by the local false discovery rate (lfdr), the posterior probability that the null hypothesis is true (Efron et al.

15667#pg19>

Dewey, D. C., Fahle, E. p. 2

Where the paper says it

We use the released unshrunk gap estimates and their standard errors, with economically disadvantaged status defined by each state’s classification. The gaps are on SEDA’s cohort-standardized scale, in student standard-deviation units relative to national reference cohorts (Fahle et al.

p. 2
Where the paper says it

We use the released unshrunk gap estimates and their standard errors, with economically disadvantaged status defined by each state’s classification. The gaps are on SEDA’s cohort-standardized scale, in student standard-deviation units relative to national reference cohorts (Fahle et al.

M., Kane, T. J., Reardon, S. p. 0
Where the paper says it

We analyze the Stanford Education Data Archive (SEDA) 2025. 2 (Reardon et al.

F., and Staiger, D. O. (2025). Pivoting from Pandemic Recovery to Long-Term Reform: A District-Level Analysis <ref:2611. p. 0
Where the paper says it

ca xiayin@fudan. cn Abstract Unequal recovery from pandemic learning losses raises concerns about widening achievement gaps between economically disadvantaged students and their peers.

15667#pg19>

Fu, L., Gang, B., James, G. M., and Sun, W. p. 1

Where the paper says it

A key idea is to use information across units to rank hypotheses by the local false discovery rate (lfdr), the posterior probability that the null hypothesis is true (Efron et al. , 2001; Sun and Cai, 2007).

p. 1
Where the paper says it

A key idea is to use information across units to rank hypotheses by the local false discovery rate (lfdr), the posterior probability that the null hypothesis is true (Efron et al. , 2001; Sun and Cai, 2007).

p. 1
Where the paper says it

A key idea is to use information across units to rank hypotheses by the local false discovery rate (lfdr), the posterior probability that the null hypothesis is true (Efron et al. , 2001; Sun and Cai, 2007).

p. 1
Where the paper says it

A key idea is to use information across units to rank hypotheses by the local false discovery rate (lfdr), the posterior probability that the null hypothesis is true (Efron et al. , 2001; Sun and Cai, 2007).

(2022). Heteroscedasticity-adjusted ranking and thresholding for large-scale multiple testing <ref:2611. p. 1
Where the paper says it

These problems share a union null hypothesis that allows at least one nonpositive effect, with the remaining effects unrestricted. Empirical Bayes methods are widely used in large-scale multiple testing.

15667#pg19>

Gang, B. and Banerjee, T. p. 7

Where the paper says it

Since the reflection maps have absolute Jacobian one, the masked log-likelihood is ℓmask(G) = X i∈I log(X d r=0 f (i) G (x (i)[r] ) ) + X i /∈I log f (i) G (X(i) ). 14) We approximate G by a mixture of point masses on a finite grid, as in Jiang and Zhang (2009) and Gang and Banerjee (2025).

(2025). Large-scale multiple testing of composite null hypotheses under heteroskedasticity <ref:2611. p. 1
Where the paper says it

Empirical Bayes methods are widely used in large-scale multiple testing. A key idea is to use information across units to rank hypotheses by the local false discovery rate (lfdr), the posterior probability that the null hypothesis is true (Efron et al.

15667#pg19>

Heller, R. and Rosset, S. p. 7

Where the paper says it

Expectations are taken over both the random effect vectors and their observations, with the covariance matrices held fixed. The next theorem specializes the ordering argument in Heller and Rosset (2021, Theorem 2.

(2021). Optimal control of false discovery criteria in the two-group model <ref:2611. p. 7
Where the paper says it

11) independently across districts, where G is a probability distribution on R d . This induces a two-group model according to whether ∆(i) ∈ C.

15667#pg4>

Jiang, W. and Zhang, C. p. 2

Where the paper says it

The resulting comparisons must remain valid when the same data are used to learn the ranking. Related work on union null hypotheses includes simultaneous knockoffs (Dai and Zheng, 2023) and the joint mirror procedure (Deng, He and Zhang, 2024).

-H. (2009). General maximum likelihood empirical Bayes estimation of normal means <ref:2611. p. 7
Where the paper says it

This motivates scores that approximate the lfdr ordering while satisfying the masking requirement in Section 3. To estimate the lfdr, we fit G by maximum likelihood using only the information in M, following the multivariate empirical Bayes approach of Soloff, Guntuboyina and Sen (2025).

15667#pg19>

Lei, L. and Fithian, W. p. 1

Where the paper says it

Symmetry and exchangeability provide a basis for such calibration. Knockoff procedures (Barber and Candès, 2015; Ren and Candès, 2023) and masking methods (Lei and Fithian, 2018; Leung and Sun, 2022) show how learned rankings can be combined with finite-sample FDR control.

(2018). AdaPT: An interactive procedure for multiple testing with side information <ref:2611. p. 1
Where the paper says it

It therefore leaves information beyond the weakest change unused and does not adapt the ranking to the distribution of change vectors across districts. The same testing problem arises beyond achievement gaps whenever the goal is to identify units whose mean vectors have all positive components.

15667#pg4>

Leung, D. and Sun, W. (2022). ZAP: Z-value adaptive procedures for false discovery rate control with side information <ref:2611. p. 0

Where the paper says it

The procedure achieves finite-sample false discovery rate (FDR) control under a Gaussian observation model, without requiring accurate estimation of the posterior null probabilities used for ranking. Simulations demonstrate power gains from incorporating information about the distribution of effect vectors.

15667#pg4>

Reardon, S. F., Fahle, E. M., Ho, A. D., Shear, B. R., Saliba, J., Min, J., Shim, J. and Kalogrides, D. (2026). Stanford Education Data Archive Technical Documentation: SEDA 2025. p. 0

Where the paper says it

This study seeks to identify districts whose mathematics gaps widened in every annual interval from 2022 to 2025. The analysis uses estimates from the Stanford Education Data Archive (SEDA) for 5,600 U.

2 <ref:2611.15667#pg19>

Ren, Z. and Candès, E. (2023). Knockoffs with side information <ref:2611. p. 0

Where the paper says it

Since academic achievement is linked to college attendance and adult earnings (Chetty, Friedman and Rockoff, 2014), identifying these districts can help locate communities where the achievement gap between disadvantaged students and their peers continues to widen. 1 arXiv:2611.

15667#pg4>

Soloff, J. A., Guntuboyina, A. p. 7

Where the paper says it

This motivates scores that approximate the lfdr ordering while satisfying the masking requirement in Section 3. To estimate the lfdr, we fit G by maximum likelihood using only the information in M, following the multivariate empirical Bayes approach of Soloff, Guntuboyina and Sen (2025).

and Sen, B. p. 7
Where the paper says it

This motivates scores that approximate the lfdr ordering while satisfying the masking requirement in Section 3. To estimate the lfdr, we fit G by maximum likelihood using only the information in M, following the multivariate empirical Bayes approach of Soloff, Guntuboyina and Sen (2025).

(2025). Multivariate, heteroscedastic empirical Bayes via nonparametric maximum likelihood <ref:2611. p. 7
Where the paper says it

This motivates scores that approximate the lfdr ordering while satisfying the masking requirement in Section 3. To estimate the lfdr, we fit G by maximum likelihood using only the information in M, following the multivariate empirical Bayes approach of Soloff, Guntuboyina and Sen (2025).

15667#pg19>

Subramanian, A., Narayan, R., Corsello, S. M. et al. (2017). A next generation connectivity map: L1000 platform and the first 1,000,000 profiles <ref:2611. p. 1

Where the paper says it

In dose-response screening, one may seek genes whose mean expression increases at every dose increment for a fixed drug, cell type, and exposure time. Large drug-perturbation resources such as LINCS L1000 provide expression profiles for investigating such questions (Subramanian et al.

15667#pg4>

Sun, W. and Cai, T. p. 1

Where the paper says it

A key idea is to use information across units to rank hypotheses by the local false discovery rate (lfdr), the posterior probability that the null hypothesis is true (Efron et al. , 2001; Sun and Cai, 2007).

T. (2007). Oracle and adaptive compound decision rules for false discovery rate control <ref:2611. p. 0
Where the paper says it

It uses an estimated distribution of effect vectors to guide ranking and covariance-adjusted mirror observations to calibrate selection. The procedure achieves finite-sample false discovery rate (FDR) control under a Gaussian observation model, without requiring accurate estimation of the posterior null probabilities used for ranking.

15667#pg4>

U.S. Census Bureau (2024). p. 0

Where the paper says it

Pandemic learning losses were larger among children from lower socioeconomic backgrounds (Betthäuser et al. , 2023), and the average within-district socioeconomic mathematics gap in the United States increased from 2022 to 2024 (Dewey et al.

2024 Cartographic Boundary Files <ref:2611. p. 0
Where the paper says it

Since academic achievement is linked to college attendance and adult earnings (Chetty, Friedman and Rockoff, 2014), identifying these districts can help locate communities where the achievement gap between disadvantaged students and their peers continues to widen. 1 arXiv:2611.

15667#pg19>

Zhao, Z. and Sun, W. (2025a). A conformalized empirical Bayes method for multiple testing with side information <ref:2611. p. 1

Where the paper says it

Empirical Bayes methods are widely used in large-scale multiple testing. A key idea is to use information across units to rank hypotheses by the local false discovery rate (lfdr), the posterior probability that the null hypothesis is true (Efron et al.

15667#pg19>

Zhao, Z. and Sun, W. (2025b). False discovery rate control for structured multiple testing: Asymmetric rules and conformal Q-values <ref:2611. p. 0

Where the paper says it

At a target FDR of 25%, the SEDA analysis identifies eight districts in five states with evidence of widening gaps in every annual interval. Keywords: Achievement gaps; Empirical Bayes; False discovery rate; Mirror statistics; Multiple testing.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Empirical Bayes mirror procedure for selecting units with all positive components can be used to identify persistent widening of achievement gaps in U.S. school districts, achieving a finite sample false discovery rate FDR control without requiring accurate estimation of posterior null probabilities used for ranking.

  2. The system can rank candidate discoveries using an empirical Bayes score based on the posterior probability of the union null, which is estimated by fitting a mixture distribution on a finite grid to masked data from eligible districts.

  3. The AI system can perform selection by setting a threshold T based on the comparison of candidate discoveries and controls, ensuring that a null observation at least as likely to be a control as a candidate discovery holds without knowing which mean component is nonpositive.

  4. The system can estimate the mixing distribution G by maximizing a masked log-likelihood using only information in M, and then construct an empirical Bayes score where Larger scores correspond to smaller lfdr values at the all-positive vectors x(i)[0].

  5. The AI system can compare different selection methods, such as EB mirror, fixed mirror, and full-data NPMLE, to determine which method provides better power when learning the distribution of effect vectors is incorporated into the ranking.

Abstract

Unequal recovery from pandemic learning losses raises concerns about widening achievement gaps between economically disadvantaged students and their peers. This study seeks to identify districts whose mathematics gaps widened in every annual interval from 2022 to 2025. The analysis uses estimates from the Stanford Education Data Archive (SEDA) for 5,600 U.S. school districts. Identifying such districts requires accounting for correlated errors and differing precision in the estimated annual changes, as well as multiplicity. The true annual gap changes in each district form an effect vector, and persistent widening means that every component is positive. To address this class of multiple testing problems, we propose an empirical Bayes mirror procedure. It uses an estimated distribution of effect vectors to guide ranking and covariance-adjusted mirror observations to calibrate selection. The procedure achieves finite-sample false discovery rate (FDR) control under a Gaussian observation model, without requiring accurate estimation of the posterior null probabilities used for ranking. Simulations demonstrate power gains from incorporating information about the distribution of effect vectors. At a target FDR of 25%, the SEDA analysis identifies eight districts in five states with evidence of widening gaps in every annual interval.

Sources

Related papers