Anchored multiple testing: a transparent use of e-closure to improve FDR procedures

The recent e-closure method can recover every procedure that controls FDR (and other expectation losses). But the recovered e-collection is “self-referential” and gives no insight on how to improve the procedure (if improvable), and some recent improvements have been somewhat opaque. We introduce an elementary new technique called anchoring that exploits looseness in existing FDR proofs to enlarge a baseline multiple-testing procedure’s self-referential local e-value. The resulting e-closure thus transparently retains every baseline discovery (and usually adding more) and controlling the false discovery rate under the same conditions as the baseline.

The main contributions are:

- Anchored-BH dominates 1 BH under PRDS and is incomparable in general with canonical closed BH.

- Anchored-BY dominates BY under arbitrary dependence and is incomparable in general with closed BY.

- Anchored e-BH dominates e-BH and is incomparable to closed e-BH.

- Anchored-dBH improves the conditional-calibration procedure of Fithian and Lei (2022).

- Anchored ordered testing exploits a different looseness.

The anchoring principle is generic: For any baseline rejection set, suppose a deterministic function satisfies E[FDP(A)/alpha] ≤ qS whenever all hypotheses in S are true. For intersection e-values, define ES = FDP(A)/alpha + (1 - qS)FS. This is a local e-value, and every measurable certified superset of A controls FDR at α and contains all of A’s discoveries.

The paper demonstrates improvements to various procedures:

- Anchored-BH dominates BH under PRDS.

- Anchored-2BH dominates the union of BH at level α/2 in each Gaussian tail, under the appropriate covariance restrictions.

- Anchored-BY dominates BY under arbitrary dependence.

The paper concludes that while the new procedures are not necessarily admissible, they can safely replace earlier methods whenever those were employed, offering simplicity and generality. The computational complexity for anchored methods is generally in sorting or quadratic time.

Relevant excerpts include:

Anchored multiple testing: A transparent use of e-closure to improve FDR procedures.

The resulting e-closure thus transparently retains every baseline discovery (and usually adding more) and controlling the false discovery rate under the same conditions as the baseline. (Abstract)

Anchored-BH dominates 1 BH under PRDS and is incomparable in general with canonical closed BH. (Summary of main contributions)

The anchoring principle also applies directly to e-BH Wang and Ramdas (2022). (Section 7)

Corollary C.2: Under (68), the values in (73) are local e-values. Every certified superset B¯− obtained by e-closure satisfies B¯− ⊇ BHbeta (p) pointwise, FDR(B¯−) ≤ α. (Section C.3)

The coefficient 1 − S/m is not universal: every extension needs its own budget proof. (Discussion)

Anchored multiple testing: A transparent use of e-closure to improve FDR procedures. (Overall theme)

""

1. Anchored multiple testing procedure for Benjamini Hochberg (BH) under PRDS p-values can recover every baseline discovery while controlling FDR at alpha, as shown by Anchored-BH dominates 1 BH under PRDS.

2. The system can utilize a generic form of adaptivity to unknown null proportions by using the local e-value formula: E S = FDP S / alpha + (1 - S / m) F S.

3. For two-sided Gaussian p-values, the system can employ Anchored-2BH to dominate the union of BH at level alpha/2 in each Gaussian tail under appropriate covariance restrictions.

4. The AI system can improve dependence-adjusted BH procedures by using Anchored e-BH, which dominates e-BH and is incomparable to closed e-BH.

5. The system can perform ordered testing enhancements using Anchored ordered testing, which exploits a different looseness and retains the baseline certificate even when the baseline is empty.

6. For negative dependence, the system can use Anchored-NDBH to control FDR under conditions where "every true intersection S satisfies E FDP S (B-) <= alpha qS".

7. The AI system can implement a joint calibration extension of Anchored-BH by using the formula E band S = E B S + (1 - n/m) Sum over i in S x S, i, d n >= 0, which is a valid e-collection pointwise dominating (E B S).

8. For ordered procedures like SeqStep+, the system can use conditional residual anchoring to retain the baseline and control FDR at alpha by defining a local e-collection E S = 1/alpha Sum over i in S a i.