Testing Sign Agreement

arXiv:2608.10294 · econ.EM · Submitted 2026-08-10 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.

Ruth: We're looking at the paper: "Testing Sign Agreement".

Warren: Here's the gist of the paper. This article develops two novel tests for testing sign agreement among a finite number of parameters, which arises in empirical settings such as detecting treatment effects with opposite signs across subgroups, outcomes, or time periods.

Ruth: The null hypothesis (H 0) is that all parameters are either all nonnegative or all nonpositive: " H 0: mu j at least 0 for all j = 1,..., k or mu j at most 0 for all j = 1,..., k " (1.1).

Warren: The alternative hypothesis (H 1) is that the parameters disagree in sign: " H 1: there exist i not equal to j such that mu i > 0 and mu j < 0." The authors propose two distinct procedures for testing this null hypothesis: a least favorable test and a conditional test.

Ruth: The least favorable test is constructed using the worst-case null critical value, while the conditional test first screens components with large positive or negative estimates and then tests the remaining sign-unresolved components conditionally on that screening event.

Paper summary: Ruth: So, looking at the paper "Testing Sign Agreement," it’s really about providing two distinct ways to check if a set of parameters all have the same sign or if they disagree.

Warren: The authors propose the least favorable test, which uses a worst-case critical value, and the conditional test, which screens big values first and then tests what's left.

Iris: The main contribution here is proving uniform asymptotic validity for both tests over a wide class of nonparametric distributions. This means their size control is strong even when we don't assume the parameters follow a simple normal distribution.

Ruth: It matters because this moves beyond just pointwise validity, which can be too restrictive in finite samples and under certain conditions.

Warren: The paper suggests that neither test is better than the other; one wins near boundary configurations where signs are tightly constrained, and the other wins when parameters are well separated from zero.

Iris: The authors recommend using the least favorable test for two parameters and the conditional test as a default choice when you have three or more parameters.

Ruth: What this means for us is that whether you're looking at treatment effects across different groups or time points, these tests give us a robust way to tell if we've found evidence of opposing signs.

Warren: It’s a practical tool for researchers because it handles the dependence between your estimators better than many existing methods, even though implementing the critical values requires numerical work.

Conclusion: Ruth: So we're wrapping up this look at the paper "Testing Sign Agreement." It’s basically about finding two specific ways to check if a bunch of parameters all have the same sign or if they disagree.

Warren: Right, and it uses these two methods: one is called the least favorable test, which looks at the absolute worst-case scenario, and the other is a conditional test that screens out big estimates first.

Iris: The authors are showing that both of these procedures work well even when we don't know much about the underlying data distribution. They prove they have uniform asymptotic validity over a huge range of possibilities.

Ruth: That means their tests aren't just good for one specific kind of data; they hold up generally, which is important when you’re working with real-world empirical results from things like treatment effects.

Warren: And the simulation study backs this up, showing that the actual rejection rates stay close to what we expect even in different experimental setups. It doesn't just work on paper; it works in practice.

Iris: I think the main thing here is how these tests handle dependence between your different estimators, which is a common problem when you have multiple things you're trying to measure at once.

Ruth: Exactly. And they give us some practical advice too, suggesting the least favorable test for two parameters and the conditional test for more parameters. It’s a guide on which tool to use when you’re dealing with sign tests.

Warren: It's a solid tool for researchers because it offers a way to reliably detect if you have conflicting signs in your analysis. But we still need to remember that implementing those critical values requires some numerical work, which is the practical hurdle there.

University of Warwick

econ.EM

Submitted: 2026-08-10

Updated: 2026-11-15

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 92/100

The gist: This article develops two novel tests for testing sign agreement among a finite number of parameters, which arises in empirical settings such as detecting treatment effects with opposite signs across…

Key concepts

Null Hypothesis ($H_0$)
This hypothesis assumes that all tested parameters ($\mu_j$) are either entirely non-negative (all $\geq 0$) or entirely non-positive (all $\leq 0$). It represents the situation where there is no disagreement in the signs of the parameters being tested.
Least Favorable Test
This test is constructed using a worst-case critical value. It aims to be robust by testing against the most difficult scenario under which the null hypothesis might still be true, making it reliable when sign restrictions are tight.
Conditional Test
This procedure first screens parameters with large positive or negative estimates. Then, it tests the remaining parameters based on whether that initial screening event occurred. This method is effective when several components are far from zero, as it simplifies the problem through dimensional reduction.

Terminology

Summary

This article develops two novel tests for testing sign agreement among a finite number of parameters, which arises in empirical settings such as detecting treatment effects with opposite signs across subgroups, outcomes, or time periods. p. 0 The null hypothesis (H 0) is that all parameters are either all nonnegative or all nonpositive: H 0: mu j at least 0 for all j = 1,..., k or mu j at most 0 for all j = 1,..., k (1.1). The alternative hypothesis (H 1) is that the parameters disagree in sign: " H 1: there exist i not equal to j such that mu i > 0 and mu j < 0."

The authors propose two distinct procedures for testing this null hypothesis: a least favorable test and a conditional test. The least favorable test is constructed using the worst-case null critical value, while the conditional test first screens components with large positive or negative estimates and then tests the remaining sign-unresolved components conditionally on that screening event. p. 0 Both procedures are capable of accommodating arbitrary dependence among estimators, and they can be implemented using various statistics such as the maximum, sum-of-squares, or quasi-likelihood ratio statistic. p. 7

The theoretical foundation establishes the uniform asymptotic validity of both tests in Theorem 1 and Theorem 2, providing results for both studentized sample means and general regular estimators under specified conditions (Assumption 1). p. 2 The simulation study supports these theoretical findings, demonstrating that empirical maximum null rejection probabilities remain close to the nominal significance level across various designs. p. 3

The comparison of power reveals a systematic tradeoff: the least favorable test is more powerful near boundary configurations where sign restrictions bind, whereas the conditional test exhibits greater power gains when several components are well-separated from zero due to dimensional reduction in Step II. p. 0 The authors recommend the least favorable test for k=2 and the conditional test as a default choice for k at least 3. p. 0

The uniform asymptotic validity of the tests is established using a nonparametric bootstrap critical value in (4. p. 99) for Theorem 1, where it is shown that the limiting rejection probability of the conditional test at B=0 is bounded by gamma = alpha - tau, and for each likelihood parameter in the set of tests, alpha to infinity n to infinity E P[psi n] at most tau + gamma = alpha. p. 2 The proof relies on convergence results such as x in R J n(x, sqrt n mu n(P n), P n hat) - F,I(x) to 0 for all asymptotically relevant A I, and the result that if a uniform lower bound on the smallest eigenvalue of the correlation matrix is positive, then P n[omega n hat 0] to 1. p. 1

The comparison with intersection–union adaptations of moment-inequality procedures shows that no single procedure dominates uniformly. p. 3 Furthermore, the screening-and-conditioning principle underlying the conditional test is not specific to sign agreement and may also be useful for one-sided moment-inequality testing, where clearly slack moments could be screened out before testing the remaining, potentially binding inequalities conditional on the screening event. p. 3 Other extensions include allowing the number of parameters to increase with sample size or extending the framework to an infinite collection of parameters. p. 0 Implementation under a general correlation matrix requires numerical evaluation of least favorable and conditional critical values. p. 0

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Implement novel tests for sign agreement among finite parameters using a least favorable test that uses a worst-case null critical value to control asymptotic size uniformly over a large class of nonparametric distributions.

  2. Develop a conditional test procedure that screens components with large positive or negative estimates and then tests the remaining sign unresolved components conditionally on the screening event, improving power when some components are well separated from zero.

  3. Enhance inference for local average treatment effects by testing instrument validity using sign agreement hypotheses across correlated outcomes, accommodating arbitrary correlation structures among estimators.

  4. Apply a two-step conditional test to one-sided moment inequality problems by screening out clearly slack moments before testing the remaining inequalities under a reduced null hypothesis conditional on the screening event.

  5. Extend the framework to general regular estimators by showing that uniform asymptotic validity holds when the estimator admits a uniformly asymptotically linear representation with uniformly integrable standardized influence functions and consistently estimated marginal scales and correlation matrices.

Sources

Related papers