Separate versus Pooled Winsorization in Difference-in-Differences: Bias, Efficiency, and Finite-Sample Guarantees

arXiv:2606.15097 · stat.ME, stat.AP · Submitted 2026-06-13 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Separate versus Pooled Winsorization in Difference-in-Differences".

Jane: Here's the gist of the paper. Comparing group means is foundational to many statistical areas, including two-sample studies, randomized trials, and difference-in-differences designs; a common remedy is to winsorize the data before estimating the target mean contrast The core theoretical finding The paper proves that separate winsorization can yield a sub-Gaussian estimator for general linear contrasts of group means, whereas pooled winsorization cannot attain such a guarantee in general The authors study whether the deviation ∆ˆ − ∆ satisfies a sub-Gaussian high-probability bound Simulation studies confirm that separate winsorization remains nearly unbiased and concentrates well around the truth, while pooled winsorization can have substantial bias This supports the recommendation to winsorize within each group rather than after pooling Impossibility of Pooled Winsorization The authors establish an impossibility result for pooled winsorization, showing that no deterministic rule for selecting the pooled winsorization level can attain the sub-Gaussian rate Specifically, Theorem 2.2 demonstrates that under a balanced group design with n0 = n1 = m, there exist distributions P0, P1 in P(σ2), such that for every deterministic rule ε, the lower bound in (4) holds This implies that pooled winsorization cannot yield a sub-Gaussian mean estimator The proof involves constructing distributions where the pooled order statistics segregate by group, leading to a dichotomy where neither weak nor strong winsorization admits a sub-Gaussian bound Guarantees for Separate Winsorization In contrast to pooled winsorization, separate winsorization attains the sub-Gaussian rate Theorem 2.3 shows that for any fixed c > 1, there exists a deterministic rule of ε for ∆ˆ (sep) that achieves the desired order The deviation bound is given by P(∆ˆ (sep) − ∆ ≤ Lσ s log(48/δ) nmin) ≥ 1 − δ This result extends to the general linear contrast ∆ defined in (1) with K ≥ 2 groups, showing that the deviation radius grows at most linearly with the number of groups through the factor K Practical Performance and Extensions Numerical results confirm these theoretical findings, showing that separate winsorization performs similarly to the naive estimator under Gaussian outcomes and substantially improves RMSE under heavy-tailed outcomes, especially for Pareto and lognormal designs Table 1 shows that pooled winsorization introduces substantial negative bias, particularly in imbalanced designs, and its estimates often fall outside the theoretical 95% CLT interval The extension to multi-group mean contrasts confirms that separate winsorization retains the same sub-Gaussian rate when performed within groups This highlights that winsorization should also be carried out within groups for any multi-group mean contrast The general mean contrast formulation in (1) covers several common multigroup comparison problems, including difference-in-differences and multi-arm treatment comparisons Technical Derivations The proof for the separate winsorization bound relies on a decomposition technique for the winsorized sample mean from

8: The deviation is bounded using triangle inequality across three components: (I) involving threshold perturbation, (II) involving empirical and population thresholds, and (III) involving the difference between population and empirical means Lemma B.3 demonstrates that the empirical winsorized mean concentrates around its expected value with high probability The final bound in Theorem 4.1 is achieved by applying a union bound over all groups, yielding an event of probability at least 1 − δ/6 on which the deviation is bounded by L = Cgrp This establishes the sub-Gaussian rate for multi-group contrasts The proof also details Lemma B.4, which provides a deterministic bound on the winsorization bias, guaranteeing that systematic error scales at most with the square root of the winsorization level The final step uses Lemma B.5 to bound the distance between a population quantile and the mean This result plays a crucial role in designing our proposed winsorization level Supplementary Material The supplementary material provides technical support for the manuscript, including proofs of Theorems 2.2, 2.3, and 4.1 Appendix D presents additional simulation studies examining the sensitivity of both estimators to the choice of winsorization level ε Figure 1 summarizes results for the balanced group design, showing separate winsorization has negligible absolute bias across all tested epsilon values Figure 2 reports results for the imbalanced group design, which are qualitatively similar to those in the balanced design The simulation data further suggests that separate winsorization remains stable across the range of ε values, while pooled winsorization becomes increasingly biased as ε grows These results reinforce the theoretical message that winsorization should be performed separately within groups The simulation studies also show that separate winsorization generally maintains high containment, while pooled winsorization can have substantially lower containment when the winsorization level is above 0.05 The theoretical calibration of ε from Theorem 2.3 is shown to be empirically advantageous for both estimators The paper concludes that separate winsorization is the preferred method for robust estimation in group mean contrasts The results are supported by extensive simulation studies and rigorous finite-sample theory.

Title and authors: Tom: So we're looking at this paper titled "Separate versus Pooled Winsorization in Difference-in-Differences: Bias, Efficiency, and Finite-Sample Guarantees." It sounds like they're digging into how to make sure our estimates for group means are really robust when the data is heavy-tailed.

Jane: Yeah, so the core question here is whether it matters if you calculate those winsorization thresholds once for all groups together, or once for each group separately before doing your main comparison.

Lu: Basically, they're looking at two strategies: pooled winsorization, which computes thresholds from the combined sample across all groups, and separate winsorization, where you compute them within each group individually.

Meng: I mean when we're shipping models to production on real traffic, I always worry about those pooled things introducing a lot of bias if one group is really weird compared to the other.

Lalam: The paper sets up this comparison because conventional estimators, like the naive sample mean difference, just don't behave well enough in these heavy-tailed settings for finite samples >.

The paper's summary: Tom: The main finding they push is that separate winsorization can give you a sub-Gaussian estimator for general linear contrasts of group means, but pooled winsorization just can't guarantee that same level of performance in general.

Jane: That’s a big deal because it means there’s a structural reason why pooling those thresholds together isn't as reliable as treating each group independently.

Lu: They prove an impossibility result for pooled winsorization, showing that no single deterministic rule for picking the pooled winsorization level can achieve that sub-Gaussian rate.

Meng: So, if you're trying to deploy a model and you have different user segments, this suggests you should definitely be winning those thresholds within each segment rather than just using one global threshold for everyone.

Lalam: The simulation studies back this up by showing separate winsorization stays nearly unbiased and concentrates well around the truth, while pooled winsorization can introduce substantial bias depending on the design.

The paper's improvements: Tom: One of the main contributions is that they provide a concrete bound for separate winsorization, showing it attains this sub-Gaussian rate even when you have multiple groups involved in a general linear contrast.

Jane: They extend this beyond just the simplest two-group comparison to cover more complex things like difference-in-differences estimators and multi-arm treatment comparisons, which is what we see in many real applications.

Lu: The proof for the separate winsorization bound uses a decomposition technique involving threshold perturbation and empirical versus population thresholds, ultimately showing that systematic error scales at most linearly with the number of groups.

Meng: That linear scaling with the number of groups is good because it means as you add more segments to your analysis, the risk doesn't explode; it just grows predictably.

Lalam: They also give a deterministic bound on the winsorization bias, which guarantees that systematic error scales at most with the square root of your winsorization level, which helps us design a practical threshold.

Conclusion: Tom: So to wrap up, the paper "Separate versus Pooled Winsorization in Difference-in-Differences: Bias, Efficiency, and Finite-Sample Guarantees" strongly recommends using separate winsorization within each group for robust estimation of group mean contrasts.

Jane: It seems the key message is that pooling those thresholds introduces substantial negative bias, especially in imbalanced designs where one group has very different data than the others.

Lu: The paper confirms that for any multi-group mean contrast, winsorization should also be carried out within groups because separate winsorization retains the same sub-Gaussian rate when performed this way.

Meng: From an engineering standpoint, it means we need to build our pipelines to handle group-specific data distributions separately if we want stable estimates at scale.

Lalam: This work gives us a firm theoretical calibration for choosing that winsorization level epsilon, which is empirically advantageous for both estimators, so it’s the preferred method for robust estimation in these settings.

Washington University in St Louis · University of Colorado Boulder · Suffolk University

stat.ME, stat.AP

Submitted: 2026-06-13

Updated: 2026-11-14

License: http://creativecommons.org/publicdomain/zero/1.0/

Importance score: 86/100

The gist: Comparing group means is foundational to many statistical areas, including two-sample studies, randomized trials, and difference-in-differences designs…

Key concepts

Separate Winsorization
This technique involves applying different winzor levels to the data within each individual group separately. The paper shows this method successfully achieves a sub-Gaussian estimator, meaning the error in the estimate is tightly controlled with high probability, regardless of whether the underlying data is Gaussian or heavy-tailed.
Pooled Winsorization
This method applies a single winzor level to all data combined across all groups before estimating the mean. The authors prove this approach cannot guarantee a sub-Gaussian rate. Instead, pooled winsorization can introduce substantial bias, especially in imbalanced designs, making it less reliable.
Sub-Gaussian Estimator
This is a strong statistical guarantee indicating that the estimator's deviation from the true value is bounded by an exponential function of the standard deviation. Achieving this rate means the estimator concentrates very tightly around the truth with high probability, which is a desirable property for reliable statistical inference.

Terminology

Summary

Comparing group means is foundational to many statistical areas, including two-sample studies, randomized trials, and difference-in-differences designs; a common remedy is to winsorize the data before estimating the target mean contrast

The core theoretical finding

The paper proves that separate winsorization can yield a sub-Gaussian estimator for general linear contrasts of group means, whereas pooled winsorization cannot attain such a guarantee in general The authors study whether the deviation ∆ˆ − ∆ satisfies a sub-Gaussian high-probability bound Simulation studies confirm that separate winsorization remains nearly unbiased and concentrates well around the truth, while pooled winsorization can have substantial bias This supports the recommendation to winsorize within each group rather than after pooling

Impossibility of Pooled Winsorization

The authors establish an impossibility result for pooled winsorization, showing that no deterministic rule for selecting the pooled winsorization level can attain the sub-Gaussian rate Specifically, Theorem 2.2 demonstrates that under a balanced group design with n0 = n1 = m, there exist distributions P0, P1 in P(σ2), such that for every deterministic rule ε, the lower bound in (4) holds This implies that pooled winsorization cannot yield a sub-Gaussian mean estimator The proof involves constructing distributions where the pooled order statistics segregate by group, leading to a dichotomy where neither weak nor strong winsorization admits a sub-Gaussian bound

Guarantees for Separate Winsorization

In contrast to pooled winsorization, separate winsorization attains the sub-Gaussian rate Theorem 2.3 shows that for any fixed c > 1, there exists a deterministic rule of ε for ∆ˆ (sep) that achieves the desired order The deviation bound is given by P(∆ˆ (sep) − ∆ ≤ Lσ s log(48/δ) nmin) ≥ 1 − δ This result extends to the general linear contrast ∆ defined in (1) with K ≥ 2 groups, showing that the deviation radius grows at most linearly with the number of groups through the factor K

Practical Performance and Extensions

Numerical results confirm these theoretical findings, showing that separate winsorization performs similarly to the naive estimator under Gaussian outcomes and substantially improves RMSE under heavy-tailed outcomes, especially for Pareto and lognormal designs Table 1 shows that pooled winsorization introduces substantial negative bias, particularly in imbalanced designs, and its estimates often fall outside the theoretical 95% CLT interval The extension to multi-group mean contrasts confirms that separate winsorization retains the same sub-Gaussian rate when performed within groups This highlights that winsorization should also be carried out within groups for any multi-group mean contrast The general mean contrast formulation in (1) covers several common multigroup comparison problems, including difference-in-differences and multi-arm treatment comparisons

Technical Derivations

The proof for the separate winsorization bound relies on a decomposition technique for the winsorized sample mean from [8] The deviation is bounded using triangle inequality across three components: (I) involving threshold perturbation, (II) involving empirical and population thresholds, and (III) involving the difference between population and empirical means Lemma B.3 demonstrates that the empirical winsorized mean concentrates around its expected value with high probability The final bound in Theorem 4. p. 1

Where the paper says it

Specifically, we study whether the deviation |∆ˆ − ∆| satisfies a sub-Gaussian high-probability bound. Finite-sample guarantees for mean estimation have been extensively studied in the single-group setting [2, 4, 10, 8].

1 is achieved by applying a union bound over all groups, yielding an event of probability at least 1 − δ/6 on which the deviation is bounded by L = Cgrp This establishes the sub-Gaussian rate for multi-group contrasts The proof also details Lemma B. p. 4
Where the paper says it

We also extend the key idea and core proof strategy of separate winsorization to multi-group mean contrasts setting, and we show that, under this extension, a sub-Gaussian estimator can still be obtained by performing winsorization separately within each group. Details of this extension are provided in Section 4.

4, which provides a deterministic bound on the winsorization bias, guaranteeing that systematic error scales at most with the square root of the winsorization level The final step uses Lemma B. p. 5
Where the paper says it

3], showing that separate winsorization remains stable while pooled winsorization is more sensitive to the winsorization level. Based on 5,000 Monte Carlo replications, we summarize performance across the three estimators in terms of bias, root mean squared error (RMSE), and containment in the theoretical 95% CLT confidence interval for the naive estimator.

5 to bound the distance between a population quantile and the mean This result plays a crucial role in designing our proposed winsorization level

Supplementary Material

The supplementary material provides technical support for the manuscript, including proofs of Theorems 2.2, 2.3, and 4.1 Appendix D presents additional simulation studies examining the sensitivity of both estimators to the choice of winsorization level ε Figure 1 summarizes results for the balanced group design, showing separate winsorization has negligible absolute bias across all tested epsilon values Figure 2 reports results for the imbalanced group design, which are qualitatively similar to those in the balanced design The simulation data further suggests that separate winsorization remains stable across the range of ε values, while pooled winsorization becomes increasingly biased as ε grows These results reinforce the theoretical message that winsorization should be performed separately within groups The simulation studies also show that separate winsorization generally maintains high containment, while pooled winsorization can have substantially lower containment when the winsorization level is above 0. p. 5

Where the paper says it

3], showing that separate winsorization remains stable while pooled winsorization is more sensitive to the winsorization level. Based on 5,000 Monte Carlo replications, we summarize performance across the three estimators in terms of bias, root mean squared error (RMSE), and containment in the theoretical 95% CLT confidence interval for the naive estimator.

05 The theoretical calibration of ε from Theorem 2. p. 1
Where the paper says it

We provide the first answer to this question for winsorized estimators, showing that separate winsorization can yield a sub-Gaussian estimator, whereas pooled winsorization cannot achieve such a guarantee in general. To the best of our knowledge, the choice between the two procedures has received scarce theoretical attention despite the practical prevalence of pooled winsorization.

3 is shown to be empirically advantageous for both estimators The paper concludes that separate winsorization is the preferred method for robust estimation in group mean contrasts The results are supported by extensive simulation studies and rigorous finite-sample theory. p. 0
Where the paper says it

Separate versus pooled winsorization for group mean contrasts: a finite-sample theory Chao Cheng Department of Statistics and Data Science Washington University in St. Louis, MO, USA chaoc@wustl.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Robust estimation of group mean contrasts can be achieved by using separate winsorization within each group rather than pooled winsorization across all groups, as separate winsorization attains this rate while pooled winsorization cannot achieve such a guarantee in general.

  2. The improved AI system can estimate linear group mean contrasts in multi-group settings with sub-Gaussian high-probability bounds, even under heavy-tailed data, by employing separate winsorization thresholds for each group.

  3. The system will be able to handle complex statistical problems like difference-in-differences and multi-arm treatment comparisons by extending the separate winsorization strategy to general linear contrasts as shown in Theorem 4.1, ensuring that for any multi-group mean contrast, winsorization should also be carried out within groups.

Abstract

Problem Definition: Extreme outcomes are common in operational data, and empirical difference-in-differences (DiD) studies often winsorize them to limit their influence. Yet winsorization can change both the estimated treatment effect and its sampling uncertainty. We ask when winsorization improves DiD estimation and whether cutoffs should be computed separately within each group-period cell or from observations pooled across cells. Methodology/Results: We develop finite-sample and asymptotic analyses in a two-group, two-period DiD setting. For continuous outcomes with finite variance, separate winsorization achieves a uniform sub-Gaussian deviation guarantee under a calibrated winsorization fraction, while neither raw DiD nor pooled winsorization can achieve the same guarantee. We identify conditions under which separate winsorization is asymptotically unbiased, whereas pooled winsorization with a fixed fraction may remain biased. Under these conditions, standard tail classifications alone do not establish an efficiency ranking. Yet a two-sided decreasing-failure-rate (DFR) condition on each cell's error distribution can guarantee strictly smaller asymptotic variance for separate winsorization than for raw DiD at every fixed winsorization fraction. Numerical results support both analyses. Managerial Implications: Our results guide practitioners on whether and how to winsorize outcomes in DiD studies, clarifying the advantages and limitations of raw DiD, separate winsorization, and pooled winsorization. The DFR condition arising from our efficiency analysis offers a simple diagnostic for when winsorization can be expected to lower asymptotic variance while keeping the treatment-effect estimate unbiased, giving theoretical grounding to a practice that has so far rested mainly on convention.

Related papers