Winsorizing Simulator
Understand how winsorizing handles outliers and improves statistical precision
What is Winsorizing?
Winsorizing is a statistical technique for handling skewed distributions and outliers by capping extreme values at a specified percentile threshold, rather than removing them entirely. It's particularly effective for metrics with heavy tails or extreme values.
For example, with 99th percentile winsorizing:
- All values above the 99th percentile are capped to the 99th percentile value
- All other values remain unchanged
- No data points are removed, preserving sample size
Why it works: In skewed distributions, extreme outliers inflate variance and widen confidence intervals, making it harder to detect real effects. By capping these values, you reduce noise without losing the information that these data points exist.
Key Benefits
- Reduces variance: Controls the influence of extreme outliers
- Narrows confidence intervals: More precise estimates
- Preserves sample size: No data points discarded
- Improves power: Better ability to detect true effects
- Robust estimation: More stable estimates of central tendency
Choosing the Right Threshold
A lower percentile caps more values and produces greater variance reduction:
- 90-95%: Stronger CI reduction, risks introducing bias
- 99%: Conservative, only addresses extreme outliers
- Common practice: 95th-99th percentile is typical
How to Apply in A/B Tests
Calculate the percentile threshold using the combined population (both control and treatment together), then apply this single threshold to all data.
- Do: Pool all data, find the 99th percentile, cap both groups at that value
- Don't: Calculate separate thresholds for control and treatment
Since both groups share the same threshold, winsorizing can affect each group's mean differently depending on their outlier distribution. A group with more extreme outliers will see a larger reduction in mean.
When to Use
- Revenue per user with whale customers
- Session duration with tabs left open
- Ad spend with high-value campaigns
- Time-to-complete with extreme outliers
- Retention metrics with unusual engagement
When NOT to Use
- Binary metrics (conversion, CTR)
- Count metrics with natural bounds
- When outliers are key business outcomes
- Already normally distributed metrics
- Treatment targets high-value users
Important Note
Winsorizing should be pre-specified in your analysis plan, not applied post-hoc to achieve desired results. Document your winsorizing strategy before analyzing results.
Winsorizing Parameters
Configure parameters and click "Run Simulation" to see results
Understanding the Results
Single Sample Mode: Compare statistics between original and winsorized data to see how outlier treatment affects mean, standard deviation, and confidence intervals.
- Distribution Chart: Shows original values with red markers indicating outliers that will be capped
- Histogram Comparison: Visual comparison of original vs. winsorized distributions
- CI Width Reduction: Percentage decrease in confidence interval width after winsorizing
- Standard Deviation: Reduction in variability after capping extreme values
A/B Test Mode: See how winsorizing affects statistical significance and effect size estimates in a two-sample comparison.
- P-Value Comparison: How statistical significance changes with and without winsorizing
- Effect Size: More stable lift estimates with reduced outlier influence
- Confidence Intervals: Narrower CIs typically lead to more decisive test results
Key insight: Winsorizing reduces the impact of extreme outliers while preserving sample size. By capping values at the upper percentile threshold, you prevent a few extreme observations from dominating your variance estimates and inflating confidence intervals. This often results in more statistically sensitive tests without removing legitimate data points.
Normalisation helps make metrics comparable across different groups and noisy data
Make your metrics comparable with Normalisation →