Imbalanced Flights Simulator
Explore how imbalanced sample splits affect statistical power and required sample sizes in A/B tests
Parameters
Why Imbalance Increases Variance
The standard error of the difference between two groups is calculated as:
SE = σ × √(1/ncontrol + 1/ntreatment)
As one group becomes smaller, its 1/n term gets larger, increasing the overall standard error and reducing statistical power.
Distribution Visualization
Sampling distributions for control (blue) and treatment (green) groups. As imbalance increases, standard errors differ, reducing overlap and statistical power.
Power at Different Splits – Ability to Detect a True Difference
Statistical power represents your ability to detect a true difference when one exists. At a 50/50 split, you achieve 80% power. Imbalanced splits reduce your ability to detect true effects.
Extra Sample Size Required vs 50/50 Split
Percentage increase in total sample size needed to maintain the same statistical power as a balanced 50/50 split.
CUPED can reduce variance introduced by imbalanced flights
Go to CUPED Variance Reduction →