Clever Marketing Tools
TOOLS / CALCULATORS

A/B test calculator

Statistical significance, required sample size and test duration. Runs in your browser — nothing is stored or sent anywhere.

How the A/B test calculator works

This calculator has two modes. The significance test checks whether the difference between two variants is statistically significant. The sample size calculator tells you how many visitors you need before starting a test. Everything runs in your browser.

Statistical significance test

Enter visitors and conversions for both variants. The calculator uses a two-proportion z-test:

z = (p1 - p2) / sqrt(p_pool * (1 - p_pool) * (1/n1 + 1/n2))

The z-score is converted to a two-tailed p-value using the normal distribution. If the p-value is below the threshold (0.10 for 90%, 0.05 for 95%, 0.01 for 99%), the result is statistically significant — meaning the observed difference is unlikely to be due to chance alone.

Sample size calculator

Enter your baseline conversion rate and the minimum effect you want to detect (as a relative percentage). The calculator uses standard z-scores for your chosen confidence level and 80% statistical power to compute the required sample size per variant. Divide by your daily traffic to estimate test duration.

NOTE
Running a test for too few days inflates false positives. Even if you reach the sample size quickly, run the test for at least one full business cycle (typically 7 days) to capture day-of-week effects.

Interpreting the results

A statistically significant result means the difference is unlikely due to random chance at your chosen confidence level. It does not guarantee the effect is large enough to matter in practice — always check the absolute and relative lift alongside significance. A 0.01 percentage point lift can be statistically significant with enough traffic but commercially meaningless.

DISCLOSURESome links on this page are affiliate links. They never change what we recommend.

Related tools