A/B Test Significance Calculator — Free Tool
Check whether the winner of your A/B test is real. Enter impressions and conversions for both variants to see each conversion rate, the relative lift, the p-value, and a plain-language verdict.
Variant A (control)
Variant B (challenger)
Features
- Conversion rate for each variant, side by side
- Relative lift — how much the challenger beats the control
- Two-tailed p-value from a two-proportion z-test
- Refuses to call a winner below 100 impressions or 5 conversions per arm
- Runs entirely in your browser — nothing is uploaded
Frequently asked questions
What does statistical significance actually tell me?
It tells you one narrow thing: how likely a gap this large would be if the two variants performed identically. A p-value under 0.05 means that gap would be unusual by chance alone. It does NOT tell you the difference is large enough to matter commercially, that it will hold next month, that variant B caused it, or that you can stop the test — those are judgements you still have to make yourself.
Why does it sometimes say 'not enough data' instead of giving a verdict?
Because a p-value computed from a handful of events is noise dressed up as a result. If either arm has fewer than 100 impressions or fewer than 5 conversions, the test is too underpowered to be informative, so the calculator says so instead of printing a number people would act on.
Which test does this use?
A two-tailed two-proportion z-test on the pooled proportion — the standard check for comparing two conversion rates. The p-value comes from a normal-distribution approximation, which is accurate at the sample sizes the tool allows a verdict for.
Is 95% confidence the right threshold?
It is the convention, not a law. 95% means you accept roughly a 1-in-20 chance of calling a winner that is not one. If the change is cheap and reversible you might act on weaker evidence; if it is expensive, wait for more data or use a stricter threshold.
Is my data sent anywhere?
No. Every calculation happens in your browser. Nothing is uploaded, stored, or logged, and no account is required.
