Free tools · Testing
A/B Test Calculator
Check whether your winning version really won, or whether it was noise.
Enter visitors and conversions for both variants to test whether the difference is real.
Marker is your 95% target. The bar turns lime once you clear it.
Significance is the chance this gap is real, not random. At 95% confidence you would be fooled by luck only about 1 test in 20.
FAQ
A/B Test Calculator, answered.
What does statistical significance mean in an A/B test?
That the gap you are seeing is unlikely to be luck. A 95 percent confidence level means roughly a 1 in 20 chance you are looking at a fluke. It describes the data. It says nothing about how much better your winner will be next month.
What is a p-value?
The p-value is the chance of seeing a gap this big if the two versions were identical. Under 0.05 is the usual bar for calling a result real. It answers one narrow question and nothing else.
How long should I run an A/B test?
Long enough to reach your sample size, and at least one full business cycle, usually a week or two. Stopping the moment it looks significant is how teams pick losers. Early results swing about and reverse all the time.
My test is significant but the uplift is tiny. Does it matter?
Probably not. Significance only says the effect is real. It says nothing about whether the effect is big enough to care about. With enough traffic a 0.1 percent lift becomes significant and still will not pay for the work of building it.
Where this stops being a calculator
Count a sale twice and your test crowns the wrong winner. We check the tracking before anyone runs an experiment, so the numbers going into this calculator mean what you think they mean.
Two marketers. Your whole account.
Book a call and tell us what you want to grow, or get the free website report first.