Optimizely sample size calculator, with the formula shown
Free, no signup, no email. Checked 6 September 2026.
If you searched for Optimizely's sample size calculator, their tool is at optimizely.com/sample-size-calculator and you should use it if you want Optimizely's own numbers. This is not that tool and we are not affiliated with Optimizely. It is the same standard calculation, run here, with the formula written out so you can check it.
How many visitors do you need?
The formula, written out
Most calculators hide this. There is no good reason to. This is a two proportion test in the pooled variance form, which is the standard fixed horizon calculation.
Let p1 be your baseline conversion rate and p2 be the rate you want to be able to detect, so p2 = p1 × (1 + MDE). Let p̄ be the average of p1 and p2. Then the visitors needed per variation is:
| n per variation | ⌈ [ z1−α/2 · √(2p̄(1−p̄)) + zpower · √(p1(1−p1) + p2(1−p2)) ]2 ÷ (p2 − p1)2 ⌉ |
|---|
Rounded up, two tailed. With the defaults above, z1−α/2 is about 1.96 and zpower is about 0.84. Double the result for the total across a two arm test.
The thing worth internalising is the denominator. Required traffic scales with the inverse square of the effect you are chasing, so halving your minimum detectable effect roughly quadruples the visitors you need. That single relationship explains most arguments about whether a test is worth running.
This page runs the same function our product runs. It is a direct port of the production implementation and is pinned to it by tests, so this calculator cannot quietly disagree with the results screen inside ABTestly. You can read more in our guide to minimum detectable effect.
What Optimizely's calculator does
We checked their tool on 6 September 2026. It asks for three things: a baseline conversion rate, described as "Your control group's expected conversion rate"; a minimum detectable effect, described as "The minimum relative change in conversion rate you would like to be able to detect"; and a statistical significance level, set to 95% by default with 80% and 99% available. The page is headed "Powered by Optimizely Experimentation's stats engine".
Two things it does not do. It does not show a statistical power input, and it does not publish the formula behind the result.
Where the two can disagree
If you put the same numbers into both tools and get different answers, power is the usual reason. Power is the probability of detecting a real effect of the size you specified. At 80% power you accept a one in five chance of missing a real improvement. Move to 90% and the required traffic rises noticeably. Because Optimizely's calculator does not display a power setting or state the value it assumes, the two tools are not necessarily solving the same problem.
There is a second and more interesting reason. Optimizely says its calculator is based on the formula used in its Stats Engine, and Stats Engine uses sequential testing. A sequential procedure does not need a fixed sample size the way a fixed horizon test does. Optimizely's own statistics documentation lists sample size calculation as "Not required" under Sequential and Bayesian, and "Required before starting" only under Fixed Horizon. So if you are running on Stats Engine, the honest answer is that this number is a planning aid for budgeting traffic rather than a stopping rule you are obliged to hit.
Use it for planning, not for stopping
Whichever tool produced your number, decide it before you launch and do not revise it once results arrive. Recalculating a sample size after seeing the data, then stopping when the two meet, is the peeking problem wearing a lab coat. We wrote about why that inflates false positives in the peeking problem, and about turning a sample size into a calendar date in how long to run an A/B test.
If you want the significance test on results you already have, or the effect your existing traffic can detect, those are on our main A/B test calculator.
Quick answers
Is this Optimizely's sample size calculator?
No. This is our calculator, built by ABTestly, and we are not affiliated with Optimizely. Optimizely hosts its own calculator at optimizely.com/sample-size-calculator. We built this page because the maths is standard and worth showing openly, and because Optimizely's tool does not publish the formula behind it. If you want Optimizely's own numbers, use Optimizely's own tool.
What formula does this use?
A two proportion test in the pooled variance form. With p1 as the baseline rate and p2 as p1 multiplied by one plus the minimum detectable effect, and p bar as the average of the two, the visitors needed per variation is the square of z at one minus alpha over two times the square root of two p bar times one minus p bar, plus z at the power times the square root of p1 times one minus p1 plus p2 times one minus p2, all divided by the square of p2 minus p1, rounded up. The page writes this out in full below.
Why does this give a different number to Optimizely's calculator?
Most often because of statistical power. Optimizely's calculator asks for a baseline rate, a minimum detectable effect and a significance level, but does not show a power input or publish the value it assumes. Power changes the answer substantially, so two calculators that disagree on power will disagree on visitors. Optimizely also states that its calculator is based on the formula used in its Stats Engine, which uses sequential testing, and a sequential procedure does not need a fixed sample size in the same way a fixed horizon test does.
How many visitors does an A/B test need?
It depends almost entirely on your baseline conversion rate and how small an effect you want to detect. Smaller effects cost dramatically more traffic, because the required sample size scales with the inverse square of the effect you are chasing. Halving the minimum detectable effect roughly quadruples the visitors you need. Put your own numbers into the calculator above rather than trusting a rule of thumb.
Should I use 80% or 95% significance?
We would use 95%. At 80% significance you accept a one in five chance of calling a win when there is no real effect, which is a high price for a shorter test. We offer 80% here only because Optimizely's calculator offers it and people want to reproduce the number. In our own product the threshold is fixed at 95% deliberately, so that the decision rule cannot be loosened after the data starts arriving.
Can I stop the test once it hits significance?
Not under a fixed horizon test, which is what this calculator plans for. Checking repeatedly and stopping at the first significant reading inflates the false positive rate well beyond the threshold you chose. Decide the sample size before launch, run to it, and read the result once. If you want to monitor continuously you need a sequential method designed for it, which is a different calculation.
Sources checked
Checked on 6 September 2026. Tools and documentation change, so verify before you rely on a number.
- Optimizely's own calculator: optimizely.com/sample-size-calculator, retrieved 6 September 2026
- Optimizely on statistical methods and when a sample size is required: Statistical analysis methods overview, retrieved 6 September 2026
- Our implementation and its tests: docs.abtestly.com
- Related: A/B test calculator, minimum detectable effect, the peeking problem, how long to run a test, Optimizely pricing, VWO vs Optimizely
Optimizely is a trademark of Optimizely, Inc. ABTestly is not affiliated with or endorsed by Optimizely. This calculator is ours, not theirs, and the two can return different numbers for the reasons set out above.
The ABTestly team