Buyer guide
How much traffic do you need for A/B testing?
Summary: Whether you can A/B test depends on traffic, baseline conversion rate and the minimum effect you need to detect. Lower conversion rates and smaller effects need far more visitors. When traffic is too low, research-led fixes and testing on high-traffic steps are better starting points.
Three numbers decide feasibility
- Baseline conversion rate of the step you want to test.
- Minimum detectable effect (MDE): the smallest relative change worth detecting.
- Traffic reaching that step per week.
The lower the baseline and the smaller the effect, the more visitors you need. Halving the MDE roughly quadruples the sample required.
A worked example
Suppose a product page converts at 3% and you want to detect a 10% relative lift (3.0% to 3.3%) with typical settings (95% confidence, 80% power). You need on the order of 50,000 visitors per variation. With 20,000 visitors a week split across two variations, that test needs about five weeks.
If you only care about a 20% lift, the sample falls to roughly a quarter of that. This is why agencies push for bigger, bolder changes on low-traffic sites.
When traffic is too low
- Test higher in the funnel where traffic is larger, such as pricing pages or sign-up starts.
- Use stronger changes that could plausibly move results by 20% or more.
- Lean on research: session recordings, surveys and usability tests find problems you can fix without a test.
- Ship and monitor low-risk fixes with a clear before-and-after measure.
Good agencies check feasibility before proposing a testing program. Wavesy, for example, states that where testing is not practical, research and targeted improvements may be a better start.
Don’t stop tests early
Peeking at results daily and stopping on the first significant reading inflates false positives. Agree the sample size and duration before launch, run full business cycles (usually whole weeks) and only then decide.
See which tools support your setup in our experimentation tools directory.
Topics: A/B testing, SaaS and B2B CRO