A/B Test Sample Size Calculator

What sample size do I need for an A/B test? Sample size depends on your baseline conversion rate, the minimum effect size you want to detect, and your statistical confidence level. A typical ecommerce A/B test needs 8,000 to 30,000 visitors per variant to detect a 10% lift at 95% confidence on a 2.5% baseline conversion rate. Running A/B tests without proper sample size planning is one of the most common mistakes in ecommerce CRO. Ending a test too early leads to false positives — you implement a change that appeared to win but was actually just random noise. Running a test too long wastes time and traffic you could be using to test other ideas.

The calculation is based on statistical power analysis. You input your current baseline conversion rate, the minimum improvement you want to detect (the MDE), your desired significance level (typically 95%), and statistical power (typically 80%). The calculator then determines the sample size needed per variant to reliably detect that improvement if it exists.

The MDE is the critical input that most people get wrong. Setting it too small (like 2%) requires enormous sample sizes that could take months to achieve. Setting it too large (like 50%) means you will only detect massive improvements and miss valuable smaller wins. For most ecommerce pages, a 10 to 20 percent relative MDE is practical — it means you are looking for a change that moves a 2.5% conversion rate to 2.75% or higher.

Daily traffic determines how long the test takes. A page with 500 daily visitors testing two variants needs each variant to see the full sample, so the duration equals total sample divided by daily visitors. Low-traffic pages may need weeks or months for a single test, which is why high-traffic pages like the homepage, product pages, and cart page should be prioritized for testing. Use this calculator before starting any test to set expectations with your team and avoid the temptation to peek at results and call winners prematurely.