Sample size depends on four things. You need to understand what each one does, not how the formula works, your testing tool or a calculator will do the arithmetic once you can supply the inputs.

Number	What It Means	What Happens If It Changes
Baseline rate	What your metric is now, e.g. 3.2% convert	Lower baselines need much bigger samples
Effect you want to detect	The smallest lift worth knowing about, e.g. +12% relative	Halve the effect and you roughly quadruple the sample
Confidence level	How sure you want to be it's not chance, usually 95%	Higher confidence needs more traffic
Power	Your chance of spotting a real effect, usually 80%	Lower power means real effects get missed

The Second Row Is Where Everything Goes Wrong

The relationship between effect size and sample size is roughly quadratic. Detecting a 10% lift might need 30,000 users per side. Detecting a 5% lift needs around 120,000. Detecting 2% needs somewhere near 750,000. This is why "we'll test this small copy change" is usually not a real plan, the arithmetic simply doesn't allow it.

A/B Testing for Product Managers

Module 1Why Most A/B Tests Waste Three Weeks

Module 2Is This Even Worth Testing?

Module 3Writing a Hypothesis You Can Be Wrong About

Module 4Choosing the Metric

Module 5How Long It Needs to Run

Module 6Running It Without Ruining It

Module 7Reading the Result

Module 8Deciding and Shipping

Module 9The Six Ways Tests Lie

Module 10When A/B Testing Is the Wrong Tool

Module 11Final Project: Design, Run and Report One Real Test

Module 12Appendix: The Prompt Library

Module 13Glossary: Words Explained

The Four Numbers