The most common error in A/B testing by a wide margin. You check the result on day four, it's significant, you stop the test and ship. The problem is that significance fluctuates during a test, and if you keep checking you will eventually catch a moment where it crosses the line by chance.

WHAT PEEKING DOES

A test with no real difference between variants, checked every day for three weeks:

Day	Result	Note
Day 3	+6%, significant	Stop here and "win"
Day 5	+2%, not significant
Day 8	-1%, not significant
Day 11	+4%, not significant
Day 14	-5%, significant	Or stop here and "lose"
Day 18	+1%, not significant
Day 21	0%, not significant	The truth

There was never any difference. Daily checking gave two opportunities to conclude confidently and wrongly.

Checking every day roughly triples your chance of a false positive compared with checking once at the end.

The Practical Fix

Decide the end date before you start and don't look at outcomes until then. If you genuinely need early stopping, for a risky change, or a long test, ask whoever owns your analytics to set up a sequential testing method that accounts for repeated checks. That exists and it's legitimate. Informal daily checking is not the same thing.

A/B Testing for Product Managers

Module 1Why Most A/B Tests Waste Three Weeks

Module 2Is This Even Worth Testing?

Module 3Writing a Hypothesis You Can Be Wrong About

Module 4Choosing the Metric

Module 5How Long It Needs to Run

Module 6Running It Without Ruining It

Module 7Reading the Result

Module 8Deciding and Shipping

Module 9The Six Ways Tests Lie

Module 10When A/B Testing Is the Wrong Tool

Module 11Final Project: Design, Run and Report One Real Test

Module 12Appendix: The Prompt Library

Module 13Glossary: Words Explained

Peeking