Most failed A/B tests don't fail because the change was bad. They fail because the test was never capable of producing an answer. Knowing which of these four you're heading for saves you the three weeks.

How It Fails	What It Looks Like	Root Cause
Too small to see anything	Result is flat. Nobody knows if it worked.	Not enough traffic for the effect you're looking for
Measured the wrong thing	Metric went up, business didn't	The metric could be won without creating value
Contaminated	Result looks odd; someone changed something mid-test	No process protecting the test while it ran
Unusable regardless of outcome	"Interesting" result, nobody changes anything	You never decided in advance what each outcome would mean

The Test That Should Never Have Run

Before designing anything, ask: what will I do if this wins, and what will I do if it loses? If both answers are the same, or if either answer is "nothing," you have no reason to run the test. This single question retires more bad tests than any other technique in this course.

A/B Testing for Product Managers

Module 1Why Most A/B Tests Waste Three Weeks

Module 2Is This Even Worth Testing?

Module 3Writing a Hypothesis You Can Be Wrong About

Module 4Choosing the Metric

Module 5How Long It Needs to Run

Module 6Running It Without Ruining It

Module 7Reading the Result

Module 8Deciding and Shipping

Module 9The Six Ways Tests Lie

Module 10When A/B Testing Is the Wrong Tool

Module 11Final Project: Design, Run and Report One Real Test

Module 12Appendix: The Prompt Library

Module 13Glossary: Words Explained

The Four Deaths