Most tests come back flat. This is the most common outcome and the most consistently misread. A flat result means one of three quite different things, and you have to work out which.

What Flat Might Mean	How to Tell	What to Do
There genuinely is no difference	The range is narrow and centred near zero	Believe it. Don't ship. Learn from it.
The effect is smaller than you could detect	The range is wide, spanning both directions	You learned nothing. Note that honestly.
The effect exists but not for everyone	Pre-declared segments show different directions	Investigate, but carefully (see Chapter 8)

"Not Significant" Is Not "No Difference"

This is the single most important sentence in the chapter. A test that was too small to detect a 5% lift returns "not significant" whether the true effect was 0% or 5%. Reporting that as "the change made no difference" is a factual error, and it leads teams to abandon ideas that were working. Say "we couldn't detect a difference" instead, it's accurate and it's a different conclusion.

A/B Testing for Product Managers

Module 1Why Most A/B Tests Waste Three Weeks

Module 2Is This Even Worth Testing?

Module 3Writing a Hypothesis You Can Be Wrong About

Module 4Choosing the Metric

Module 5How Long It Needs to Run

Module 6Running It Without Ruining It

Module 7Reading the Result

Module 8Deciding and Shipping

Module 9The Six Ways Tests Lie

Module 10When A/B Testing Is the Wrong Tool

Module 11Final Project: Design, Run and Report One Real Test

Module 12Appendix: The Prompt Library

Module 13Glossary: Words Explained

The Flat Result