An A/A test splits traffic between two identical versions. By definition there should be no difference. If your tool reports one, something is wrong with your setup, and you've found it before it corrupted a real test.

What an A/A Test Catches	How It Shows Up
The split isn't actually random	One side gets systematically different users
Tracking fires differently	One side shows fewer events for no reason
Users move between variants	Numbers don't add up across sessions
Your tool is misconfigured	A "significant" difference between identical things
Your natural variation is larger than you thought	Useful context for reading real results later

Worth the Week, Once

Run one A/A test when you set up your testing infrastructure, and again after any major change to it. You don't need one before every test. One week spent confirming the plumbing works is far cheaper than discovering six months later that every result you've shipped on was distorted.

A/B Testing for Product Managers

Module 1Why Most A/B Tests Waste Three Weeks

Module 2Is This Even Worth Testing?

Module 3Writing a Hypothesis You Can Be Wrong About

Module 4Choosing the Metric

Module 5How Long It Needs to Run

Module 6Running It Without Ruining It

Module 7Reading the Result

Module 8Deciding and Shipping

Module 9The Six Ways Tests Lie

Module 10When A/B Testing Is the Wrong Tool

Module 11Final Project: Design, Run and Report One Real Test

Module 12Appendix: The Prompt Library

Module 13Glossary: Words Explained

The A/A Test