This is the most consequential fork and the most commonly got wrong. A test compares two things you already thought of. Research tells you what to think of. Running tests when you needed research produces a long series of flat results and a slowly growing sense that testing doesn't work.

Test It	Go and Watch Five Users
You have two credible options and must pick one	You don't know why people drop off
You understand the problem, you're choosing a solution	You're guessing at the problem
The change is small and isolated	You're considering a substantial redesign
You've already run research and formed a view	Your last three tests were all flat
The decision is reversible and the stakes are moderate	The stakes are high and you're unsure

That fourth row is worth acting on. Three flat tests in a row is a signal, and it usually means you're testing variations on a guess rather than testing a well-understood problem. The fix is not a fourth test.

COPY THIS PROMPT - Deciding Whether to Test

Help me decide whether this is worth A/B testing, or whether I

should do something else.

THE CHANGE I'M CONSIDERING

[what you'd change, in 3-4 sentences]

WHY I THINK IT WILL HELP

[your reasoning, and where it came from]

TRAFFIC AND BASELINE

[roughly how many users hit this per week, and what the current

conversion or completion rate is]

Please answer:

1. What would I do if this won? What would I do if it lost?

If those are the same, say so plainly.

2. Given my traffic, roughly what size of effect could I realistically

detect in three to four weeks? Show your working.

3. Is the effect I'm hoping for bigger or smaller than that?

4. Based on all of the above, should I: test it, ship it and watch,

or go and do research first? Pick one and justify it.

5. If you'd recommend research instead, what specifically should

I go and find out?

A/B Testing for Product Managers

Module 1Why Most A/B Tests Waste Three Weeks

Module 2Is This Even Worth Testing?

Module 3Writing a Hypothesis You Can Be Wrong About

Module 4Choosing the Metric

Module 5How Long It Needs to Run

Module 6Running It Without Ruining It

Module 7Reading the Result

Module 8Deciding and Shipping

Module 9The Six Ways Tests Lie

Module 10When A/B Testing Is the Wrong Tool

Module 11Final Project: Design, Run and Report One Real Test

Module 12Appendix: The Prompt Library

Module 13Glossary: Words Explained

Test, or Go and Talk to Someone?