There's a real cost to over-testing, and it isn't only time. A team that tests everything moves slowly, and slow teams make fewer changes, and fewer changes means less learning overall. Some things should just go out.

Ship It Without Testing	Why
Fixing something clearly broken	You're not going to keep the broken version
Legal, compliance, accessibility	Not optional, so the result can't change anything
Very small changes on modest traffic	You physically cannot detect the effect
Changes you'd keep even if the metric dipped slightly	The decision is already made; testing is theatre
Anything where the reversible cost is tiny	Ship, watch, revert if needed, cheaper than a test
The first version of something genuinely new	Nothing to compare against; you need usage, not a comparison

Ship-and-Watch Is a Legitimate Method

For small changes on moderate traffic, shipping to everyone and watching the metric for two weeks is often the right call. It's less rigorous, you can't separate your change from everything else happening, but it's fast, and for low-stakes reversible changes that trade is usually worth making. Just don't call the result an A/B test.

A/B Testing for Product Managers

Module 1Why Most A/B Tests Waste Three Weeks

Module 2Is This Even Worth Testing?

Module 3Writing a Hypothesis You Can Be Wrong About

Module 4Choosing the Metric

Module 5How Long It Needs to Run

Module 6Running It Without Ruining It

Module 7Reading the Result

Module 8Deciding and Shipping

Module 9The Six Ways Tests Lie

Module 10When A/B Testing Is the Wrong Tool

Module 11Final Project: Design, Run and Report One Real Test

Module 12Appendix: The Prompt Library

Module 13Glossary: Words Explained

When to Just Ship It