Lesson 3

Novelty and Primacy

Two opposite effects, both caused by the change being new rather than being better or worse.

Effect	What Happens	Who It Hits	How to Spot It
Novelty	New thing gets extra attention because it's new; gain fades	Existing users, mostly	Effect shrinks week over week; check week 4 after rollout
Primacy	Existing users are disrupted and perform worse temporarily	Existing users, mostly	Effect improves over time as people adjust

Both distort short tests on established products. The practical defences: run long enough to see whether the effect is stable, look at new users separately from existing ones where you can, and always do the week-4 check after rollout. A gain that has vanished by week four was novelty.

Novelty and Primacy | A/B Testing for Product Managers