Learn · Module 15 · Statistics and experiments · Lesson 3 of 5
A/B tests
An A/B test is the most reliable way to know if a change helped. Most mistakes happen before it starts.
In short
- Random assignment means a real difference is caused by the change.
- Decide the metric and sample size before starting; small effects need thousands of users per group.
- Don't stop early when p dips below 0.05; report the effect size with a confidence interval.
This lesson is part of Pro
The first two lessons of every module are free. Pro opens every lesson and every project.
See Pro plansFrom ₹119 a month. No auto-renewal.