Practical deep-dives into ab testing, experiment design, and statistics related. The core topics every data analyst needs to master.
Running experiments without a fixed sample size inflates Type I error rates. This article explains how alpha spending functions keep false positive rates under control, using a practical two-look scenario with interim analyses spaced one week apart.
When you test many metrics simultaneously, your Family-Wise Error Rate explodes. This article compares Bonferroni correction and Benjamini-Hochberg FDR control, with concrete guidance on which to apply in recommendation and advertising experiments.