Null Controlled-Treatment Experiment

An Null Controlled-Treatment Experiment is a controlled-treatment experiment where all the treatments are identical.

• AKA: Null Test.
• Context:
  • It can range, depending on the number of treatments, from being a Bivariate Null Test(A/A test) to being an Multivariate Null Test.
  • It can be used to test an Experimentation System and its Null Hypothesis Test.
  • …
• Counter-Example(s):
  • a Placebo-Controlled Experiment.
  • an A/B Test.
• See: Null Hypothesis.

---

---

References

2007

• (Kohavi et al., 2007) ⇒ Ron Kohavi, Randal M. Henne, and Dan Sommerfield. (2007). “Practical Guide to Controlled Experiments on the Web: Listen to Your Customers Not to the Hippo.” In: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. doi:10.1145/1281192.1281295
  • QUOTE: A/A Test. Sometimes called an Null Test (Peterson, 2004). Instead of an A/B test, you exercise the experimentation system, assigning users to one of two groups, but expose them to exactly the same experience. An A/A test can be used to (i) collect data and assess its variability for power calculations, and (ii) test the experimentation system (the Null hypothesis should be rejected about 5% of the time when a 95% confidence level is used).

2004

• (Peterson, 2004) ⇒ Eric T. Peterson. (2004). “Web Analytics Demystified: A Marketer's Guide to Understanding How Your Web Site Affects Your Business, s.1". :Celilo Group Media and Cafe Press. ISBN:0974358428.

---