A/B Test

Jump to: navigation, search

An A/B Test is a two-treatment controlled experiment with two different active treatments.




  • (Wikipedia, 2011) ⇒ http://en.wikipedia.org/wiki/A/B_testing
    • QUOTE: A/B testing is a methodology of using randomized experiments with two variants, A and B, which are the Control and Treatment in the controlled experiment. Such experiments are commonly used in web development and marketing, as well as in more traditional forms of advertising. Other names include randomized controlled experiments, online controlled experiments, and split testing. In online settings, such as web design (especially user experience design), the goal is to identify changes to web pages that increase or maximize an outcome of interest (e.g., click-through rate for a banner advertisement). As the name implies, two versions (A and B) are compared, which are identical except for one variation that might impact a user's behavior. Version A might be the currently used version (Control), while Version B is modified in some respect (Treatment). For instance, on an e-commerce website the purchase funnel is typically a good candidate for A/B testing, as even marginal improvements in drop-off rates can represent a significant gain in sales. Significant improvements can be seen through testing elements like copy text, layouts, images and colors.[1] Multivariate testing or bucket testing is similar to A/B testing, but tests more than two different versions at the same time.

      While the approach is identical to a between-subjects design, which is commonly used in a variety of research traditions, A/B testing is seen as a significant change in philosophy and business strategy in Silicon Valley.[2][3][4] A/B testing as a philosophy of web development brings the field into line with a broader movement toward evidence-based practice.


  • http://promote.autonomy.com/components/pagenext.jsp?topic=TD::GLOSSARY_OF_TERMS
    • QUOTE: A/B Testing is the most simplistic way of conducting direct marketing and website tests. In such tests, the "A" option is the control, or current champion. The "B" option is the challenger being tested in an attempt to provide better results than "A." During a split run, visitors are randomly shown or offered the "A" or the "B" option. The difference between the two response rates is then evaluated for statistical significance. While simple to conduct and understand, A/B testing is much less informative and much more costly if more than two factors need to be tested, and has a much lower efficiency than multivariable experimental designs.
  • "Split Testing Guide for Online Stores". webics.com.au. August 27, 2012. http://www.webics.com.au/blog/google-adwords/split-testing-guide-for-online-retailers/. Retrieved 2012-08-28. 
  • http://www.wired.com/business/2012/04/ff_abtesting/
  • http://www.wired.com/wiredenterprise/2012/05/test-everything/
  • http://boingboing.net/2012/04/26/ab-testing-the-secret-engine.html