2015 WebsiteOptimizationProblemandIt

• (Iitsuka & Matsuo, 2015) ⇒ Shuhei Iitsuka, and Yutaka Matsuo. (2015). “Website Optimization Problem and Its Solutions.” In: Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2015). ISBN:978-1-4503-3664-2 doi:10.1145/2783258.2783351

Subject Headings: Online Controlled Experiment, Website Optimization.

Cited By

• http://scholar.google.com/scholar?q=%222015%22+Website+Optimization+Problem+and+Its+Solutions
• http://dl.acm.org/citation.cfm?id=2783258.2783351&preflayout=flat#citedby

Quotes

Author Keywords

• A/b testing; controlled experiments; experimental design; website optimization

Abstract

Online controlled experiments are widely used to improve the performance of websites by comparison of user behavior related to different variations of the given website. Although such experiments might have an important effect on the key metrics to maximize, small-scale websites have difficulty applying this methodology because they have few users. Furthermore, the candidate variations increase exponentially with the number of elements that must be optimized. A testing method that finds a high-performing variation with a few samples must be devised to address these problems. As described herein, we formalize this problem as a website optimization problem and provide a basis to apply existing search algorithms to this problem. We further organize existing testing methods and extract devices to make the experiments more effective. By combining organized algorithms and devices, we propose a rapid testing method that detects high-performing variations with few users. We evaluated our proposed method using simulation experiments. Results show that it outperforms existing methods at any website scale. Moreover, we implemented our proposed method as an optimizer program and used it on an actual small-scale website. Results show that our proposed method can achieve 57% higher performance variation than that of the generally used A/B testing method. Therefore, our proposed method can optimize a website with fewer samples. The website optimization problem has broad application possibilities that are applicable not only to websites but also to manufactured goods.

References

;

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2015 WebsiteOptimizationProblemandIt	Yutaka Matsuo Shuhei Iitsuka			Website Optimization Problem and Its Solutions				10.1145/2783258.2783351		2015