2012 PracticalBayesianOptimizationof

• (Snoek et al., 2012) ⇒ Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. (2012). “Practical Bayesian Optimization of Machine Learning Algorithms.” In: Proceedings of the 25th International Conference on Neural Information Processing Systems (NIPS-2012).

Subject Headings: Bayesian Optimization.

Notes

• Computing Resource(s):
  • GitHub Repository: https://github.com/JasperSnoek/spearmint
• Other Link(s):
  • NIPS: http://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms
  • ACM DL: https://dl.acm.org/doi/10.5555/2999325.2999464
  • ArXiv: https://arxiv.org/abs/1206.2944
  • DBLP: https://dblp.org/rec/html/conf/nips/SnoekLA12

Cited By

• Google Scholar: ~ 3,347 Citations, Retrieved:2020-06-07.
• Semantic Scholar: ~ 2,660 Citations, Retrieved:2020-06-07.
• MS Academic: ~ 3,013 Citations, Retrieved:2020-06-07.
• ACM DL: ~ 211 Citations, Retrieved:2020-06-07.

Quotes

Abstract

The use of machine learning algorithms frequently involves careful tuning of learning parameters and model hyperparameters. Unfortunately, this tuning is often a “black art” requiring expert experience, rules of thumb, or sometimes brute-force search. There is therefore great appeal for automatic approaches that can optimize the performance of any given learning algorithm to the problem at hand. In this work, we consider this problem through the framework of Bayesian optimization, in which a learning algorithm's generalization performance is modeled as a sample from a Gaussian process (GP). We show that certain choices for the nature of the GP, such as the type of kernel and the treatment of its hyperparameters, can play a crucial role in obtaining a good optimizer that can achieve expert-level performance. We describe new algorithms that take into account the variable cost (duration) of learning algorithm experiments and that can leverage the presence of multiple cores for parallel experimentation. We show that these proposed algorithms improve on previous automatic procedures and can reach or surpass human expert-level optimization for many algorithms including latent Dirichlet allocation, structured SVMs and convolutional neural networks.

BiTeX

@inproceedings{2012_PracticalBayesianOptimizationof,
  author    = {Jasper Snoek and
               Hugo Larochelle and
               Ryan P. Adams},
  editor    = {Peter L. Bartlett and
               Fernando C. N. Pereira and
               Christopher J. C. Burges and
               Leon Bottou and
               Kilian Q. Weinberger},
  title     = {Practical Bayesian Optimization of Machine Learning Algorithms},
  booktitle = {Advances in Neural Information Processing Systems 25 (NIPS 2012): 26th Annual
               Conference on Neural Information Processing Systems 2012. Proceedings
               of a meeting held December 3-6, 2012, Lake Tahoe, Nevada, United States},
  pages     = {2960--2968},
  year      = {2012},
  url       = {http://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms},
}

---

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2012 PracticalBayesianOptimizationof	Hugo Larochelle Jasper Snoek Ryan P. Adams			Practical Bayesian Optimization of Machine Learning Algorithms						2012