2015 SAMEButDifferentFastandHighQual

• (Zhao et al., 2015) ⇒ Huasha Zhao, Biye Jiang, John F. Canny, and Bobby Jaros. (2015). “SAME But Different: Fast and High Quality Gibbs Parameter Estimation.” In: Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2015). ISBN:978-1-4503-3664-2 doi:10.1145/2783258.2783416

Subject Headings:

Cited By

• http://scholar.google.com/scholar?q=%222015%22+SAME+But+Different%3A+Fast+and+High+Quality+Gibbs+Parameter+Estimation
• http://dl.acm.org/citation.cfm?id=2783258.2783416&preflayout=flat#citedby

Quotes

Author Keywords

• Algorithm design and analysis; efficiency; gibbs sampling; latent dirichlet allocation; parallel and vector implementations; parameter learning; simulated annealing

Abstract

Gibbs sampling is a workhorse for Bayesian inference but has several limitations when used for parameter estimation, and is often much slower than non-sampling inference methods. SAME(State Augmentation for Marginal Estimation) [15, 8] is an approach to MAP parameter estimation which gives improved parameter estimates over direct Gibbs sampling. SAME can be viewed as cooling the posterior parameter distribution and allows annealed search for the MAP parameters, often yielding very high quality estimates. But it does so at the expense of additional samples per iteration and generally slower performance. On the other hand, SAME dramatically increases the parallelism in the sampling schedule, and is an excellent match for modern (SIMD) hardware. In this paper we explore the application of SAME to graphical model inference on modern hardware. We show that combining SAME with factored sample representation (or approximation) gives throughput competitive with the fastest symbolic methods, but with potentially better quality. We describe experiments on Latent Dirichlet Allocation, achieving speeds similar to the fastest reported methods (online Variational Bayes) and lower cross-validated loss than other LDA implementations. The method is simple to implement and should be applicable to many other models.

References

;

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2015 SAMEButDifferentFastandHighQual	John F. Canny Huasha Zhao Biye Jiang Bobby Jaros			SAME But Different: Fast and High Quality Gibbs Parameter Estimation				10.1145/2783258.2783416		2015