2011 LargeScaleMatrixFactorizationwi

• (Gemulla et al., 2011) ⇒ Rainer Gemulla, Erik Nijkamp, Peter J. Haas, and Yannis Sismanis. (2011). “Large-scale Matrix Factorization with Distributed Stochastic Gradient Descent.” In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2011) Journal. ISBN:978-1-4503-0813-7 doi:10.1145/2020408.2020426

Subject Headings:

Cited By

• http://scholar.google.com/scholar?q=%222011%22+Large-scale+Matrix+Factorization+with+Distributed+Stochastic+Gradient+Descent
• http://dl.acm.org/citation.cfm?id=2020408.2020426&preflayout=flat#citedby

Quotes

Author Keywords

• Algorithms; distributed matrix factorization; experimentation; mapreduce; parallel and vector implementations; performance; recommendation system; stochastic gradient descent

Abstract

We provide a novel algorithm to approximately factor large matrices with millions of rows, millions of columns, and billions of nonzero elements. Our approach rests on stochastic gradient descent (SGD), an iterative stochastic optimization algorithm. We first develop a novel "stratified " SGD variant (SSGD) that applies to general loss-minimization problems in which the loss function can be expressed as a weighted sum of “stratum losses". We establish sufficient conditions for convergence of SSGD using results from stochastic approximation theory and regenerative process theory. We then specialize SSGD to obtain a new matrix-factorization algorithm, called DSGD, that can be fully distributed and run on web-scale datasets using, e.g., MapReduce. DSGD can handle a wide variety of matrix factorizations. We describe the practical techniques used to optimize performance in our DSGD implementation. Experiments suggest that DSGD converges significantly faster and has better scalability properties than alternative algorithms.

References

;

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2011 LargeScaleMatrixFactorizationwi	Rainer Gemulla Peter J. Haas Yannis Sismanis Erik Nijkamp			Large-scale Matrix Factorization with Distributed Stochastic Gradient Descent				10.1145/2020408.2020426		2011