2011 LargeScaleMatrixFactorizationwi
- (Gemulla et al., 2011) ⇒ Rainer Gemulla, Erik Nijkamp, Peter J. Haas, and Yannis Sismanis. (2011). “Large-scale Matrix Factorization with Distributed Stochastic Gradient Descent.” In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2011) Journal. ISBN:978-1-4503-0813-7 doi:10.1145/2020408.2020426
Subject Headings:
Notes
Cited By
- http://scholar.google.com/scholar?q=%222011%22+Large-scale+Matrix+Factorization+with+Distributed+Stochastic+Gradient+Descent
- http://dl.acm.org/citation.cfm?id=2020408.2020426&preflayout=flat#citedby
Quotes
Author Keywords
- Algorithms; distributed matrix factorization; experimentation; mapreduce; parallel and vector implementations; performance; recommendation system; stochastic gradient descent
Abstract
We provide a novel algorithm to approximately factor large matrices with millions of rows, millions of columns, and billions of nonzero elements. Our approach rests on stochastic gradient descent (SGD), an iterative stochastic optimization algorithm. We first develop a novel "stratified " SGD variant (SSGD) that applies to general loss-minimization problems in which the loss function can be expressed as a weighted sum of “stratum losses". We establish sufficient conditions for convergence of SSGD using results from stochastic approximation theory and regenerative process theory. We then specialize SSGD to obtain a new matrix-factorization algorithm, called DSGD, that can be fully distributed and run on web-scale datasets using, e.g., MapReduce. DSGD can handle a wide variety of matrix factorizations. We describe the practical techniques used to optimize performance in our DSGD implementation. Experiments suggest that DSGD converges significantly faster and has better scalability properties than alternative algorithms.
References
;
Author | volume | Date Value | title | type | journal | titleUrl | doi | note | year | |
---|---|---|---|---|---|---|---|---|---|---|
2011 LargeScaleMatrixFactorizationwi | Rainer Gemulla Peter J. Haas Yannis Sismanis Erik Nijkamp | Large-scale Matrix Factorization with Distributed Stochastic Gradient Descent | 10.1145/2020408.2020426 | 2011 |