1999 MakingLargeScaleSVMLearningPractical

• (Joachims, 1999a) ⇒ Thorsten Joachims. (1999). “Making Large-Scale SVM Learning Practical.” In: (Schölkopf et al., 1999).

Subject Headings: SVMlight, SVM Learning Algorithm.

Notes

• Website: http://svmlight.joachims.org/

Cited By

• ~3,232 http://scholar.google.com/scholar?q=.+%22Making+Large-Scale+SVM+Learning+Practical%22+1999

2004

• (Hastie et al., 2004) ⇒ Trevor Hastie, Saharon Rosset, Robert Tibshirani, and Ji Zhu. (2004). “The Entire Regularization Path for the Support Vector Machine.” In: The Journal of Machine Learning Research, 5.
  • QUOTE: Software packages, such as the widely used SVMlight (Joachims, 1999), provide default settings for C, which are then used without much further exploration. A recent introductory document (Hsu et al., 2003) supporting the LIBSVM package does encourage grid search for C.

2003

• (Blei, Ng & Jordan, 2003) ⇒ David M. Blei, Andrew Y. Ng , and Michael I. Jordan. (2003). “Latent Dirichlet Allocation.” In: The Journal of Machine Learning Research, 3.

2001

• (Chang & Lin, 2001) ⇒ Chih-Chung Chang, and Chih-Jen Lin. (2001). “LIBSVM: a library for support vector machines."

Quotes

Abstract

Training a support vector machine (SVM) leads to a quadratic optimization problem with bound constraints and one linear equality constraint. Despite the fact that this type of problem is well understood, there are many issues to be considered in designing an SVM learner. In particular, for large learning tasks with many training examples, o -the-shelf optimization techniques for general quadratic programs quickly become intractable in their memory and time requirements. SVMlight is an implementation of an SVM learner which addresses the problem of large tasks. This chapter presents algorithmic and computational results developed for SVMlightV2.0, which make large-scale SVM training more practical. The results give guidelines for the application of SVMs to large domains.

Table of Contents

11.1 Introduction
11.2	General Decomposition Algorithm
11.3	Selecting a Good Working Set
   11.3.1	Convergence
   11.3.2	How to Solve OP3
11.4	Shrinking: Reducing the Size of OP1
11.5	Efficient Implementation
    11.5.1	Termination Criteria
    11.5.2	Computing the Gradient and the Termination Criteria Effici ently
    11.5.3	Computational Resources Needed in Each Iteration
    11.5.4	Caching Kernel Evaluations
    11.5.5	How to Solve OP2 (QP Subproblems)
    11.6        Related Work
11.7	Experiments
   11.7.1	How Does Training Time Scale with the Number of Training Examples?
       11.7.1.1	Income Prediction
       11.7.1.2	Classifying Web Pages
       11.7.1.3	Ohsumed Data Set
       11.7.1.4	Dectecting Faces in Images
    11.7.2	What Is the Influence of the Working Set Selection Strateg y?
    11.7.3	What Is the Influence of Caching?
    11.7.4	What Is the Influence of Shrinking?
    11.8	Conclusions

References

• Bernhard E. Boser, Isabelle M. Guyon, and Vladimir N. Vapnik. A training algorithm for optimal margin classifiers. In D. Haussler, editor, Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, pages 144{152, Pittsburgh, PA, July (1992). ACM Press.
• P. E. Gill, W. Murray, and M. H. Wright. Practical Optimization. Academic Press, 1981.
• Thorsten Joachims. Text categorization with support vector machines. In European Conference on Machine Learning (ECML), 1998.
• E. Osuna, R. Freund, and F. Girosi. Support vector machines: Training and applications. A.I. Memo (in press), MIT A. I. Lab., 1996.
• E. Osuna, R. Freund, and F. Girosi. An improved training algorithm for support vector machines. In J. Principe, L. Gile, N. Morgan, and E. Wilson, editors, Neural Networks for Signal Processing VII | Proceedings of the 1997 IEEE Workshop, pages 276 { 285, New York, 1997a. IEEE.
• E. Osuna, R. Freund, and F. Girosi. Training support vector machines: An application to face detection. In, editor, Proceedings CVPR'97,, 1997b. .
• J. Platt. Sequential minimal optimization: A fast algorithm for training support vector machines. Technical Report MSR-TR-98-14, Microsoft Research, 1998.
• R. Vanderbei. Loqo: An interior point code for quadratic programming. Technical Report SOR 94-15, Princeton University, 1994.
• Vladimir N. Vapnik. The Nature of Statistical Learning Theory. Springer Verlag, New York, 1995.
• J. Werner. Optimization - Theory and Applications. Vieweg, 1984.
• G. Zoutendijk. Methods of Feasible Directions: a Study in Linear and Non-linear Programming. Elsevier, 1970.

---

,

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
1999 MakingLargeScaleSVMLearningPractical	Thorsten Joachims			Making Large-Scale SVM Learning Practical			http://www.joachims.org/publications/joachims 99a.pdf			1999