SVM-based Regression Algorithm: Difference between revisions

Latest revision as of 13:26, 2 August 2022

Context:
- It can be implemented by an SVM Regression System (to solve an SVM regression task).
Example(s):
- sklearn.svm.SVR [1].
- sklearn.svm.LinearSVR.
- libsvm, with -s svm_type 3 (epsilon-SVR) or 4 (nu-SVR).
- …
Counter-Example(s):
See: Kernel-based Regression Algorithm.

http://en.wikipedia.org/wiki/Support_vector_machine#Regression
- A version of SVM for regression was proposed in 1996 by Vladimir Vapnik, Harris Drucker, Chris Burges, Linda Kaufman and Alex Smola.^[1] This method is called support vector regression (SVR). The model produced by support vector classification (as described above) depends only on a subset of the training data, because the cost function for building the model does not care about training points that lie beyond the margin. Analogously, the model produced by SVR depends only on a subset of the training data, because the cost function for building the model ignores any training data close to the model prediction (within a threshold [math]\displaystyle{ \epsilon }[/math]). Another SVM version known as least squares support vector machine (LS-SVM) has been proposed in Suykens and Vandewalle.^[2]

↑ Harris Drucker, Chris J.C. Burges, Linda Kaufman, Alex Smola and Vladimir Vapnik (1997). “Support Vector Regression Machines". Advances in Neural Information Processing Systems 9, NIPS 1996, 155–161, MIT Press.
↑ Suykens J.A.K., Vandewalle J., Least squares support vector machine classifiers, Neural Processing Letters, vol. 9, no. 3, Jun. 1999, pp. 293–300.

(Smola & Schölkopf, 2004) ⇒ Alex J. Smola and Bernhard Schölkopf. (2004). “A Tutorial on Support Vector Regression.” In: Statistics and Computing, 14(3). doi:10.1023/B:STCO.0000035301.49549.88.
- QUOTE: ... we give an overview of the basic ideas underlying Support Vector (SV) machines for function estimation.
(Cherkassky & Ma, 2004) ⇒ Vladimir Cherkassky, and Yunqian Ma. (2004). “Practical Selection of SVM Parameters and Noise Estimation for SVM Regression.” In: Neural Netw., 17(1). doi:10.1016/S0893-6080(03)00169-2

@@ Line 27: / Line 27: @@
 === 2004 ===
 * ([[Smola & Schölkopf, 2004]]) ⇒ [[Alex J. Smola]] and [[Bernhard Schölkopf]]. ([[2004]]). “A Tutorial on Support Vector Regression.” In: Statistics and Computing, 14(3). [http://dx.doi.org/10.1023/B:STCO.0000035301.49549.88 doi:10.1023/B:STCO.0000035301.49549.88].
 ** QUOTE: ... we give an overview of the basic ideas underlying [[SVM-based Regression Algorithm|Support Vector (SV) machines for function estimation]].
 * (Cherkassky & Ma, 2004) ⇒ Vladimir Cherkassky, and Yunqian Ma. ([[2004]]). “Practical Selection of SVM Parameters and Noise Estimation for SVM Regression.” In: Neural Netw., 17(1). [http://dx.doi.org/10.1016/S0893-6080(03)00169-2 doi:10.1016/S0893-6080(03)00169-2]