2001 SupportVMActiveLearningWAtoTextCl

(Tong & Koller, 2001) ⇒ Simon Tong, and Daphne Koller. (2001). “Support Vector Machine Active Learning with Applications to Text Classification.” In: The Journal of Machine Learning Research, 2. doi:10.1162/153244302760185243

Subject Headings: Active Learning, SVM Learning, Text Classification Task.

Notes

Cited by

Quotes

Abstract

Support vector machines have met with significant success in numerous real-world learning tasks. However, like most machine learning algorithms, they are generally applied using a randomly selected training set classified in advance. In many settings, we also have the option of using pool-based active learning. Instead of using a randomly selected training set, the learner has access to a pool of unlabeled instances and can request the labels for some number of them. We introduce a new algorithm for performing active learning with support vector machines, i.e., an algorithm for choosing which instances to request next. We provide a theoretical motivation for the algorithm using the notion of a version space. We present experimental results showing that employing our active learning method can significantly reduce the need for labeled training instances in both the standard inductive and transductive settings.

References

Christopher J.C. Burges, A Tutorial on Support Vector Machines for Pattern Recognition, Data Mining and Knowledge Discovery, v.2 n.2, p.121-167, June 1998 doi:10.1023/A:1009715923555
Colin Campbell, Nello Cristianini, Alex J. Smola, Query Learning with Large Margin Classifiers, Proceedings of the Seventeenth International Conference on Machine Learning, p.111-118, June 29-July 02, 2000
G. Cauwenberghs and T. Poggio. Incremental and decremental support vector machine learning. In Advances in Neural Information Processing Systems, volume 13, 2001.
Corinna Cortes, Vladimir Vapnik, Support-Vector Networks, Machine Learning, v.20 n.3, p.273-297, Sept. 1995 doi:10.1023/A:1022627411411
I. Dagan and S. Engelson. Committee-based sampling for training probabilistic classifiers. In: Proceedings of the Twelfth International Conference on Machine Learning, pages 150-157. Morgan Kaufmann, 1995..
Susan Dumais, John Platt, David Heckerman, Mehran Sahami, Inductive learning algorithms and representations for text categorization, Proceedings of the seventh International Conference on Information and knowledge management, p.148-155, November 02-07, 1998, Bethesda, Maryland, United States doi:10.1145/288627.288651
Yoav Freund, H. Sebastian Seung, Eli Shamir, Naftali Tishby, Selective Sampling Using the Query by Committee Algorithm, Machine Learning, v.28 n.2-3, p.133-168, Aug./Sept. 1997 doi:10.1023/A:1007330508534
D. Heckerman, J. Breese, and K. Rommelse. Troubleshooting Under Uncertainty. Technical Report MSR-TR-94-07, Microsoft Research, 1994.
Ralf Herbrich, Thore Graepel, Colin Campbell, Bayes point machines, The Journal of Machine Learning Research, 1, p.245-279, 9/1/2001 doi:10.1162/153244301753683717
Eric Horvitz, Geoffrey Rutledge, Time-dependent utility and action under uncertainty, Proceedings of the seventh conference (1991) on Uncertainty in artificial intelligence, p.151-158, June 1994, Los Angeles, California, United States
Thorsten Joachims, Text Categorization with Suport Vector Machines: Learning with Many Relevant Features, Proceedings of the 10th European Conference on Machine Learning, p.137-142, April 21-23, 1998
Thorsten Joachims, Making large-scale support vector machine learning practical, Advances in kernel methods: support vector learning, MIT Press, Cambridge, MA, 1999
Thorsten Joachims, Transductive Inference for Text Classification using Support Vector Machines, Proceedings of the Sixteenth International Conference on Machine Learning, p.200-209, June 27-30, 1999
K. Lang. Newsweeder: Learning to filter netnews. In: Proceedings of The International Conference on Machine Learning, pages 331-339, 1995.
Jean-Claude Latombe, Robot Motion Planning, Kluwer Academic Publishers, Norwell, MA, 1991
D. Lewis and J. Catlett. Heterogeneous uncertainty sampling for supervised learning. In: Proceedings of the Eleventh International Conference on Machine Learning, pages 148-156. Morgan Kaufmann, 1994.
David D. Lewis, William A. Gale, A sequential algorithm for training text classifiers, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.3-12, July 03-06, 1994, Dublin, Ireland. 18. David A. McAllester, PAC-Bayesian model averaging, Proceedings of the twelfth annual conference on Computational learning theory, p.164-170, July 07-09, 1999, Santa Cruz, California, United States doi:10.1145/307400.307435
A. McCallum. Bow: A toolkit for statistical language modeling, text retrieval, classification and clustering, www.cs.cmu.edu/~mccallum/bow, 1996.
Andrew McCallum, Kamal Nigam, Employing EM and Pool-based Active Learning for Text Classification, Proceedings of the Fifteenth International Conference on Machine Learning, p.350-358, July 24-27, 1998
T. Mitchell. Generalization as search. Artificial Intelligence, 28:203-226, 1982.
J. Rocchio. Relevance feedback in information retrieval. In Gerard M. Salton, editor, The SMART retrieval system: Experiments in automatic document processing. Prentice-Hall, 1971.
23. (Schohn & Cohn, 2000) ⇒ Greg Schohn, and David Cohn. (2000). “Less is More: Active Learning with Support Vector Machines.” In: Proceedings of the Seventeenth International Conference on Machine Learning (ICML 2000).
Fabrizio Sebastiani, Machine learning in automated text categorisation, Centre National de la Recherche Scientifique, Paris, France, 1999.
H. S. Seung, M. Opper, H. Sompolinsky, Query by committee, Proceedings of the fifth annual workshop on Computational learning theory, p.287-294, July 27-29, 1992, Pittsburgh, Pennsylvania, United States doi:10.1145/130385.130417.
John Shawe-Taylor, Nello Cristianini, Further results on the margin distribution, Proceedings of the twelfth annual conference on Computational learning theory, p.278-285, July 07-09, 1999, Santa Cruz, California, United States doi:10.1145/307400.307470
Vladimir Vapnik, Estimation of Dependences Based on Empirical Data: Springer Series in Statistics (Springer Series in Statistics), Springer-Verlag New York, Inc., Secaucus, NJ, 1982
V. Vapnik. Statistical Learning Theory. Wiley, 1998.

,

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2001 SupportVMActiveLearningWAtoTextCl	Daphne Koller Simon Tong			Support Vector Machine Active Learning with Applications to Text Classification			http://jmlr.csail.mit.edu/papers/volume2/tong01a/tong01a.pdf	10.1162/153244302760185243