2009 SemiSupervisedActiveLearningfor

Jump to: navigation, search

Subject Headings: Semi-Supervised Active Learning, Linguistic Sequence Labeling.


Cited By



While Active Learning (AL) has already been shown to markedly reduce the annotation efforts for many sequence labeling tasks compared to random selection, AL remains unconcerned about the internal structure of the selected sequences (typically, sentences). We propose a semi-supervised AL approach for sequence labeling where only highly uncertain subsequences are presented to human annotators, while all others in the selected sequences are automatically labeled. For the task of entity recognition, our experiments reveal that this approach reduces annotation efforts in terms of manually labeled tokens by up to 60% compared to the standard, fully supervised AL scheme.


  • 1. Avrim Blum, Tom Mitchell, Combining Labeled and Unlabeled Data with Co-training, Proceedings of the Eleventh Annual Conference on Computational Learning Theory, p.92-100, July 24-26, 1998, Madison, Wisconsin, USA doi:10.1145/279943.279962
  • 2. A. P. Dempster, N. M. Laird, and D. B. Rubin. 1977. Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society, 39(1):1--38.
  • 3. Sean P. Engelson, Ido Dagan, Minimizing Manual Annotation Cost in Supervised Training from Corpora, Proceedings of the 34th Annual Meeting on Association for Computational Linguistics, p.319-326, June 24-27, 1996, Santa Cruz, California doi:10.3115/981863.981905
  • 4. Trausti Kristjansson, Aron Culotta, Paul Viola, Andrew McCallum, Interactive Information Extraction with Constrained Conditional Random Fields, Proceedings of the 19th National Conference on Artifical Intelligence, p.412-418, July 25-29, 2004, San Jose, California
  • 5. S. Kulick, A. Bies, M. Liberman, M. Mandel, R. T. McDonald, M. S. Palmer, and A. I. Schein. 2004. Integrated Annotation for Biomedical Information Extraction. In Proceedings of the HLT-NAACL 2004 Workshop 'Linking Biological Literature, Ontologies and Databases: Tools for Users', Pages 61--68.
  • 6. John D. Lafferty, Andrew McCallum, Fernando C. N. Pereira, Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data, Proceedings of the Eighteenth International Conference on Machine Learning, p.282-289, June 28-July 01, 2001
  • 7. D. D. Lewis and J. Catlett. 1994. Heterogeneous Uncertainty Sampling for Supervised Learning. In ICML'94 -- Proceedings of the 11th International Conference on Machine Learning, Pages 148--156.
  • 8. Linguistic Data Consortium. 2001. Message Understanding Conference (MUC) 7. LDC2001T02. FTP FILE. Philadelphia: Linguistic Data Consortium.
  • 9. Andrew McCallum, Kamal Nigam, Employing EM and Pool-Based Active Learning for Text Classification, Proceedings of the Fifteenth International Conference on Machine Learning, p.350-358, July 24-27, 1998
  • 10. Ion Muslea, Steven Minton, Craig A. Knoblock, Active + Semi-supervised Learning = Robust Multi-View Learning, Proceedings of the Nineteenth International Conference on Machine Learning, p.435-442, July 08-12, 2002
  • 11. Grace Ngai, David Yarowsky, Rule Writing Or Annotation: Cost-efficient Resource Usage for Base Noun Phrase Chunking, Proceedings of the 38th Annual Meeting on Association for Computational Linguistics, p.117-125, October 03-06, 2000, Hong Kong doi:10.3115/1075218.1075234
  • 12. D. Pierce and C. Cardie. 2001. Limitations of Co-training for Natural Language Learning from Large Datasets. In EMNLP'01 -- Proceedings of the 2001 Conference on Empirical Methods in Natural Language Processing, Pages 1--9.
  • 13. L. R. Rabiner. 1989. A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proceedings of the IEEE, 77(2):257--286.
  • 14. Burr Settles, Mark Craven, An Analysis of Active Learning Strategies for Sequence Labeling Tasks, Proceedings of the Conference on Empirical Methods in Natural Language Processing, October 25-27, 2008, Honolulu, Hawaii
  • 15. B. Settles, M. Craven, and L. Friedland. 2008. Active Learning with Real Annotation Costs. In Proceedings of the NIPS 2008 Workshop on 'Cost-Sensitive Machine Learning', Pages 1--10.
  • 16. H. S. Seung, M. Opper, H. Sompolinsky, Query by Committee, Proceedings of the Fifth Annual Workshop on Computational Learning Theory, p.287-294, July 27-29, 1992, Pittsburgh, Pennsylvania, USA doi:10.1145/130385.130417
  • 17. K. Tomanek, J. Wermter, and U. Hahn. 2007. An Approach to Text Corpus Construction Which Cuts Annotation Costs and Maintains Corpus Reusability of Annotated Data. In EMNLP-CoNLL'07 -- Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Language Learning, Pages 486--495.
  • 18. David Yarowsky, Unsupervised Word Sense Disambiguation Rivaling Supervised Methods, Proceedings of the 33rd Annual Meeting on Association for Computational Linguistics, p.189-196, June 26-30, 1995, Cambridge, Massachusetts doi:10.3115/981658.981684;

 AuthorvolumeDate ValuetitletypejournaltitleUrldoinoteyear
2009 SemiSupervisedActiveLearningforKatrin Tomanek
Udo Hahn
Semi-supervised Active Learning for Sequence Labeling2009
AuthorKatrin Tomanek + and Udo Hahn +
titleSemi-supervised Active Learning for Sequence Labeling +
year2009 +