2010 ActiveLearningforBiomedicalCita

• (Wallace et al., 2010) ⇒ Byron C. Wallace, Kevin Small, Carla E. Brodley, and Thomas A. Trikalinos. (2010). “Active Learning for Biomedical Citation Screening.” In: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2010). doi:10.1145/1835804.1835829

Subject Headings:

Cited By

• http://scholar.google.com/scholar?q=%22Active+learning+for+biomedical+citation+screening%22+2010
• http://portal.acm.org/citation.cfm?id=1835829&preflayout=flat#citedby

Quotes

Abstract

Active learning (AL) is an increasingly popular strategy for mitigating the amount of labeled data required to train classifiers, thereby reducing annotator effort. We describe a real-world, deployed application of AL to the problem of biomedical citation screening for systematic reviews at the Tufts Medical Center's Evidence-based Practice Center. We propose a novel active learning strategy that exploits a priori domain knowledge provided by the expert (specifically, labeled features) and extend this model via a Linear Programming algorithm for situations where the expert can provide ranked labeled features. Our methods outperform existing AL strategies on three real-world systematic review datasets. We argue that evaluation must be specific to the scenario under consideration. To this end, we propose a new evaluation framework for finite-pool scenarios, wherein the primary aim is to label a fixed set of examples rather than to simply induce a good predictive model. We use a method from medical decision theory for eliciting the relative costs of false positives and false negatives from the domain expert, constructing a utility measure of classification performance that integrates the expert preferences. Our findings suggest that the expert can, and should, provide more information than instance labels alone. In addition to achieving strong empirical results on the citation screening problem, this work outlines many important steps for moving away from simulated active learning and toward deploying AL for real-world applications.

References

,

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2010 ActiveLearningforBiomedicalCita	Kevin Small Carla E. Brodley Byron C. Wallace Thomas A. Trikalinos			Active Learning for Biomedical Citation Screening		KDD-2010 Proceedings		10.1145/1835804.1835829		2010