2009 DesignChallengesandMisconceptio

(Ratinov & Roth, 2009) ⇒ Lev Ratinov, and Dan Roth. (2009). “Design Challenges and Misconceptions in Named Entity Recognition.” In: Proceedings of the Thirteenth Conference on Computational Natural Language Learning.

Subject Headings: Supervised NER Algorithm.

Notes

Cited By

Quotes

Abstract

We analyze some of the fundamental design challenges and misconceptions that underlie the development of an efficient and robust NER system. In particular, we address issues such as the representation of text chunks, the inference approach needed to combine local NER decisions, the sources of prior knowledge and how to use them within an NER system. In the process of comparing several solutions to these challenges we reach some surprising conclusions, as well as develop an NER system that achieves 90.8 F₁ score on the CoNLL-2003 NER shared task, the best reported result for this dataset.

…

System Resources Used F1
+ LBJ-NER Wikipedia, Nonlocal Features, Word-class Model 90.80
- (Suzuki and Isozaki, 2008) Semi-supervised on 1G-word unlabeled data 89.92
- (Ando and Zhang, 2005) Semi-supervised on 27M-word unlabeled data 89.31
- (Kazama and Torisawa, 2007a) Wikipedia 88.02
- (Krishnan and Manning, 2006) Non-local Features 87.24
- (Kazama and Torisawa, 2007b) Non-local Features 87.17
+ (Finkel et al., 2005) Non-local Features 86.86
Table 7: Results for CoNLL03 data reported in the literature. publicly available systems marked by +.

…

8 Conclusions

We have presented a simple model for NER that uses expressive features to achieve new state of the art performance on the Named Entity recognition task. We explored four fundamental design decisions: text chunks representation, inference algorithm, using non-local features and external knowledge. We showed that BILOU encoding scheme significantly outperforms BIO and that, surprisingly, a conditional model that does not take into account interactions at the output level performs comparably to beam-search or Viterbi, while being considerably more efficient computationally. We analyzed several approaches for modeling non-local dependencies and found that none of them clearly outperforms the others across several datasets. Our experiments corroborate recently published results indicating that word class models learned on unlabeled text can be an alternative to the traditional semi-supervised learning paradigm. NER proves to be a knowledge-intensive task, and it was reassuring to observe that knowledge-driven techniques adapt well across several domains. We observed consistent performance gains across several domains, most interestingly in Webpages, where the named entities had less context and were different in nature from the named entities in the training set. Our system significantly outperforms the current state of the art and is available to download under a research license.

References

;

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2009 DesignChallengesandMisconceptio	Lev Ratinov Dan Roth			Design Challenges and Misconceptions in Named Entity Recognition						2009