2008 TheAnnotationConundrum

• (Liberman, 2008) ⇒ Mark Liberman. (2008). “The Annotation Conundrum". Invited Talk. In: Proceedings of the Workshop on Building & Evaluation Resources for Biomedical Text Mining collocated with LREC-2008.

Subject Headings:

Notes

• Mark Liberman is with the Linguistic Data Consortium (LDC) which is based in Pennsylvania University
• Suggested that the consensus is now that the annotation effort is significant, particularly if you strive for high inter-annotator agreement.
• Annotation efforts can take over a year.
• He compared the challenge to the problem that natural language database query research encountered in the late 80s - it was bogged by knowledge engineering requirements. However, it appears that in the case of information extraction, the payoff exists in some cases to pay the price.
• He suggested that we loosen the requirement for a GOLD baseline, just as was done for translation.
• He suggested that we invest in ontologies.
• I asked where he saw the corpora for the Biomedical domain in the next five years. He suggested that it would more likely still be distributed than be available for positive feedback. He offered LDC as a placeholder for the PPLRE data.
• The loosing of a gold-standard requirement is helpful to our PPLRE project because we have focused on more data rather than very clean data.

References

---

,

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2008 TheAnnotationConundrum	Mark Liberman			The Annotation Conundrum		Proceedings of the Workshop on Building & Evaluation Resources for Biomedical Text Mining collocated with LREC-2008				2008