2015 FromLighttoRichEREAnnotationofE

• (Song et al., 2015) ⇒ Zhiyi Song, Ann Bies, Stephanie Strassel, Tom Riese, Justin Mott, Joe Ellis, Jonathan Wright, Seth Kulick, Neville Ryant, and Xiaoyi Ma. (2015). “From Light to Rich ERE: Annotation of Entities, Relations, and Events.” In: Proceedings of the 3rd Workshop on EVENTS at the NAACL-HLT.

Subject Headings: Event Taxonomy, Light ERE Taxonomy, Rich ERE Taxonomy.

Cited By

• http://scholar.google.com/scholar?q=%222015%22+From+Light+to+Rich+ERE%3A+Annotation+of+Entities%2C+Relations%2C+and+Events

Quotes

Abstract

We describe the evolution of the Entities, Relations and Events (ERE) annotation task, created to support research and technology development within the DARPA DEFT program. We begin by describing the specification for Light ERE annotation, including the motivation for the task within the context of DEFT. We discuss the transition from Light ERE to a more complex Rich ERE specification, enabling more comprehensive treatment of phenomena of interest to DEFT.

1 Introduction

DARPA's Deep Exploration and Filtering of Text (DEFT) program aims to improve state-of-the-art capabilities in automated deep natural language processing, with a particular focus on technologies dealing with inference, causal relationships, and anomaly detection (DARPA, 2012). Evaluations within the DEFT program focus on a variety of component technologies, united by a common focus on the problem of populating a knowledge base with information about entities and events and the relationships among them. Given the variety of approaches and evaluations within DEFT, we set out to define an annotation task that would be supportive of multiple research directions and evaluations, and that would provide a useful foundation for more specialized annotation tasks like inference and anomaly. The resulting Entities, Relations and Events (ERE) annotation task has evolved over the course of the program, from a fairly lightweight treatment of entities, relations and events in text, to a richer representation of phenomena of interest to the program.

While previous approaches such as ACE (Doddington et al., 2004), LCTL (Simpson et al., 2008), OntoNotes (Pradhan et al., 2007), Machine Reading (Strassel et al., 2010), TimeML (Boguraev and Ando, 2005), Penn Discourse Treebank (Prasad et al., 2014), and Rhetorical Structure Theory (Mann and Thompson, 1988) laid some of the groundwork for this type of resource, the DEFT program requires annotation of complex and hierarchical event structures that go beyond any of the existing (and partially-overlapping) task definitions. Recognizing the effort required to define such an annotation task for multiple languages and genres, we decided to adopt a multi-phased approach, starting with a fairly lightweight implementation and introducing additional complexity over time.

In the first phase of the program, we defined Light ERE as a simplified form of ACE annotation, with the goal of being able to rapidly produce consistently labeled data in multiple languages (Aguilar et al., 2014). In Phase 2, Rich ERE expands entity, relation and event ontologies and expands the notion of what is taggable. Rich ERE also introduces the notion of Event Hopper to address the pervasive challenge of event coreference, particularly with respect to event mention and event argument granularity variation within and across documents, thus paving the way for the important goal of creating (hierarchical or nested) cross-document event representations.

In the remaining sections we describe the Light ERE annotation specification and the resources produced under this spec. We discuss the motivation for transitioning from Light ERE to Rich ERE, and present the Rich ERE specification in detail, along with developments in smart data selection and annotation consistency analysis. We conclude with a discussion of annotation challenges and future directions.

2 Related Annotation Efforts

…

References

;

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2015 FromLighttoRichEREAnnotationofE	Stephanie Strassel Seth Kulick Joe Ellis Zhiyi Song Ann Bies Tom Riese Justin Mott Jonathan Wright Neville Ryant Xiaoyi Ma			From Light to Rich ERE: Annotation of Entities, Relations, and Events