PPLRE Manual Evaluation Task
Jump to navigation
Jump to search
This section of the PPLRE Project documentation describes the PPLRE Automated Evaluation Task. See also PPLRE Automated Evaluation Task.
Task Overview
The manual evaluation task allows the Domain Experts to validate the results of a Relation Recognition Algorithm. This evaluation task typically occurs in batch mode where hundreds of predictions are gathered and presented to the expert.
Future Plans
The next round of evaluations is scheduled for April 2007.
Open questions about this round include:
- How many records to present?
- How many records to present from each of the available PPLRE Relation Recognition Algorithms?
- How many random records to present?
Task History
There have been four rounds of manual evaluation:
- Commenced 2006-05-06 and concluded: 2006-05-11
- Commenced 2006-05-12 and concluded: 2006-06-13
- ~ July 2006
- ~ December 2006