PPLRE Manual Evaluation Task
The manual evaluation task allows the Domain Experts to validate the results of a Relation Recognition Algorithm. This evaluation task typically occurs in batch mode where hundreds of predictions are gathered and presented to the expert.
The next round of evaluations is scheduled for April 2007.
Open questions about this round include:
- How many records to present?
- How many records to present from each of the available PPLRE Relation Recognition Algorithms?
- How many random records to present?
There have been four rounds of manual evaluation:
- Commenced 2006-05-06 and concluded: 2006-05-11
- Commenced 2006-05-12 and concluded: 2006-06-13
- ~ July 2006
- ~ December 2006