NarrativeQA Dataset
Jump to navigation
Jump to search
A NarrativeQA Dataset is a QA dataset that consists of over 1,500 stories from books and movie scripts as well as human generated summaries.
- Context:
- Example(s):
- …
- Counter-Example(s):
- a CoQA Dataset,
- a MS COCO Dataset,
- a NewsQA Dataset,
- a RACE Dataset,
- a SearchQA Dataset,
- a SQuAD Dataset.
- See: Question-Answering System, Natural Language Processing Task, Natural Language Understanding Task, Natural Language Generation Task.
References
2018
- (Kocisky et al., 2018) ⇒ Tomas Kocisky, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gabor Melis, and Edward Grefenstette. (2018). “The NarrativeQA Reading Comprehension Challenge.” In: Trans. Assoc. Comput. Linguistics, 6.
- QUOTE: To encourage progress on deeper comprehension of language, we present a new dataset and set of tasks in which the reader must answer questions about stories by reading entire books or movie scripts. These tasks are designed so that successfully answering their questions requires understanding the underlying narrative rather than relying on shallow pattern matching or salience. (...)
- QUOTE: To encourage progress on deeper comprehension of language, we present a new dataset and set of tasks in which the reader must answer questions about stories by reading entire books or movie scripts. These tasks are designed so that successfully answering their questions requires understanding the underlying narrative rather than relying on shallow pattern matching or salience.
| Dataset | Documents | Questions | Answers |
|---|---|---|---|
| MCTest (Richardson et al., 2013) | 660 short stories, grade school level | 2640 human generated, based on the document | multiple choice |
| CNN/Daily Mail (Hermann et al., 2015) | 93K+220K news articles | 387K+997K Cloze-form, based on highlights | entities |
| Children’s Book Test (CBT) (Hill et al., 2016) | 687K of 20 sentence passages from 108 children’s books | Cloze-form, from the 21st sentence | multiple choice |
| BookTest (Bajgar et al., 2016) | 14.2M, similar to CBT | Cloze-form, similar to CBT | multiple choice |
| SQuAD (Rajpurkar et al., 2016) | 23K paragraphs from 536 Wikipedia articles | 108K human generated, based on the paragraphs | spans |
| NewsQA (Trischler et al., 2017) | 13K news articles from the CNN dataset | 120K human generated, based on headline, highlights | spans |
| MS MARCO (Nguyen et al., 2016) | 1M passages from 200K+ documents retrieved using the queries | 100K search queries | human generated, based on the passages |
| SearchQA (Dunn et al., 2017) | 6.9m passages retrieved from a search engine using the queries | 140k human generated Jeopardy! questions | human generated Jeopardy! answers |
| NarrativeQA (this paper) | 1,572 stories (books, movie scripts) & human generated summaries | 46,765 human generated, based on summaries | human generated, based on summaries |