DARPA Deep Exploration and Filtering of Text (DEFT) Project (2012-2018)
A DARPA Deep Exploration and Filtering of Text (DEFT) Project (2012-2018) is a DARPA R&D project with a goal to identify explicit and implicit information from multiple unstructured text sources to support automated text analytics and human text analysts.
- Example(s):
- See: DARPA Muse.
References
2017
- https://www.darpa.mil/program/deep-exploration-and-filtering-of-text
- QUOTE: Department of Defense (DoD) operators and analysts collect and process copious amounts of data from a wide range of sources to create and assess plans and execute missions. However, depending on context, much of the information that could support DoD missions may be implicit rather than explicitly expressed. Having the capability to automatically extract operationally relevant information that is only referenced indirectly would greatly assist analysts in efficiently processing data.
Automated, deep natural-language processing (NLP) technology may hold a solution for more efficiently processing text information and enabling understanding connections in text that might not be readily apparent to humans. DARPA created the Deep Exploration and Filtering of Text (DEFT) program to harness the power of NLP. Sophisticated artificial intelligence of this nature has the potential to enable defense analysts to efficiently investigate orders of magnitude more documents so they can discover implicitly expressed, actionable information contained within them.
By building on the NLP technologies developed in other DARPA programs and ongoing academic research into deep language understanding and artificial intelligence, DEFT aims to address remaining capability gaps related to inference, causal relationships and anomaly detection. Improving human language technology to incorporate these capabilities is essential for enabling automated exposure of important content to facilitate analysis.
As a further aid to analysis, DEFT also aims to enable the capability to integrate individual facts into large domain models as information is processed to support assessment, planning, prediction and the initial stages of report writing. If successful, DEFT will allow analysts to move from limited, linear processing of huge sets of data to a nuanced, strategic exploration of available information.
The development of an automated solution may involve contributions from the linguistics and computer science fields in the areas of artificial intelligence, computational linguistics, machine learning, natural-language understanding, discourse and dialogue analysis, and others.
- QUOTE: Department of Defense (DoD) operators and analysts collect and process copious amounts of data from a wide range of sources to create and assess plans and execute missions. However, depending on context, much of the information that could support DoD missions may be implicit rather than explicitly expressed. Having the capability to automatically extract operationally relevant information that is only referenced indirectly would greatly assist analysts in efficiently processing data.
2014
- "DEFT: Deep Exploration and Filtering of Text." University of Illinois at Urbana-Champaign, Cognitive Computation Group
- Period: 2012-2017
- The objective of DARPA’s DEFT program is to create capabilities for deep natural language understanding and use them to aid analysts in identifying information sources that contain new developments of interest. The goal of the Cognitive Computation Group team at the University of Illinois is to combine Natural Language Processing (NLP), Machine Learning (ML), and Knowledge Representation and Reasoning (KRR) techniques into new technologies that support the DEFT mission. We will develop technologies applicable to a range of DEFT scenarios and do it in a way that our components can be integrated seamlessly within DEFT applications. In doing so, we will advance the understanding of the scientific community dealing with large scale natural language understanding challenges.
Our project, a broad range purposeful textual inference system, is built on two pillars: 1) an innovative learning and inference approach emphasizing joint inference over a component-based architecture, and 2) a textual inference approach that offers a radical re-thinking of the architecture supporting relational analysis in NLP. Rather than focusing on annotating specific types of relations and events, we focus on a generic but goal oriented inference engine that determines, given an instantiated or typed relation and a set of documents, whether the relation is entailed by the documents. We then automatically reduce HLT tasks such as relation and event extraction to instances of this generic capability.
The generic purposeful textual inference capability constitutes the first algorithmic component. It will interact, via our flexible inference mechanism and a supporting multilayer representation of natural language analyses, with three additional algorithmic components: (2) Sentence Level Extended Semantic Role Labeling component that provides a complete and coherent predicate-argument representation of sentences covering all predicate types. (3) A Discourse Analysis component that addresses discourse phenomena including relations between events, temporal grounding of events and relations, and time lining of events, and (4) a Profiling component that provides a new way of representing, aggregating and supporting the use of knowledge about concepts and entities in NLP. Our models are designed with a uniform API so that they can be seamlessly integrated within DEFT applications. The principles of our joint-inference component supporting architecture (JOINCA) underlie all the algorithmic components.
2014
- (Onyshkevych, 2013) ⇒ Boyan Onyshkevych. (2013). “Deep Exploration and Filtering of Text (DEFT)." DARPA Program
- QUOTE: Department of Defense (DoD) operators and analysts collect and process copious amounts of data from a wide range of sources to create and assess plans and execute missions. However, depending on context, much of the information that could support DoD missions may be implicit rather than explicitly expressed. Having the capability to automatically extract operationally relevant information that is only referenced indirectly would greatly assist analysts in efficiently processing data.
Automated, deep natural-language processing (NLP) technology may hold a solution for more efficiently processing text information and enabling understanding connections in text that might not be readily apparent to humans. DARPA created the Deep Exploration and Filtering of Text (DEFT) program to harness the power of NLP. Sophisticated artificial intelligence of this nature has the potential to enable defense analysts to efficiently investigate orders of magnitude more documents so they can discover implicitly expressed, actionable information contained within them.
By building on the NLP technologies developed in other DARPA programs and ongoing academic research into deep language understanding and artificial intelligence, DEFT aims to address remaining capability gaps related to inference, causal relationships and anomaly detection. Improving human language technology to incorporate these capabilities is essential for enabling automated exposure of important content to facilitate analysis.
As a further aid to analysis, DEFT also aims to enable the capability to integrate individual facts into large domain models as information is processed to support assessment, planning, prediction and the initial stages of report writing. If successful, DEFT will allow analysts to move from limited, linear processing of huge sets of data to a nuanced, strategic exploration of available information.
The development of an automated solution may involve contributions from the linguistics and computer science fields in the areas of artificial intelligence, computational linguistics, machine learning, natural-language understanding, discourse and dialogue analysis, and others.
- QUOTE: Department of Defense (DoD) operators and analysts collect and process copious amounts of data from a wide range of sources to create and assess plans and execute missions. However, depending on context, much of the information that could support DoD missions may be implicit rather than explicitly expressed. Having the capability to automatically extract operationally relevant information that is only referenced indirectly would greatly assist analysts in efficiently processing data.