Annotation Task

An Annotation Task is a semantic analysis task that requires the addition of one or more annotation items to an artifact to demarcate the structure.

• Context:
  • Input: an Artifact.
  • output: an Annotated Artifact.
  • Task Performance Metrics: Accuracy, Precision, Recall, F-Measure, Annotator Effort.
  • It can range from being a Text Annotation Task (e.g. document annotation, wikitext annotation, software code annotation) to being an Image Annotation Task to being an Audio Annotation Task to being a Multimedia Annotation Task.
  • It can range from being a Manual Annotation Task (with a human annotator) to being a Computer-Assisted Annotation Task (with an Interactive Annotation System) to being a Fully-Automated Annotation Task (solved by a fully-automated annotation system).
  • It can range from being a Syntactic Annotation Task (of syntactic structure) to being from a Semantic Annotation Task (of semantic structure)
  • It can range from being a Simple Annotation Task to being a Complex Annotation Task.
  • It can range from being an Independent Annotation Task to being a Collaborative Annotation Task.
  • It can be solved by an Annotation System.
  • It can support a Curation Task.
  • It can involves Metadata Creation.
  • It can require Domain-Specific Knowledge for accurate annotation, especially in fields like Biomedical Annotation or Legal Annotation.
  • It often requires adherence to specific Annotation Guidelines to ensure consistency and reliability across annotators and datasets.
  • …
• Example(s):
  • a Text Annotation Task
    • document annotation
    • wikitext annotation
    • software code annotation
  • an Image Annotation Task, such as labeling objects in images for computer vision datasets.
  • an Audio Annotation Task, such as transcribing speech or annotating audio files for sound events.
  • a Multimedia Annotation Task, including video annotation for events or object tracking.
  • a Tagging Task, involving assigning keywords or tags to pieces of content for easier retrieval or categorization.
  • a Syntactic Annotation Task, such as POS Tagging.
  • a Semantic Annotation Task, such as Concept Mention Annotation Task, Knowledge Annotation Task.
  • a Chatbot Answer Scoring Task, evaluating chatbot responses for relevance and accuracy.
  • A Video Annotation Task, where videos are annotated for events or object tracking.
  • A Dataset Annotation Task, for creating or enhancing datasets used in machine learning and AI research.
  • …
• Counter-Example(s):
  • a Syntactic Parsing Task, which involves analyzing the syntactic structure of sentences rather than annotating.
  • an Editing Task, focused on correcting language rather than adding metadata or labels.
  • a Data Cleaning Task, which involves removing or correcting inaccuracies in data without necessarily adding annotations.
• See: Complex-Input Classification, Data Annotation, Machine Learning, Natural Language Processing.

---

---

References

2011

• (Wikipedia - Annotation, 2009) ⇒ http://en.wikipedia.org/wiki/Annotation
  • For DNA annotation, a previously unknown sequence representation of genetic material is enriched with information relating genomic position to intron-exon boundaries, regulatory sequences, repeats, gene names and protein products. This annotation is stored in genomic databases as Mouse Genome Informatics, FlyBase, and WormBase. Educational materials on some aspects of biological annotation from this year's Gene Ontology annotation camp and similar events are available at the Gene Ontology website. The National Center for Biomedical Ontology (www.bioontology.org) develops tools for automated annotation of database records based on the textual descriptions of those records.

    In the digital imaging community the term annotation is commonly used for visible metadata superimposed on an image without changing the underlying master image, such as sticky notes, virtual laser pointers, circles, arrows, and black-outs (cf. redaction).

    … legal publishers such as Thomson West and Lexis Nexis publish annotated versions of statutes, providing information about court cases that have interpreted the statutes. Both the federal United States Code and state statutes are subject to interpretation by the courts, and the annotated statutes are valuable tools in legal research.

    In linguistics, annotation include comments and metadata; these non-transcriptional annotations are also non-linguistic. A collection of texts with linguistic annotations is known as a corpus (plural corpora). The Linguistic Annotation Wiki describes tools and formats for creating and managing linguistic annotations.

2009

• (WordNet, 2009) ⇒ http://wordnetweb.princeton.edu/perl/webwn?s=annotation
  • S: (n) annotation, annotating (the act of adding notes)
  • …

2009

• http://en.wiktionary.org/wiki/annotate
  • To add annotation

2009

• http://en.wiktionary.org/wiki/annotation#Noun
  • the process of writing such comment or commentary
  • a critical or explanatory commentary or analysis.
  • a comment added to a text

2006

• (Bukhardt et al., 2006) ⇒ Kyle Burkhardt, Bohdan Schneider, Jeramia Ory. (2006). “A Biocurator Perspective: Annotation at the Research Collaboratory for Structural Bioinformatics Protein Data Bank.” In: PLoS Computational Biology, 2(10). doi:10.1371/journal.pcbi.0020099
  • QUOTE: The goal of annotation is to make each entry not only self-consistent but also consistent with the rest of the archive. To this end, annotators help authors represent their data in the best possible way. Annotators routinely review the incoming data and perform many standard inspections (see Box 1).

Box 1. Annotators Work to Represent PDB Data in the Best Possible Way by:
   * Reviewing entry for self-consistency
   * Matching given title to structure
   * Correcting format errors in data and coordinates
   * Checking sequence using BLAST [13]
   * Inserting sequence database reference
   * Providing protein name and synonyms
   * Checking scientific name of the source organism
   * Confirming chemical consistency between ligand name and the 3-D coordinates
   * Adding information describing the biological assembly
   * Checking entry visually
   * Generating validation reports
   * Finding citation references with PubMed [14]

---