Information Retrieval Evaluation Measure

Jump to navigation Jump to search

An Information Retrieval Evaluation Measure is a Performance Measure that is used to evaluate the effectiveness of an Information Retrieval Systems.



  • (Wikipedia, 2023) ⇒ Retrieved:2023-9-29.
    • Evaluation measures for an information retrieval (IR) system assess how well an index, search engine or database returns results from a collection of resources that satisfy a user's query. They are therefore fundamental to the success of information systems and digital platforms. The success of an IR system may be judged by a range of criteria including relevance, speed, user satisfaction, usability, efficiency and reliability. However, the most important factor in determining a system's effectiveness for users is the overall relevance of results retrieved in response to a query. Evaluation measures may be categorised in various ways including offline or online, user-based or system-based and include methods such as observed user behaviour, test collections, precision and recall, and scores from prepared benchmark test sets. Evaluation for an information retrieval system should also include a validation of the measures used, i.e. an assessment of how well they measure what they are intended to measure and how well the system fits its intended use case. Measures are generally used in two settings: online experimentation, which assesses users' interactions with the search system, and offline evaluation, which measures the effectiveness of an information retrieval system on a static offline collection.


  • (Wikipedia, 2018) ⇒ Retrieved:2018-2-18.
    • Evaluation measures for an information retrieval system are used to assess how well the search results satisfied the user's query intent. Such metrics are often split into kinds: online metrics look at users' interactions with the search system, while offline metrics measure relevance, in other words how likely each result, or SERP page as a whole, is to meet the information needs of the user.

      The mathematical symbols used in the formulas below mean:

      • [math]\displaystyle{ X \cap Y }[/math] - Intersection - in this case, specifying the documents in both sets X and Y
      • [math]\displaystyle{ | X | }[/math] - Cardinality - in this case, the number of documents in set X
      • [math]\displaystyle{ \int }[/math] - Integral.
      • [math]\displaystyle{ \sum }[/math] - Summation.
      • [math]\displaystyle{ \Delta }[/math] - Symmetric difference