Precision @ K
Jump to navigation
Jump to search
A Precision @ K is an information discovery evaluation metric that ...
- Example(s):
- Precision@20.
- …
- Counter-Example(s):
- See: Binary Classification, Positive Predictive Value, Accuracy And Precision.
References
2018
- (Wikipedia, 2018) ⇒ https://en.wikipedia.org/wiki/Evaluation_measures_(information_retrieval)#Precision Retrieved:2018-6-22.
- Precision is the fraction of the documents retrieved that are relevant to the user's information need. : [math]\displaystyle{ \mbox{precision}=\frac{|\{\mbox{relevant documents}\}\cap\{\mbox{retrieved documents}\}|}{|\{\mbox{retrieved documents}\}|} }[/math] In binary classification, precision is analogous to positive predictive value. Precision takes all retrieved documents into account. It can also be evaluated at a given cut-off rank, considering only the topmost results returned by the system. This measure is called precision at n or P@n.
Note that the meaning and usage of "precision" in the field of information retrieval differs from the definition of accuracy and precision within other branches of science and statistics.
- Precision is the fraction of the documents retrieved that are relevant to the user's information need. : [math]\displaystyle{ \mbox{precision}=\frac{|\{\mbox{relevant documents}\}\cap\{\mbox{retrieved documents}\}|}{|\{\mbox{retrieved documents}\}|} }[/math] In binary classification, precision is analogous to positive predictive value. Precision takes all retrieved documents into account. It can also be evaluated at a given cut-off rank, considering only the topmost results returned by the system. This measure is called precision at n or P@n.
2018
- (Rybakov et al., 2018) ⇒ Oleg Rybakov, Vijai Mohan, Avishkar Misra, Scott LeGrand, Rejith Joseph, Kiuk Chung, Siddharth Singh, Qian You, Eric Nalisnick, Leo Dirac, and Runfei Luo. (2018). “The Effectiveness of a Two-layer Neural Network for Recommendations.”
- QUOTE: Precision at K is the accuracy of the predicted recommendations with respect to the actual purchases: : [math]\displaystyle{ Precision@K = \frac{1}{C} \frac {\Sigma^{C-1}_{c=0} \mid \{Rec_c\} \cap \{T_c\}) \mid} {K}, (1) }[/math] where K is the position/rank of a recommendation, c is the customer index, Recc is top K recommended items for customer c, Tc is actual consumptions for customer c represented as the set of items the customer purchased in the evaluation period (where interaction can be purchases, watches, listens), jRecj is the number of items in set Rec, Rec \ T is the intersection between sets Rec and T, and C is the number of customers.
While having high precision is necessary, it is not sufficient.
- QUOTE: Precision at K is the accuracy of the predicted recommendations with respect to the actual purchases: : [math]\displaystyle{ Precision@K = \frac{1}{C} \frac {\Sigma^{C-1}_{c=0} \mid \{Rec_c\} \cap \{T_c\}) \mid} {K}, (1) }[/math] where K is the position/rank of a recommendation, c is the customer index, Recc is top K recommended items for customer c, Tc is actual consumptions for customer c represented as the set of items the customer purchased in the evaluation period (where interaction can be purchases, watches, listens), jRecj is the number of items in set Rec, Rec \ T is the intersection between sets Rec and T, and C is the number of customers.