Confusion Matrix

A confusion matrix is a square matrix that represents the count of a classification function's predictions with respect to the actual outcome on some labeled learning set.

• Context:
  • It has size L, where L is the number target labels.
  • It can illustrate a predictive algorithm's:
    • likelihood of predicting each one of L classes.
    • Predictions areas where the Function encounters difficulties.
  • It can be a 2x2 Table when it is a Binary Classification Model
• Example:
  • The sample confusion matrix below shows a confusion matrix for a classification task with the three (c=3) output classes: A, B and C. The test set used to evaluate the algorithm contained 100 cases with a distribution of 30 As, 35 Bs and 35 Cs. A perfect classifier would have only made predictions along the diagonal, but the results below show that the algorithm was only correct on (20+25+24)/100 = 69% of the cases. The matrix also shows that the classifier often confuses dairy for cans (11 incorrect) and cans for dairy (9 wrong).
    
    

• See: Cost-Benefit Matrix, Classification Task, Pearson, Correlation Coefficient.

References

2009

• (Wikipedia, 2009) http://en.wikipedia.org/wiki/Confusion_matrix
  • In the field of artificial intelligence, a confusion matrix is a visualization tool typically used in supervised learning (in unsupervised learning it is typically called a matching matrix). Each column of the matrix represents the instances in a predicted class, while each row represents the instances in an actual class. One benefit of a confusion matrix is that it is easy to see if the system is confusing two classes (i.e. commonly mislabelling one as another).
  • When a data set is unbalanced (when the number of samples in different classes vary greatly) the error rate of a classifier is not representative of the true performance of the classifier. This can easily be understood by an example: If there are for example 990 samples from class A and only 10 samples from class B, the classifier can easily be biased towards class A. If the classifier classifies all the samples as class A, the accuracy will be 99%. This is not a good indication of the classifier's true performance. The classifier has a 100% recognition rate for class A but a 0% recognition rate for class B.

1998

• (Kohavi & Provost, 1998) => Ron Kohavi, and Foster Provost. (1998). "Glossary of Terms." In: Machine Leanring 30(2-3).
  • Confusion matrix: A matrix showing the predicted and actual classifications. A confusion matrix is of size LxL, where L is the number of different label values. The following confusion matrix is for L=2:
    

    actual     \    predicted	negative	positive
    Negative	a	b
    Positive	c	d