Accuracy Metric

An accuracy metric is a classifier performance metric based on the proportion of the classifier's correct classifications to incorrect classifications (on labeled testing records).

• AKA: Accuracy Function, Accuracy, Accuracy Measure.
• Context:
  • It can be estimated by an Accuracy Estimation Process.
  • It can be calculated:
    • by (TP+TN)/(TP+TN+FP+FN), for a Two-Class Problem.
    • by counting the correct classifications and dividing by the number of classifications made.
  • It can be reported as the rate which a case will be labeled with the right category, if the Predictive Model is a Classifier.
  • It can be reported as the average distance between the predicted label and the correct value, if the Predictive Model is an Estimator.
  • It is (typically) required that the Test Case be unseen during the Training Phase.
  • It can be the Inverse Function to the Error Rate Function.
• Example(s):
  • A Classifier may be said to Predict whether a Customer responds to a promotional campaign with 85.5% Accuracy.
• Counter-Example(s):
  • An Estimator may be said to Predict the amount a Customer will spend on a given month within $55.
  • True Positive Rate.
  • True Negative Rate.
  • False Positive Rate.
• See: Precision, Recall, F-Measure, Statistical Significance, Cross-validation.

References

2002

• ((Gabor Melli, 2002)) => Gabor Melli. (2002). "PredictionWorks' Data Mining Glossary.
  • Accuracy: The measure of a model's ability to correctly label a previously unseen test case. If the label is categorical (classification), accuracy is commonly reported as the rate which a case will be labeled with the right category. For example, a model may be said to predict whether a customer responds to a promotional campaign with 85.5% accuracy. If the label is continuous, accuracy is commonly reported as the average distance between the predicted label and the correct value. For example, a model may be said to predict the amount a customer will spend on a given month within $55. See also Accuracy Estimation, Classification, Estimation, Model, and Statistical Significance.

1998

• (Kohavi & Provost, 1998) => Ron Kohavi, and Foster Provost. (1998). "Glossary of Terms." In: Machine Leanring 30(2-3).
  • Accuracy (error rate): The rate of correct (incorrect) predictions made by the model over a data set (cf. coverage). Accuracy is usually estimated by using an independent test set that was not used at any time during the learning process. More complex accuracy estimation techniques, such as cross-validation and the bootstrap, are commonly used, especially with data sets containing a small number of instances.