Decision Threshold
(Redirected from decision threshold)
Jump to navigation
Jump to search
A Decision Threshold is a value above which an instance can be considered to belong to the positive class.
See: Threshold, Decision Function, ROC Curve, Classification Algorithm, Classification Tree, Probability Threshold.
References
2011
- (Sammut & Webb, 2011) ⇒ Claude Sammut (editor), and Geoffrey I. Webb (editor). (2011). “Decision Threshold.” In: (Sammut & Webb, 2011) p.263
- Decision Threshold: The decision threshold of a binary classifier that outputs scores, such as decision trees or naive Bayes, is the value above which scores are interpreted as positive classifications. Decision thresholds can be either fixed if the classifier outputs calibrated scores on a known scale (e.g., 0.5 for a probabilistic classifier), or learned from data if the scores are uncalibrated.
2006
- (Chen et al., 2006) ⇒ Chen, J. J., Tsai, C. A., Moon, H., Ahn, H., Young, J. J., & Chen, C. H. (2006). Decision threshold adjustment in class prediction. SAR and QSAR in Environmental Research, 17(3), 337-352. DOI: 10.1080/10659360600787700.
- Abstract: Standard classification algorithms are generally designed to maximize the number of correct predictions (concordance). The criterion of maximizing the concordance may not be appropriate in certain applications. In practice, some applications may emphasize high sensitivity (e.g., clinical diagnostic tests) and others may emphasize high specificity (e.g., epidemiology screening studies). This paper considers effects of the decision threshold on sensitivity, specificity, and concordance for four classification methods: logistic regression, classification tree, Fisher’s linear discriminant analysis, and a weighted k-nearest neighbor. We investigated the use of decision threshold adjustment to improve performance of either sensitivity or specificity of a classifier under specific conditions. We conducted a Monte Carlo simulation showing that as the decision threshold increases, the sensitivity decreases and the specificity increases; but, the concordance values in an interval around the maximum concordance are similar. For specified sensitivity and specificity levels, an optimal decision threshold might be determined in an interval around the maximum concordance that meets the specified requirement. Three example data sets were analyzed for illustrations.
2004
- (Chawla et al., 2004) ⇒ Nitesh Chawla, Nathalie Japkowicz, and Aleksander Kolcz. (2004). “Editorial: Special issue on learning from imbalanced data sets.” In: ACM SIGKDD Explorations Newsletter, 6(1). doi:10.1145/1007730.1007733 PDF
- QUOTE: At the data level, these solutions include many different forms of re-sampling such as random oversampling with replacement, random undersampling, directed oversampling (in which no new examples are created, but the choice of samples to replace is informed rather than random), directed undersampling (where, again, the choice of examples to eliminate is informed), oversampling with informed generation of new samples, and combinations of the above techniques. At the algorithmic level, solutions include adjusting the costs of the various classes so as to counter the class imbalance, adjusting the probabilistic estimate at the tree leaf (when working with decision trees), adjusting the decision threshold, and recognition-based (i.e., learning from one class) rather than discrimination-based (two class) learning.
1997
- (Bradley, 1997) ⇒ Andrew P. Bradley. (1997). “The Use of the Area under the ROC Curve in the Evaluation of Machine Learning Algorithms." Pattern recognition 30, no. 7
- QUOTE: We compare and discuss the use of AUC to the more conventional overall accuracy and find that AUC exhibits a number of desirable properties when compared to overall accuracy: increased sensitivity in Analysis of Variance (ANOVA) tests; a standard error that decreased as both AUC and the number of test samples increased; decision threshold independent; and it is invariant to a priori class probabilities.