(Redirected from bias-variance tradeoff)
- See: Errors And Residuals in Statistics, Supervised Learning, Estimator, Overfitting, Expected Value, Bias, Variance.
- (Wikipedia, 2017) ⇒ https://en.wikipedia.org/wiki/Bias–variance_tradeoff Retrieved:2017-11-25.
- In statistics and machine learning, the bias–variance tradeoff (or dilemma) is the problem of simultaneously minimizing two sources of error that prevent supervised learning algorithms from generalizing beyond their training set:* The bias is error from erroneous assumptions in the learning algorithm. High bias can cause an algorithm to miss the relevant relations between features and target outputs (underfitting).
- The bias–variance decomposition is a way of analyzing a learning algorithm's expected generalization error with respect to a particular problem as a sum of three terms, the bias, variance, and a quantity called the irreducible error, resulting from noise in the problem itself.
This tradeoff applies to all forms of supervised learning: classification, regression (function fitting), and structured output learning. It has also been invoked to explain the effectiveness of heuristics in human learning.
- (Rajnarayan & Wolpert, 2011) ⇒ Dev Rajnarayan, and David Wolpert. (2011). “Bias-Variance Trade-offs; Novel Applications.” In: (Sammut & Webb, 2011) p.101
- (Bouchard & Triggs, 2004) ⇒ Guillaume Bouchard, and Bill Triggs. (2004). “The Trade-off Between Generative and Discriminative Classifiers.” In: Proceedings of COMPSTAT 2004.
- QUOTE: … The key argument is that the discriminative estimator converges to the conditional density that minimizes the negative log-likelihood classification loss against the true density p(x, y) . For finite sample sizes, there is a bias-variance tradeoff and it is less obvious how to choose between generative and discriminative classifiers.
- (Kohavi & Wolpert, 1996) ⇒ Ron Kohavi, and David H. Wolpert. (1996). “Bias Plus Variance Decomposition for Zero-One Loss Functions.” In: Proceedings of the 13th International Conference on Machine Learning (ICML 1996).