sklearn.ensemble.AdaBoostClassifier

From GM-RKB
Jump to navigation Jump to search

A sklearn.ensemble.AdaBoostClassifier is an AdaBoost Classification System within sklearn.ensemble module.

1) Import AdaBoost Classification System from scikit-learn : from sklearn.ensemble import AdaBoostClassifier
2) Create design matrix X and response vector Y
3) Create AdaBoost Classifier object: classifier_model=AdaBoostClassifier([base_estimator=None, n_estimators=50, learning_rate=1.0, algorithm=’SAMME.R’, random_state=None])
4) Choose method(s):
  • decision_function(X), computes the decision function of X.
  • fit(X, y[, sample_weight]), builds an AdaBoost classifier from the training set (X, y).
  • get_params([deep]), gets parameters for this estimator.
  • predict(X), predicts classes for X.
  • predict_log_proba(X), predicts class log-probabilities for X.
  • predict_proba(X), predicts class probabilities for X.
  • score(X, y[, sample_weight]), returns the mean accuracy on the given test data and labels.
  • set_params(**params), sets the parameters of this estimator.
  • staged_decision_function(X), computes decision function of X for each boosting iteration.
  • staged_predict(X), returns staged predictions for X.
  • staged_predict_proba(X), predicts class probabilities for X.
  • staged_score(X, y[, sample_weight]), returns staged scores for X, y.


References

2017a

2017B

  • (Wikipedia, 2017A) ⇒ https://en.wikipedia.org/wiki/AdaBoost Retrieved:2017-10-22.
    • AdaBoost, short for "Adaptive Boosting", is a machine learning meta-algorithm formulated by Yoav Freund and Robert Schapire who won the Gödel Prize in 2003 for their work. It can be used in conjunction with many other types of learning algorithms to improve their performance. The output of the other learning algorithms ('weak learners') is combined into a weighted sum that represents the final output of the boosted classifier. AdaBoost is adaptive in the sense that subsequent weak learners are tweaked in favor of those instances misclassified by previous classifiers. AdaBoost is sensitive to noisy data and outliers. In some problems it can be less susceptible to the overfitting problem than other learning algorithms. The individual learners can be weak, but as long as the performance of each one is slightly better than random guessing (e.g., their error rate is smaller than 0.5 for binary classification), the final model can be proven to converge to a strong learner.

      Every learning algorithm will tend to suit some problem types better than others, and will typically have many different parameters and configurations to be adjusted before achieving optimal performance on a dataset, AdaBoost (with decision trees as the weak learners) is often referred to as the best out-of-the-box classifier. When used with decision tree learning, information gathered at each stage of the AdaBoost algorithm about the relative 'hardness' of each training sample is fed into the tree growing algorithm such that later trees tend to focus on harder-to-classify examples.

2017c


  1. Y. Freund, and R. Schapire, “A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting”, 1997
  2. T. Hastie, R. Tibshirani and J. Friedman, “Elements of Statistical Learning Ed. 2”, Springer, 2009.
  3. J. Zhu, H. Zou, S. Rosset, T. Hastie. “Multi-class AdaBoost”, 2009.
  4. H.Drucker. “Improving Regressors using Boosting Techniques”, 1997.