Gradient Boosting Classification System
A Gradient Boosting Classification System is a Gradient Boosting System that implements a Gradient Boosting Classification Algorithm to solve a Gradient Boosting Classification Task.
- AKA: Gradient Boosting Classifier.
- Context:
- It is an Gradient Boosting System for solving a Classification problem.
- …
- Example(s):
- Counter-Example(s):
- See: Random Forests Model, Out-of-Bag Error, Partial Permutation, K-Nearest Neighbor Algorithm.
References
- (Scikit-Learn, 2017) ⇒ http://scikit-learn.org/stable/auto_examples/ensemble/plot_feature_transformation.html
- QUOTE: Transform your features into a higher dimensional, sparse space. Then train a linear model on these features.
First fit an ensemble of trees (totally random trees, a random forest, or gradient boosted trees) on the training set. Then each leaf of each tree in the ensemble is assigned a fixed arbitrary feature index in a new feature space. These leaf indices are then encoded in a one-hot fashion.
Each sample goes through the decisions of each tree of the ensemble and ends up in one leaf per tree. The sample is encoded by setting feature values for these leaves to 1 and the other feature values to 0.
The resulting transformer has then learned a supervised, sparse, high-dimensional categorical embedding of the data.
- QUOTE: Transform your features into a higher dimensional, sparse space. Then train a linear model on these features.