Principal Components Regression Algorithm
A Principal Components Regression Algorithm is a regression analysis technique that ...
- Counter-Exapmle(s):
- See: Prediction, Statistics, Regression Analysis, Principal Component Analysis, Dependent And Independent Variables, Linear Regression, Estimation, Regularization (Mathematics), Variance, Eigenvalues And Eigenvectors, Sample Mean And Sample Covariance, Covariance Matrix.
References
2015
- (Wikipedia, 2015) ⇒ http://en.wikipedia.org/wiki/principal_component_regression Retrieved:2015-10-23.
- In statistics, principal component regression (PCR) is a regression analysis technique that is based on principal component analysis (PCA). Typically, it considers regressing the outcome (also known as the response or, the dependent variable) on a set of covariates (also known as predictors or, explanatory variables or, independent variables) based on a standard linear regression model, but uses PCA for estimating the unknown regression coefficients in the model.
In PCR, instead of regressing the dependent variable on the explanatory variables directly, the principal components of the explanatory variables are used as regressors. One typically uses only a subset of all the principal components for regression, thus making PCR some kind of a regularized procedure. Often the principal components with higher variances (the ones based on eigenvectors corresponding to the higher eigenvalues of the sample variance-covariance matrix of the explanatory variables) are selected as regressors. However, for the purpose of predicting the outcome, the principal components with low variances may also be important, in some cases even more important. One major use of PCR lies in overcoming the multicollinearity problem which arises when two or more of the explanatory variables are close to being collinear. [1] PCR can aptly deal with such situations by excluding some of the low-variance principal components in the regression step. In addition, by usually regressing on only a subset of all the principal components, PCR can result in dimension reduction through substantially lowering the effective number of parameters characterizing the underlying model. This can be particularly useful in settings with high-dimensional covariates. Also, through appropriate selection of the principal components to be used for regression, PCR can lead to efficient prediction of the outcome based on the assumed model.
- In statistics, principal component regression (PCR) is a regression analysis technique that is based on principal component analysis (PCA). Typically, it considers regressing the outcome (also known as the response or, the dependent variable) on a set of covariates (also known as predictors or, explanatory variables or, independent variables) based on a standard linear regression model, but uses PCA for estimating the unknown regression coefficients in the model.
- ↑ Dodge, Y. (2003) The Oxford Dictionary of Statistical Terms, OUP. ISBN 0-19-920613-9
2008
- (Upton & Cook, 2008) ⇒ Graham Upton, and Ian Cook. (2008). "A Dictionary of Statistics, 2nd edition revised." Oxford University Press. ISBN:0199541450
- QUOTE: Partial Least Squares (PLS): A method for handling correlated explanatory variables in the context of [[multiple regression. In PLS the first stage is to determine k uncorrelated variables that are linear combinations of the explanatory variables. The combinations are chosen for their predictive ability. Principal components regression analysis uses a different technique to achieve the same objective.
1992
- (Jolliffe, 1982) ⇒ Ian T. Jolliffe. (1982). "A Note on the Use of Principal Components in Regression." Applied Statistics