sklearn.linear model.ARDRegression

From GM-RKB
Jump to: navigation, search

A sklearn.linear_model.ARDRegression is a ARD Regression System within sklearn.linear_model class.

1) Import ARD Regression model from scikit-learn : from sklearn.linear_model import ARDRegression
2) Create design matrix X and response vector Y
3) Create ARD Regression object: model=ARDRegression([n_iter=n_iter, tol=tol, alpha_1=alpha1, alpha_2=alpha1, lambda_1=lambda1, lambda_2=lambda2,...])
4) Choose method(s):
Input: Output:
#Importing modules
from sklearn.linear_model import ARDRegression
from sklearn.model_selection import cross_val_predict
from sklearn.datasets import load_boston
from sklearn.metrics import explained_variance_score, mean_squared_error
import numpy as np
import pylab as pl
boston = load_boston() #Loading boston datasets
x = boston.data # Creating Regression Design Matrix
y = boston.target # Creating target dataset
ARD= ARDRegression(alpha_1=0.01, alpha_2=0.01, lambda_1=1e-06, lambda_2=1e-06) # Create ARD regression object for the hyperparameters alpha1,2=0.01 and lambda1,2=1e-6
ARD.fit(x,y) # predicted values

#Calculaton of RMSE and Explained Variances

yp_cv = cross_val_predict(ARD, x, y, cv=10) #Calculation 10-Fold CV
Evariance=explained_variance_score(y,yp)
Evariance_cv=explained_variance_score(y,yp_cv)
RMSE =np.sqrt(mean_squared_error(y,yp))
RMSECV =sqrt(mean_squared_error(y,yp_cv)

# Printing Results

print('Method: ARDRegression Regression')
print('RMSE on the dataset: %.4f' %RMSE)
print('RMSE on 10-fold CV: %.4f' %RMSECV)
print('Explained Variance Regression Score on the dataset: %.4f' %Evariance)
print('Explained Variance Regression 10-fold CV: %.4f' %Evariance_cv)

#plotting real vs predicted data

pl.figure(1)
pl.plot(yp, y,'ro')
pl.plot(yp_cv, y,'bo', alpha=0.25, label='10-folds CV')
pl.xlabel('predicted')
pl.title('ARD Regression, alpha1,2=0.01 lambad1,2=1e-6')
pl.ylabel('real')
pl.grid(True)
pl.show()
ard boston10fold.png
(blue dots correspond to 10-Fold CV)


Method: ARD Regression
RMSE on the dataset: 4.7411
RMSE on 10-fold CV: 5.9097
Explained Variance Regression Score on the dataset : 0.7337
Explained Variance Regression 10-fold CV: 0.5863


References

2017

Bayesian ARD regression.
Fit the weights of a regression model, using an ARD prior. The weights of the regression model are assumed to be in Gaussian distributions. Also estimate the parameters lambda (precisions of the distributions of the weights) and alpha (precision of the distribution of the noise). The estimation is done by an iterative procedures (Evidence Maximization)noise).