Bayesian Ridge Regression System

A Bayesian Ridge Regression System is a Bayesian Regression System that implements an Ridge Regression Algorithm to solve a Bayesian Ridge Regression Task.

• AKA: Bayesian Ridge System.
• Context:
  • It is based on a Bayesian Probabilistic System.
  • …
• Example(s):
  • sklearn.linear_model.BayesianRidge [1]
    • Scikit-Learn Example: Computation of Bayesian Ridge Regression on a synthetic dataset.
  • …
• Counter-Example(s):
  • Ridge Regression System.
  • LASSO Regression System.
  • ARD Regression System.
  • ElasticNet System.
• See: Bayesian Statistics, Prior Probability, Posterior Probability.

---

---

References

2017

• (Scikit Learn, 2017) ⇒ 1.1.10.1. Bayesian Ridge Regression Retrieved:2017-09-17
  • QUOTE: BayesianRidge estimates a probabilistic model of the regression problem as described above. The prior for the parameter [math]\displaystyle{ w }[/math] is given by a spherical Gaussian:

    [math]\displaystyle{ p(w|\lambda) = \mathcal{N}(w|0,\lambda^{-1}\mathbf{I_{p}}) }[/math]

    The priors over [math]\displaystyle{ \alpha }[/math] and [math]\displaystyle{ \lambda }[/math] are chosen to be gamma distributions, the conjugate prior for the precision of the Gaussian.

    The resulting model is called Bayesian Ridge Regression, and is similar to the classical Ridge. The parameters [math]\displaystyle{ w }[/math], [math]\displaystyle{ \alpha }[/math] and [math]\displaystyle{ \lambda }[/math] are estimated jointly during the fit of the model. The remaining hyperparameters are the parameters of the gamma priors over [math]\displaystyle{ \alpha }[/math] and [math]\displaystyle{ \lambda }[/math]. These are usually chosen to be non-informative. The parameters are estimated by maximizing the marginal log likelihood.

    By default [math]\displaystyle{ \alpha_1 = \alpha_2 = \lambda_1 = \lambda_2 = 10^{-6} }[/math]

    (...)

---