Generalized Linear Least-Squares Algorithm

Jump to: navigation, search

A Generalized Linear Least-Squares Algorithm is a linear least-squares algorithm that estimates the unknown function parameters using linear regression.



    • In a typical linear regression model we observe data [math]\{y_i,x_{ij}\}_{i=1..n,j=1..p}[/math] on n statistical units. The response values are placed in a vector Y = (y1, ..., yn)′, and the predictor values are placed in the design matrix X = [[xij]], where xij is the value of the jth predictor variable for the ith unit. The model assumes that the conditional mean of Y given X is a linear function of X, whereas the conditional variance of the error term given X is a known matrix Ω. This is usually written as : [math] Y = X\beta + \varepsilon, \qquad \mathrm{E}[\varepsilon|X]=0,\ \operatorname{Var}[\varepsilon|X]=\Omega. [/math] Here β is a vector of unknown “regression coefficients” that must be estimated from the data.

      Suppose b is a candidate estimate for β. Then the residual vector for b will be Y − Xb. Generalized least squares method estimates β by minimizing the squared Mahalanobis length of this residual vector: : [math] \hat\beta = \underset{b}{\rm arg\,min}\,(Y-Xb)'\,\Omega^{-1}(Y-Xb), [/math] Since the objective is a quadratic form in b, the estimator has an explicit formula: : [math] \hat\beta = (X'\Omega^{-1}X)^{-1} X'\Omega^{-1}Y. [/math]