A Linear Regression Task is a regression analysis task that is based on linear predictor functions.

$y_i=f(x_i,\boldsymbol\beta)+\varepsilon_i\quad$ with $f(x_i,\beta_j )=\sum _{j=0}^{m}\beta _{j}\phi_{j}(x_i)$ for $\quad i=1,\cdots,n \;$ and $j=0,\cdots, p$
in this regression function $f(X)=f(x_i,\boldsymbol\beta)$ is a parametric regression function which is a linear combination between $p$ regression coefficients $\beta_j$ (parameters) and basis functions $\phi_j(x_i)$. Usually, the basis function are polynomial, i.e. $\phi_j(x^i)=x_i^{j}$
or
$\mathbf{Y} = \mathbf{X}\mathbf{B} + \mathbf{U},$ where $\mathbf{Y}=y_i$ is the measurement matrix, $\mathbf{X}=\phi _{j}(x_i)$ is design matrix, $\mathbf{B}=\boldsymbol{\beta}=\beta_j$ the parameters matrix and $\mathbf{U}=\varepsilon_i$ a errors matrix. This is,

$\begin{pmatrix} y_1 \\ y_2 \\ \vdots \\ y_n \end{pmatrix} = \begin{pmatrix} \phi_0(x_1) & \phi_1(x_1) & \cdots & \phi_p(x_1) \\ \phi_0(x_2) & \phi_1(x_2) & \cdots & \phi_p{x_2} \\ \vdots & \vdots & \ddots & \vdots \\ \phi_0(x_n) & \phi_1(x_n) & \cdots & \phi_p(x_n) \end{pmatrix}\begin{pmatrix} \beta_0 \\ \beta_1 \\ \vdots \\ \beta_p \end{pmatrix}+\begin{pmatrix} \varepsilon_1 \\ \varepsilon_2 \\ \vdots \\ \varepsilon_n \end{pmatrix}$

by estimating the best-fitting $\beta$ parameters that optimizes the following objective function:

$E(f)=\sum _{i=1}^{n}L(y_{i},f(x_{i},{\boldsymbol \beta }))$

$L(\cdot)$ is an error function that may be derived as a loss function or the negative of a likelihood function.

• Example(s):
• A numerical experiment resulted in the four $(x, y)$ data points ${(1, 6), (2, 5), (3, 7), (4, 10)}$, find a line $y=\beta_1+\beta_2 x$ that best fits these four points
e.g. ⇒ $y=3.5+1.4x$.
• A Simple Linear Regression Task: $y_i=\beta_0+\beta_1 x_i+\varepsilon_i,\quad i=1,\cdots ,n\;$,

$\begin{pmatrix} y_1 \\ y_2 \\ \vdots \\ y_n \end{pmatrix} = \begin{pmatrix} 1 & x_1 \\ 1 & x_2 \\ \vdots & \vdots\\ 1 & x_n \\ \end{pmatrix}\begin{pmatrix} \beta_0 \\ \beta_1 \\ \end{pmatrix}+\begin{pmatrix} \varepsilon_0 \\ \varepsilon_1 \\ \vdots \\ \varepsilon_n \end{pmatrix}$

• The regression problem: $y_i=\beta_0+\beta_1 x_i+\beta_2x_i^2+\varepsilon_i,\quad i=1,\cdots ,n\;$,

$\begin{pmatrix} y_1 \\ y_2 \\ \vdots \\ y_n \end{pmatrix} = \begin{pmatrix} 1 & x_1&x_1^2\\ 1 & x_2&x_2^2 \\ \vdots & \vdots&\vdots\\ 1 & x_n&x_n^2 \\ \end{pmatrix}\begin{pmatrix} \beta_0 \\ \beta_1 \\ \beta_2\\ \end{pmatrix}+\begin{pmatrix} \varepsilon_0 \\ \varepsilon_1 \\ \vdots \\ \varepsilon_n \end{pmatrix}$

Although, it includes a quadratic term, this is still linear in the regression parameters.

• A Multivariate Linear Regression Task.
• A Regularized Linear Regression Task.
• A Linear Least-Squares Regression Task
• Counter-Example(s):
• See: Curve Fitting, System of Linear Equations, Linear Model.