Covariance Matrix

A covariance matrix is a (symmetric) positive semi-definite matrix of covariances between elements of a random vector (in which the Main Diagonal are variances and the remaining elements are covariances).

• AKA: Dispersion Matrix, Variance-Covariance Matrix.
• Context:
  • It can range from being a Population Covariance Matrix to being a Sample Covariance Matrix.
  • It can range from being an Within-Class Covariance Matrix to being an Between-Class Covariance Matrix.
• Counter-Example(s):
  • Inverse Covariance Matrix.
• See: Statistical Inference; Covariance Matrix Estimation Algorithm; Gaussian Distribution; Gaussian Processes; Kernel Methods.

---

---

References

2013

• en.wikipedia.org/wiki/Covariance_matrix
  • In probability theory and statistics, a covariance matrix (also known as dispersion matrix or variance covariance matrix) is a matrix whose element in the i, j position is the covariance between the i ^th and j ^th elements of a random vector (that is, of a vector of random variables). Each element of the vector is a scalar random variable, either with a finite number of observed empirical values or with a finite or infinite number of potential values specified by a theoretical joint probability distribution of all the random variables.

    Intuitively, the covariance matrix generalizes the notion of variance to multiple dimensions. As an example, the variation in a collection of random points in two-dimensional space cannot be characterized fully by a single number, nor would the variances in the x and y directions contain all of the necessary information; a 2×2 matrix would be necessary to fully characterize the two-dimensional variation.

• http://en.wikipedia.org/wiki/Covariance_matrix#Definition
  • QUOTE: Throughout this article, boldfaced unsubscripted X and Y are used to refer to random vectors, and unboldfaced subscripted X_i and Y_i are used to refer to random scalars.

    If the entries in the column vector \[ \mathbf{X} = \begin{bmatrix}X_1 \\ \vdots \\ X_n \end{bmatrix}\] are random variables, each with finite variance, then the covariance matrix Σ is the matrix whose (i, j) entry is the covariance \[ \Sigma_{ij} = \mathrm{cov}(X_i, X_j) = \mathrm{E}\begin{bmatrix} (X_i - \mu_i)(X_j - \mu_j) \end{bmatrix} \] where \[ \mu_i = \mathrm{E}(X_i)\, \] is the expected value of the ith entry in the vector X. In other words, we have \[ \Sigma = \begin{bmatrix} \mathrm{E}[(X_1 - \mu_1)(X_1 - \mu_1)] & \mathrm{E}[(X_1 - \mu_1)(X_2 - \mu_2)] & \cdots & \mathrm{E}[(X_1 - \mu_1)(X_n - \mu_n)] \\ \\ \mathrm{E}[(X_2 - \mu_2)(X_1 - \mu_1)] & \mathrm{E}[(X_2 - \mu_2)(X_2 - \mu_2)] & \cdots & \mathrm{E}[(X_2 - \mu_2)(X_n - \mu_n)] \\ \\ \vdots & \vdots & \ddots & \vdots \\ \\ \mathrm{E}[(X_n - \mu_n)(X_1 - \mu_1)] & \mathrm{E}[(X_n - \mu_n)(X_2 - \mu_2)] & \cdots & \mathrm{E}[(X_n - \mu_n)(X_n - \mu_n)] \end{bmatrix}. \] The inverse of this matrix, \(\Sigma^{-1}\) is the inverse covariance matrix, also known as the concentration matrix or precision matrix; see precision (statistics). The elements of the precision matrix have an interpretation in terms of partial correlations and partial variances.^{[citation needed]}

2011

• (Zhang, 2011a) ⇒ Xinhua Zhang. (2011). "Covariance Matrix." In: (Sammut & Webb, 2011) p.235

2008

• (Upton & Cook, 2008) ⇒ Graham Upton, and Ian Cook. (2008). "A Dictionary of Statistics, 2nd edition revised." Oxford University Press. ISBN 0199541450
  • QUOTE: variance-covariance matrix (dispersion matrix) A square symmetric *matrix in which the elements on the main diagonal are *variances and the remaining elements are *covariances. ...

2006

• (Cox, 2006) ⇒ David R. Cox. (2006). "Principles of Statistical Inference." Cambridge University Press. ISBN 9780521685672

---