Sample Covariance Function

A Sample Covariance Function is a Covariance Function that is a Sample Statistic Function.

• Example(s):
  • Covariance Matrix.
• Counter-Example(s):
  • Sample Covariance Value.
  • Sample Variance Function.
  • Sample Standard Deviation Function.
  • Sample Mean Function.
• See: Probability Theory.

---

---

References

2017

• (Wikipedia, 2017) ⇒ http://en.wikipedia.org/wiki/Covariance#Definition
  • The covariance between two jointly distributed real-valued random variables X and Y with finite second moments is defined as

[math]\displaystyle{ \operatorname{cov}(X,Y) = \operatorname{E}{\big[(X - \operatorname{E}[X])(Y - \operatorname{E}[Y])\big]}, }[/math]

where E[X] is the expected value of X, also known as the mean of X. The covariance is also sometimes denoted “σ”, in analogy to variance. By using the linearity property of expectations, this can be simplified to

[math]\displaystyle{ \begin{align} \operatorname{cov}(X, Y) &= \operatorname{E}\left[\left(X - \operatorname{E}\left[X\right]\right) \left(Y - \operatorname{E}\left[Y\right]\right)\right] \\ &= \operatorname{E}\left[X Y - X \operatorname{E}\left[Y\right] - \operatorname{E}\left[X\right] Y + \operatorname{E}\left[X\right] \operatorname{E}\left[Y\right]\right] \\ &= \operatorname{E}\left[X Y\right] - \operatorname{E}\left[X\right] \operatorname{E}\left[Y\right] - \operatorname{E}\left[X\right] \operatorname{E}\left[Y\right] + \operatorname{E}\left[X\right] \operatorname{E}\left[Y\right] \\ &= \operatorname{E}\left[X Y\right] - \operatorname{E}\left[X\right] \operatorname{E}\left[Y\right]. \end{align} }[/math]

However, when [math]\displaystyle{ \operatorname{E}[XY] \approx \operatorname{E}[X]\operatorname{E}[Y] }[/math], this last equation is prone to catastrophic cancellation when computed with floating point arithmetic and thus should be avoided in computer programs when the data has not been centered before. Numerically stable algorithms should be preferred in this case.

For random vectors [math]\displaystyle{ \mathbf{X} \in \mathbb{R}^m }[/math] and [math]\displaystyle{ \mathbf{Y} \in \mathbb{R}^n }[/math], the m × n cross covariance matrix (also known as dispersion matrix or variance–covariance matrix, or simply called covariance matrix) is equal to

[math]\displaystyle{ \begin{align} \operatorname{cov}(\mathbf{X},\mathbf{Y}) & = \operatorname{E} \left[(\mathbf{X} - \operatorname{E}[\mathbf{X}]) (\mathbf{Y} - \operatorname{E}[\mathbf{Y}])^\mathrm{T}\right]\\ & = \operatorname{E}\left[\mathbf{X} \mathbf{Y}^\mathrm{T}\right] - \operatorname{E}[\mathbf{X}]\operatorname{E}[\mathbf{Y}]^\mathrm{T}, \end{align} }[/math]

where m^T is the transpose of the vector (or matrix) m.

The (i, j)-th element of this matrix is equal to the covariance cov(X_i, Y_j) between the i-th scalar component of X and the j-th scalar component of Y. In particular, cov(Y, X) is the transpose of cov(X, Y).

For a vector [math]\displaystyle{ \mathbf{X} = \begin{bmatrix}X_1 & X_2 & \dots & X_m\end{bmatrix}^\mathrm{T} }[/math] of m jointly distributed random variables with finite second moments, its covariance matrix is defined as

[math]\displaystyle{ \Sigma(\mathbf{X}) = \operatorname{cov}(\mathbf{X}, \mathbf{X}). }[/math]

Random variables whose covariance is zero are called uncorrelated. Similarly, random vectors whose covariance matrix is zero in every entry outside the main diagonal are called uncorrelated.

The units of measurement of the covariance cov(X, Y) are those of X times those of Y. By contrast, correlation coefficients, which depend on the covariance, are a dimensionless measure of linear dependence. (In fact, correlation coefficients can simply be understood as a normalized version of covariance.)

---