covariance: Definition from Answers.com

This article is about the measure of correlation between random variables. For other uses, see Covariance (disambiguation).

This article does not cite any references or sources.
Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed. (August 2009)

In probability theory and statistics, covariance is a measure of how much two variables change together. (Variance is a special case of the covariance when the two variables are identical.)

1 Definition
2 Properties
- 2.1 Incremental computation
- 2.2 Relationship to inner products
3 Covariance matrix, operator, bilinear form, and function
4 Comments
5 See also
6 External links
7 References

Definition

The covariance between two real-valued random variables X and Y, with expected values $\scriptstyle \operatorname E(X)\,=\,\mu$ and $\scriptstyle \operatorname E(Y)\,=\,\nu$ is defined as

$\operatorname{Cov}(X, Y) = \operatorname{E}((X - \mu) (Y - \nu)), \,$

where E is the expected value operator. This can also be written as

$\operatorname{Cov}(X, Y) = \operatorname{E}(X \cdot Y - \mu Y - \nu X + \mu \nu), \,$

$\operatorname{Cov}(X, Y) = \operatorname{E}(X \cdot Y) - \mu \operatorname{E}(Y) - \nu \operatorname{E}(X) + \mu \nu, \,$

$\operatorname{Cov}(X, Y) = \operatorname{E}(X \cdot Y) - \mu \nu. \,$

Random variables whose covariance is zero are called uncorrelated.

If X and Y are independent, then their covariance is zero. This follows because under independence,

$E(X \cdot Y)=E(X) \cdot E(Y)=\mu\nu. \,$

Recalling the final form of the covariance derivation given above, and substituting, we get

$\operatorname{Cov}(X, Y) = \mu \nu - \mu \nu = 0. \,$

The converse, however, is generally not true: Some pairs of random variables have covariance zero although they are not independent. Under some additional assumptions, covariance zero sometimes does entail independence, as for example in the case of multivariate normal distributions.

The units of measurement of the covariance Cov(X, Y) are those of X times those of Y. By contrast, correlation, which depends on the covariance, is a dimensionless measure of linear dependence.

Properties

If X, Y, W, and V are real-valued random variables and a, b, c, d are constant ("constant" in this context means non-random), then the following facts are a consequence of the definition of covariance:

$\operatorname{Cov}(X, a) = 0 \,$

$\operatorname{Cov}(X, X) = \operatorname{Var}(X)\,$

$\operatorname{Cov}(X, Y) = \operatorname{Cov}(Y, X)\,$

$\operatorname{Cov}(aX, bY) = ab\, \operatorname{Cov}(X, Y)\,$

$\operatorname{Cov}(X+a, Y+b) = \operatorname{Cov}(X, Y)\,$

$\operatorname{Cov}(aX+bY, cW+dV) = ac\,\operatorname{Cov}(X,W)+ad\,\operatorname{Cov}(X,V)+bc\,\operatorname{Cov}(Y,W)+bd\,\operatorname{Cov}(Y,V)\,$

For sequences X₁, ..., X_n and Y₁, ..., Y_m of random variables, we have

$\operatorname{Cov}\left(\sum_{i=1}^n {X_i}, \sum_{j=1}^m{Y_j}\right) = \sum_{i=1}^n{\sum_{j=1}^m{\operatorname{Cov}\left(X_i, Y_j\right)}}.\,$

For a sequence X₁, ..., X_n of random variables, and constants a₁, ..., a_n, we have

$\operatorname{Var}\left(\sum_{i=1}^n a_iX_i \right) = \sum_{i=1}^n a_i^2\operatorname{Var}(X_i) + 2\sum_{i,j\,:\,i<j} a_ia_j\operatorname{Cov}(X_i,X_j).$

Incremental computation

Covariance can be computed efficiently from incrementally available values using a generalization of the computational formula for the variance:

$\operatorname{Cov}(X_i, X_j) = \operatorname{E}\left((X_i-\operatorname{E}(X_i))(X_j-\operatorname{E}(X_j))\right) = \operatorname{E}(X_iX_j) -\operatorname{E}(X_i)\operatorname{E}(X_j)$

Relationship to inner products

Many of the properties of covariance can be extracted elegantly by observing that it satisfies similar properties to those of an inner product:

bilinear: for constants a and b and random variables X, Y, and U, Cov(aX + bY, U) = a Cov(X, U) + b Cov(Y, U)
symmetric: Cov(X, Y) = Cov(Y, X)
positive semi-definite: Var(X) = Cov(X, X) ≥ 0, and Cov(X, X) = 0 implies that X is a constant random variable (K).

It can be shown that the covariance is an inner product over some subspace of the vector space of random variables with finite second moment.

Covariance matrix, operator, bilinear form, and function

For column-vector-valued random variables X (with m columns) and Y (with n columns) and their expected values μ=E(X) and ν=E(Y), the covariance matrix

$\operatorname{Cov}(X, Y) := \operatorname{E}((X-\mu)(Y-\nu)^{\top}) = \operatorname{E}(X Y^{\top}) - \mu \nu^{\top}\,$

is the m×n matrix that has in row i and column j the covariance Cov(x_i, y_j) of the i^th scalar component of X and the j^th scalar component of Y. Hence, Cov(X, Y) and Cov(Y, X) are each other's transposes.

More generally, for a probability measure P on a Hilbert space H with inner product $\langle \cdot,\cdot\rangle$ , the covariance of P is the bilinear form Cov: H × H → H given by

$\mathrm{Cov}(x, y) = \int_{H} \langle x, z \rangle \langle y, z \rangle \, \mathrm{d} \mathbf{P} (z)$

for all x and y in H. The covariance operator C is then defined by

$\mathrm{Cov}(x, y) = \langle Cx, y \rangle$

(from the Riesz representation theorem, such operator exists if Cov is bounded). Since Cov is symmetric in its arguments, the covariance operator is self-adjoint (the infinite-dimensional analogy of the transposition symmetry in the finite-dimensional case). When P is a centred Gaussian measure, C is also a nuclear operator. In particular, it is a compact operator of trace class, that is, it has finite trace.

Even more generally, for a probability measure P on a Banach space B, the covariance of P is the bilinear form on the algebraic dual B^#, defined by

$\mathrm{Cov}(x, y) = \int_{B} \langle x, z \rangle \langle y, z \rangle \, \mathrm{d} \mathbf{P} (z)$

where $\langle x, z \rangle$ is now the value of the linear functional x on the element z.

Quite similarly, the covariance function of a function-valued random element (in special cases called random process or random field) z is

$\mathrm{Cov}(x, y) = \int z(x) z(y) \, \mathrm{d} \mathbf{P} (z) = E(z(x) z(y)),$

where z(x) is now the value of the function z at the point x, i.e., the value of the linear functional $u \mapsto u(x)$ evaluated at z.

Comments

The covariance is sometimes called a measure of "linear dependence" between the two random variables. That does not mean the same thing as in the context of linear algebra (see linear dependence). When the covariance is normalized, one obtains the correlation matrix. From it, one can obtain the Pearson coefficient, which gives us the goodness of the fit for the best possible linear function describing the relation between the variables. In this sense covariance is a linear gauge of dependence.

External links

MathWorld page on calculating the sample covariance

References

Statistics

Descriptive statistics

Continuous data

Location	Mean (Arithmetic, Geometric, Harmonic) · Median · Mode

Dispersion	Range · Standard deviation · Coefficient of variation · Percentile

Moments	Variance · Semivariance · Skewness · Kurtosis

Categorical data

Frequency · Contingency table

Inferential statistics
and
hypothesis testing

Inference	Confidence interval (Frequentist inference) · Credible interval (Bayesian inference) · Significance · Meta-analysis

Design of experiments	Population · Sampling · Stratified sampling · Replication · Blocking · Sensitivity and specificity

Sample size estimation	Statistical power · Effect size · Standard error

General estimation	Bayesian estimator · Maximum likelihood · Method of moments · Minimum distance · Maximum spacing

Specific tests	Z-test (normal) · Student's t-test · F-test · Chi-square test · Pearson's chi-square test · Wald test · Mann–Whitney U · Wilcoxon signed-rank test

Survival analysis	Survival function · Kaplan–Meier · Logrank test · Failure rate · Proportional hazards models

Correlation
and
regression analysis

Correlation	Pearson product-moment correlation · Rank correlation (Spearman's rho, Kendall's tau) · Confounding variable

Linear regression	Linear model · General linear model · Generalized linear model · Analysis of variance · Analysis of covariance

Nonlinear regression	Nonparametric · Semiparametric · Logistic

Statistical graphics

Bar chart · Biplot · Box plot · Control chart · Correlogram · Forest plot · Histogram · Q-Q plot · Run chart · Scatter plot · Stemplot

Category · Portal · Outline · Index

This entry is from Wikipedia, the leading user-contributed encyclopedia. It may not have been reviewed by professional editors (see full disclaimer)

Donate to Wikimedia

		Dictionary. The American Heritage® Dictionary of the English Language, Fourth Edition Copyright © 2007, 2000 by Houghton Mifflin Company. Updated in 2009. Published by Houghton Mifflin Company. All rights reserved. Read more
		Statistics Dictionary. A Dictionary of Statistics. Second edition revised. Copyright © Oxford University Press, 2008. All rights reserved. Read more
		Investment Dictionary. Copyright ©2000, Investopedia.com - Owned and Operated by Investopedia Inc. All rights reserved. Read more
		Business Dictionary. Dictionary of Business Terms. Copyright © 2000 by Barron's Educational Series, Inc. All rights reserved. Read more
		Geography Dictionary. A Dictionary of Geography. Copyright © Susan Mayhew 1992, 1997, 2004. All rights reserved. Read more
		Veterinary Dictionary. Saunders Comprehensive Veterinary Dictionary 3rd Edition. Copyright © 2007 by D.C. Blood, V.P. Studdert and C.C. Gay, Elsevier. All rights reserved. Read more
		Wikipedia. This article is licensed under the Creative Commons Attribution/Share-Alike License. It uses material from the Wikipedia article "Covariance". Read more

	covariance matrix
	population correlation coefficient (statistics)
	euclidean quantum field theory (quantum mechanics)

	Find the formula for percentage of covariance? Read answer...
	What is the difference between correlation and covariance? Read answer...
	How do you calculate Beta by using Variance and Covariance? Read answer...

	What is the difference between covariance and correlation?
	What is the differentiate between correlation and covariance?
	When do you apply covariance analysis?

covariance

Contents