Conditional independence

These pictures represent the probabilities of events R, B and G by the areas shaded red, blue and yellow respectively with respect to the total area. In both examples R and B are conditionally independent given G because but not conditionally independent given not G, because

In probability theory, two events R and B are conditionally independent given a third event G precisely if the occurrence or non-occurrence of R and B are independent events in their conditional probability distribution given G. In the standard notation of probability theory,

or equivalently,

Two random variables X and Y are conditionally independent given an event G if they are independent in their conditional probability distribution given G.

Two events R and B are conditionally independent given a σ-algebra Σ if

where denotes the conditional expectation of the indicator function of the event A, χ_A, given the sigma algebra Σ. That is,

Two random variables X and Y are conditionally independent given a σ-algebra Σ if the above equation holds for all R in σ(X) and B in σ(Y).

Two random variables X and Y are conditionally independent given a random variable W if they are independent given σ(W): the σ-algebra generated by W. This is commonly written:

If W assumes a countable set of values, this is equivalent to the conditional independence of X and Y for the events of the form [W = w].

Conditional independence of more than two events, or of more than two random variables, is defined analogously.

Uses in Bayesian statistics

Let p be the proportion of voters who will vote "yes" in an upcoming referendum. In taking an opinion poll, one chooses n voters randomly from the population. For i = 1, ..., n, let X_i = 1 or 0 according as the ith chosen voter will or will not vote "yes".

In a frequentist approach to statistical inference one would not attribute any probability distribution to p (unless the probabilities could be somehow interpreted as relative frequencies of occurrence of some event or as proportions of some population) and one would say that X₁, ..., X_n are independent random variables.

By contrast, in a Bayesian approach to statistical inference, one would assign a probability distribution to p regardless of the non-existence of any such "frequency" interpretation, and one would construe the probabilities as degrees of belief that p is in any interval to which a probability is assigned. In that model, the random variables X₁, ..., X_n are not independent, but they are conditionally independent given the value of p. In particular, if a large number of the Xs are observed to be equal to 1, that would imply a high conditional probability, given that observation, that p is near 1, and thus a high conditional probability, given that observation, that the next X to be observed will be equal to 1.

Rules of conditional independence

A set of rules governing statements of conditional independence have been derived from the basic definition.^[1][2]

Symmetry:

Decomposition: and

Weak Union:

Contraction: and

If the probabilities of X, Y, Z, W are all positive, then the following also holds:

Intersection: and

References

See also

• de Finetti's theorem
• conditional expectation