Concept of Karhunen-Loève expansion :
Karhunen-Loève theorem : In the theory of stochastic processes, the Karhunen-Loève theorem (named after Kari Karhunen and Michel Loève) is a representation of a stochastic process as an infinite linear combination of orthogonal functions, analogous to a Fourier series representation of a function on a bounded interval. In contrast to a Fourier series where the coefficients are real numbers and the expansion basis consists of sinusoidal functions (that is, sine and cosine functions), the coefficients in the Karhunen-Loève theorem are random variables and the expansion basis depends on the process. In fact, the orthogonal basis functions used in this representation are determined by the covariance function of the process. If we regard a stochastic process as a random function F, that is, one in which the random value is a function on an interval [a, b], then this theorem can be considered as a random orthonormal expansion of F.
In the case of a centered stochastic process {Xt}t [a, b] (where centered means that the expectations E(Xt) are defined and equal to 0 for all t) satisfying a technical continuity condition, admits a decomposition
Xt =∞∑k=1Zkek(t).
where Zk are pairwise uncorrelated random variables and the functions ek are continuous real-valued functions on [a, b] which are pairwise orthogonal in L2[a, b]. The general case of a process which is not centered can be represented by expanding the expectation function (which is a non-random function) in the basis ek .
Moreover, if the process is Gaussian, then the random variables Zk are Gaussian and stochastically independent. This result generalizes the Karhunen-Loève transform. An important example of a centered real stochastic process on [0,1] is the Wiener process and the Karhunen-Loève theorem can be used to provide a canonical orthogonal representation for it. In this case the expansion consists of sinusoidal functions.
The above expansion into uncorrelated random variables is also known as the Karhunen-Loève expansion or Karhunen-Loève decomposition. The empirical version (i.e., with the coefficients computed from a sample) is known as Proper orthogonal decomposition or Principal component analysis.