Autoregressive model: Information from Answers.com

In statistics and signal processing, an autoregressive (AR) model is a type of random process which is often used to model and predict various types of natural and social phenomena.

1 Definition
2 Example: An AR(1)-process
3 Calculation of the AR parameters
- 3.1 Derivation
4 Implementations in statistics packages
5 See also
6 References

Definition

The notation AR(p) refers to the autoregressive model of order p. The AR(p) model is defined as

$X_t = c + \sum_{i=1}^p \varphi_i X_{t-i}+ \varepsilon_t \,$

where $\varphi_1, \ldots, \varphi_p$ are the parameters of the model, $c$ is a constant and $\varepsilon_t$ is white noise. The constant term is omitted by many authors for simplicity.

An autoregressive model can thus be viewed as the output of an all-pole infinite impulse response filter whose input is white noise.

Some constraints are necessary on the values of the parameters of this model in order that the model remains wide-sense stationary. For example, processes in the AR(1) model with |φ₁| ≥ 1 are not stationary. More generally, for an AR(p) model to be wide-sense stationary, the roots of the polynomial $\textstyle z^p - \sum_{i=1}^p \varphi_i z^{p-i}$ must lie within the unit circle, i.e., each root $z i$ must satisfy $| z i | < 1$ .

Example: An AR(1)-process

An AR(1)-process is given by:

$X_t = c + \varphi X_{t-1}+\varepsilon_t\,$

where $\varepsilon_t$ is a white noise process with zero mean and variance $σ 2$ . (Note: The subscript on $\varphi_1$ has been dropped.) The process is wide sense stationary if $|\varphi|<1$ since it is obtained as the output of a stable filter whose input is white noise. (If $\varphi=1$ then $X t$ has infinite variance, and is therefore not wide sense stationary.) Consequently, assuming $|\varphi|<1$ , the mean $E(X t)$ is identical for all values of t. Denoting the mean by $μ$ , we get

$\mbox{E}(X_t)=\mbox{E}(c)+\varphi\mbox{E}(X_{t-1})+\mbox{E}(\varepsilon_t)\Rightarrow \mu=c+\varphi\mu+0.$

Thus

$\mu=\frac{c}{1-\varphi}.$

In particular, if $c = 0$ , then the mean is 0.

The variance can be shown to equal

$\textrm{var}(X_t)=E(X_t^2)-\mu^2=\frac{\sigma^2}{1-\varphi^2}.$

The autocovariance is given by

$B_n=E(X_{t+n}X_t)-\mu^2=\frac{\sigma^2}{1-\varphi^2}\,\,\varphi^{|n|}.$

It can be seen that the autocovariance function decays with a decay time (also called time constant) of $\tau=-1/\ln(\varphi)$ [to see this, write $B n = K φ | n |$ where $K$ is independent of $n$ . Then note that $φ | n | = e | n | lnφ$ and match this to the exponential decay law $e - n / τ$ ].

The spectral density function is the Fourier transform of the autocovariance function. In discrete terms this will be the discrete-time Fourier transform:

$\Phi(\omega)= \frac{1}{\sqrt{2\pi}}\,\sum_{n=-\infty}^\infty B_n e^{-i\omega n} =\frac{1}{\sqrt{2\pi}}\,\left(\frac{\sigma^2}{1+\varphi^2-2\varphi\cos(\omega)}\right).$

This expression is periodic due to the discrete nature of the $X j$ , which is manifested as the cosine term in the denominator. If we assume that the sampling time ( $Δ t = 1$ ) is much smaller than the decay time ( $τ$ ), then we can use a continuum approximation to $B n$ :

$B(t)\approx \frac{\sigma^2}{1-\varphi^2}\,\,\varphi^{|t|}$

which yields a Lorentzian profile for the spectral density:

$\Phi(\omega)= \frac{1}{\sqrt{2\pi}}\,\frac{\sigma^2}{1-\varphi^2}\,\frac{\gamma}{\pi(\gamma^2+\omega^2)}$

where $γ = 1 / τ$ is the angular frequency associated with the decay time $τ$ .

An alternative expression for $X t$ can be derived by first substituting $c+\varphi X_{t-2}+\varepsilon_{t-1}$ for $X t - 1$ in the defining equation. Continuing this process N times yields

$X_t=c\sum_{k=0}^{N-1}\varphi^k+\varphi^NX_{t-N}+\sum_{k=0}^{N-1}\varphi^k\varepsilon_{t-k}.$

For N approaching infinity, $\varphi^N$ will approach zero and:

$X_t=\frac{c}{1-\varphi}+\sum_{k=0}^\infty\varphi^k\varepsilon_{t-k}.$

It is seen that $X t$ is white noise convolved with the $\varphi^k$ kernel plus the constant mean. If the white noise $\varepsilon_t$ is a Gaussian process then $X t$ is also a Gaussian process. In other cases, the central limit theorem indicates that $X t$ will be approximately normally distributed when $\varphi$ is close to one.

Calculation of the AR parameters

The AR(p) model is given by the equation

$X_t = \sum_{i=1}^p \varphi_i X_{t-i}+ \varepsilon_t.\,$

It is based on parameters $\varphi_i$ where i = 1, ..., p. There is a direct correspondence between these parameters and the covariance function of the process, and this correspondence can be inverted to determine the parameters from the autocorrelation function (which is itself obtained from the covariances). This is done using the Yule-Walker equations:

$\gamma_m = \sum_{k=1}^p \varphi_k \gamma_{m-k} + \sigma_\varepsilon^2\delta_m$

where m = 0, ... , p, yielding p + 1 equations. $γ m$ is the autocorrelation function of X, $\sigma_\varepsilon$ is the standard deviation of the input noise process, and $δ m$ is the Kronecker delta function.

Because the last part of the equation is non-zero only if m = 0, the equation is usually solved by representing it as a matrix for m > 0, thus getting equation

$\begin{bmatrix} \gamma_1 \\ \gamma_2 \\ \gamma_3 \\ \vdots \\ \end{bmatrix} = \begin{bmatrix} \gamma_0 & \gamma_{-1} & \gamma_{-2} & \dots \\ \gamma_1 & \gamma_0 & \gamma_{-1} & \dots \\ \gamma_2 & \gamma_{1} & \gamma_{0} & \dots \\ \vdots & \vdots & \vdots & \ddots \\ \end{bmatrix} \begin{bmatrix} \varphi_{1} \\ \varphi_{2} \\ \varphi_{3} \\ \vdots \\ \end{bmatrix}$

solving all $\varphi$ . For m = 0 have

$\gamma_0 = \sum_{k=1}^p \varphi_k \gamma_{-k} + \sigma_\varepsilon^2$

which allows us to solve $\sigma_\varepsilon^2$ .

The above equations (the Yule-Walker equations) provide one route to estimating the parameters of an AR(p) model, by replacing the theoretical covariances with estimated values. One way of specifying the estimated covariances is equivalent to a calculation using least squares regression of values X_t on the p previous values of the same series.

Derivation

The equation defining the AR process is

$X_t = \sum_{i=1}^p \varphi_i\,X_{t-i}+ \varepsilon_t.\,$

Multiplying both sides by X_t − m and taking expected value yields

$E[X_t X_{t-m}] = E\left[\sum_{i=1}^p \varphi_i\,X_{t-i} X_{t-m}\right]+ E[\varepsilon_t X_{t-m}].$

Now, $E [X t X t - m] = γ m$ by definition of the autocorrelation function. The values of the noise function are independent of each other, and X_t − m is independent of ε_t where m is greater than zero. For m > 0, E[ε_tX_t − m] = 0. For m = 0,

$E[\varepsilon_t X_{t}] = E\left[\varepsilon_t \left(\sum_{i=1}^p \varphi_i\,X_{t-i}+ \varepsilon_t\right)\right] = \sum_{i=1}^p \varphi_i\, E[\varepsilon_t\,X_{t-i}] + E[\varepsilon_t^2] = 0 + \sigma_\varepsilon^2,$

Now we have, for m ≥ 0,

$\gamma_m = E\left[\sum_{i=1}^p \varphi_i\,X_{t-i} X_{t-m}\right] + \sigma_\varepsilon^2 \delta_m.$

Furthermore,

$E\left[\sum_{i=1}^p \varphi_i\,X_{t-i} X_{t-m}\right] = \sum_{i=1}^p \varphi_i\,E[X_{t} X_{t-m+i}] = \sum_{i=1}^p \varphi_i\,\gamma_{m-i},$

which yields the Yule-Walker equations:

$\gamma_m = \sum_{i=1}^p \varphi_i \gamma_{m-i} + \sigma_\varepsilon^2 \delta_m.$

for m ≥ 0. For m < 0,

$\gamma_m = \gamma_{-m} = \sum_{i=1}^p \varphi_i \gamma_{|m|-i} + \sigma_\varepsilon^2 \delta_m.$

Implementations in statistics packages

In R, the stats package includes an ar function. The function is documented in "Fit Autoregressive Models to Time Series".

References

Mills, Terence C. Time Series Techniques for Economists. Cambridge University Press, 1990.
Percival, Donald B. and Andrew T. Walden. Spectral Analysis for Physical Applications. Cambridge University Press, 1993.
Pandit, Sudhakar M. and Wu, Shien-Ming. Time Series and System Analysis with Applications. John Wiley & Sons, Inc., 1983.

This entry is from Wikipedia, the leading user-contributed encyclopedia. It may not have been reviewed by professional editors (see full disclaimer)

Donate to Wikimedia

	What you have to do to be model? Read answer...
	How about modeling? Read answer...
	What is a model? Read answer...

	Who is this model?
	What do you do as a model?
	What is a IT model?

Autoregressive model

Contents