AR MA and ARMA models

Building blocks for the mean

An autoregression of order $p$, AR($p$), explains a value by its own past: $x_t=c+\varphi_1x_{t-1}+\dots+\varphi_px_{t-p}+\varepsilon_t$. It is stationary when its coefficients keep it from exploding, which for AR(1) means $|\varphi|<1$, and its mean is $\mu=c/(1-\sum\varphi_i)$. A moving average of order $q$, MA($q$), explains a value by past shocks: $x_t=\mu+\varepsilon_t+\theta_1\varepsilon_{t-1}+\dots+\theta_q\varepsilon_{t-q}$. It is always stationary and has a memory of exactly $q$ steps. Combining both gives ARMA($p,q$), which often describes a series with fewer parameters than a pure AR or MA would need.

The parameters are estimated by maximum likelihood. The orders are chosen by comparing information criteria, the AIC $=2k-2\ln L$ and the BIC $=k\ln T-2\ln L$, where $k$ is the number of parameters and $L$ the likelihood: lower is better, and the penalty on $k$ stops you from adding terms that only fit noise. The BIC penalises more heavily when $T>7$ and so picks smaller models. After the fit, residuals should look like white noise, which the Ljung-Box test checks. The forecast of a stationary AR(1) returns to the mean geometrically, $\hat x_{T+h}=\mu+\varphi^{h}(x_T-\mu)$, and the forecast of an MA($q$) equals the mean from $q+1$ steps ahead. Far forecasts of any stationary ARMA are simply the mean, which is why such models are good for the next few steps and no better than the average after that.

• AR($p$) depends on its own past values; MA($q$) on the last $q$ shocks; ARMA on both.
• AIC and BIC reward fit and punish parameters; the model with the lowest value wins, the BIC being stricter.
• Forecasts of a stationary ARMA converge to the mean as the horizon grows.

Lab · what's graded

• The mean of an AR(1)15 pts
• A two-step forecast15 pts
• An MA(1) autocorrelation15 pts
• An information criterion15 pts
• A non-stationary coefficient15 pts
• Match the vocabulary15 pts
• Why far forecasts reach the mean10 pts