ARIMA models

A series that remembers itself

Many series carry information about their own future. Interest-rate spreads, volatility, and the gap between two related prices drift back toward a level rather than wander freely. ARIMA models capture it with three ideas: I, difference the series to make it stationary; AR, regress it on its own lags; MA, use the recent forecast errors. Stock returns themselves show almost no such memory, so ARIMA belongs where a series is persistent or mean-reverting.

import numpy as np

prices = np.array([100.0, 101.5, 103.2, 104.1, 106.0, 107.2])

print(np.diff(prices))              # first differences: the changes             [1.5 1.7 0.9 1.9 1.2]
print(np.diff(prices, n=2))         # second differences: the change of the changes
print(np.diff(np.log(prices)))      # the differenced log price is the log return

Differencing and autocorrelation

Prices wander and have no stable mean; their changes often do. Taking the first difference (order d = 1) removes a trend, a second one (d = 2) a curving trend. The autocorrelation function (ACF) then shows memory: its value at lag k is the correlation of the series with itself k steps earlier, sum((x[t] - m) * (x[t+k] - m)) / sum((x[t] - m)**2). A slowly decaying ACF suggests an AR model; an ACF near zero at all lags means there is nothing to forecast.

# The ACF of a persistent series decays slowly; the ACF of noise is near zero
import numpy as np

def acf(series, lags):
    dev = series - series.mean()
    total = np.sum(dev ** 2)
    return np.array([np.sum(dev[:len(series) - k] * dev[k:]) / total for k in range(1, lags + 1)])

rng = np.random.default_rng(1)
noise = rng.normal(size=500)
persistent = np.empty(500)
persistent[0] = 0.0
for t in range(1, 500):
    persistent[t] = 0.8 * persistent[t - 1] + noise[t]           # an AR(1) with phi = 0.8
print(np.round(acf(persistent, 4), 3))                           # decays slowly: 0.74, 0.58, 0.48, 0.39 in this sample
print(np.round(acf(noise, 4), 3))                                # all near 0

# Differencing a trending series makes its mean stable
import numpy as np

rng = np.random.default_rng(6)
trend = 100 + 0.5 * np.arange(200) + rng.normal(0, 1, size=200).cumsum()          # a trending random walk
changes = np.diff(trend)
print(round(float(trend[:100].mean()), 1), round(float(trend[100:].mean()), 1))     # the level drifts: the halves differ a lot
print(round(float(changes[:100].mean()), 2), round(float(changes[100:].mean()), 2))   # the changes have a similar mean in both halves

Fitting an AR(1) and forecasting

An AR(1) says x[t] = c + phi * x[t-1] + noise. It is a straight-line regression of each value on the one before it, so phi is the slope and c the intercept, from the same sums as the beta of the CAPM topic. The forecast is recursive: the first step is c + phi * last, and each later one uses the previous forecast. For |phi| < 1 the forecasts converge to the long-run mean c / (1 - phi), the speed set by phi: a mean-reverting series.

import numpy as np

def fit_ar1(series):
    a, b = series[:-1], series[1:]
    phi = np.sum((a - a.mean()) * (b - b.mean())) / np.sum((a - a.mean()) ** 2)
    return float(b.mean() - phi * a.mean()), float(phi)

def forecast_ar1(last, c, phi, steps):
    out, value = [], last
    for _ in range(steps):
        value = c + phi * value
        out.append(value)
    return np.array(out)

rng = np.random.default_rng(3)
x = np.empty(400)
x[0] = 2.0
for t in range(1, 400):
    x[t] = 0.5 + 0.75 * x[t - 1] + rng.normal(0, 0.3)             # true c = 0.5, phi = 0.75, mean 2.0
c, phi = fit_ar1(x)
print(round(c, 3), round(phi, 3))
print(np.round(forecast_ar1(4.0, c, phi, 5), 3), round(c / (1 - phi), 3))

• ARIMA = difference (I) + autoregression (AR) + past errors (MA); differencing turns a trending series into a stationary one.
• The ACF at lag k is the correlation of the series with itself k steps earlier; slow decay suggests an AR model.
• An AR(1) fit is a regression of x[1:] on x[:-1]; forecasts feed on each other and converge to c / (1 - phi) when |phi| < 1; always beat the no-change forecast.

Next up: GARCH volatility models, forecasting not the level of returns but how violently they move.

Lab · what's graded

• Remove the trend with difference25 pts
• Measure the memory with acf25 pts
• Fit the model with fit_ar125 pts
• Look ahead with forecast_ar125 pts