Markov chains

A process with a one-step memory

A Markov chain moves between a finite set of states, and the next state depends only on the current one: $P(X_{t+1}=j\mid X_t=i,\ \text{earlier history})=P_{ij}$ . The whole model is the transition matrix $P$ , whose rows are probability distributions (each row sums to 1). Markets are often described as switching between regimes such as calm and turbulent, or bull and bear, and a Markov chain is the simplest model of that switching.

P^{(n)}=P^{n},\qquad \boldsymbol\pi P=\boldsymbol\pi,\qquad \sum_i\pi_i=1

(1)

The $n$ -step transition probabilities are the entries of the matrix power $P^{n}$ , which is matrix multiplication of the previous linear algebra topics at work. A stationary distribution $\boldsymbol\pi$ satisfies $\boldsymbol\pi P=\boldsymbol\pi$ : if the chain starts with it, it keeps it forever. For a chain in which every state can reach every other (irreducible) and that does not cycle with a fixed period (aperiodic), the stationary distribution is unique and the chain converges to it from any start; $\pi_i$ is the long-run fraction of time spent in state $i$ . The time spent in a state before leaving is geometric with mean $1/(1-P_{ii})$ .

A bull and bear marketLet the daily state be bull (B) or bear (R) with

P=\begin{pmatrix}0.9&0.1\\0.2&0.8\end{pmatrix}

. Two days ahead,

P^2=\begin{pmatrix}0.83&0.17\\0.34&0.66\end{pmatrix}

: starting in a bull market, the chance of a bear market two days later is 0.17. The stationary distribution solves

0.1\pi_B=0.2\pi_R

, so

\boldsymbol\pi=(2/3,\ 1/3)

: in the long run the market spends two thirds of the time in the bull state. A bull regime lasts on average

1/(1-0.9)=10

days.

Markov property: the next state depends only on the current state.
$n$ steps: use $P^{n}$ . Stationary: solve $\boldsymbol\pi P=\boldsymbol\pi$ with the entries adding to 1.
Duration: a state with self-transition $p$ lasts $1/(1-p)$ steps on average.

Regimes are modelled, not observedA real market never announces its regime. A fitted Markov chain gives you probabilities of switching, but you must infer the current regime from returns. When the state is hidden, the model becomes a hidden Markov model, treated in a later skill. Estimated transition probabilities are themselves noisy, so durations such as 10 days carry wide uncertainty.

Lab · what's graded

Two-day transitions20 pts
The stationary distribution15 pts
How long a regime lasts15 pts
From bull to bear in two days10 pts
The Markov property15 pts
When it converges10 pts
What the stationary distribution means15 pts

A process with a one-step memory

P^{(n)}=P^{n},\qquad \boldsymbol\pi P=\boldsymbol\pi,\qquad \sum_i\pi_i=1

(1)

A bull and bear marketLet the daily state be bull (B) or bear (R) with

P=\begin{pmatrix}0.9&0.1\\0.2&0.8\end{pmatrix}

. Two days ahead,

P^2=\begin{pmatrix}0.83&0.17\\0.34&0.66\end{pmatrix}

: starting in a bull market, the chance of a bear market two days later is 0.17. The stationary distribution solves

0.1\pi_B=0.2\pi_R

, so

\boldsymbol\pi=(2/3,\ 1/3)

: in the long run the market spends two thirds of the time in the bull state. A bull regime lasts on average

1/(1-0.9)=10

days.

Markov property: the next state depends only on the current state.
$n$ steps: use $P^{n}$ . Stationary: solve $\boldsymbol\pi P=\boldsymbol\pi$ with the entries adding to 1.
Duration: a state with self-transition $p$ lasts $1/(1-p)$ steps on average.

Lab · what's graded

Two-day transitions20 pts
The stationary distribution15 pts
How long a regime lasts15 pts
From bull to bear in two days10 pts
The Markov property15 pts
When it converges10 pts
What the stationary distribution means15 pts

A process with a one-step memory

Lab · what's graded

Log in to run your code.

Markov chains

A process with a one-step memory

Lab · what's graded

Log in to run your code.