Markov chains refresher

Calm days and stormy days

A Markov chain is a system that hops between states, where the next state depends only on the current one, not on the whole past. Think of a market that is either Calm (C) or Stormy (S) each day. The rules fit in a transition matrix $A$: the entry $A_{ij}$ is the probability of going from state $i$ today to state $j$ tomorrow. Each row adds up to $1$. Say a calm day is followed by a calm day with probability $0.9$ and by a stormy day with probability $0.1$, and a stormy day is followed by a calm day with probability $0.2$ and by another stormy day with probability $0.8$.

To look two days ahead, multiply the matrix by itself: $A^{2}$ holds the two-step probabilities. Run the chain for a long time and the share of days in each state settles at the stationary distribution $\pi$, which satisfies $\pi A=\pi$: one more step does not change it. A state with self-probability $A_{ii}$ is kept for $1/(1-A_{ii})$ days on average, so persistent states produce long streaks. These are the same ideas that later let us describe market regimes: calm periods and stormy periods that last for a while.

• Markov property: tomorrow depends only on today's state, not on how we got here.
• Stationary distribution $\pi$: the long-run share of days in each state, with $\pi A=\pi$.
• Duration: a state with self-probability $p$ lasts $1/(1-p)$ days on average.

Lab · what's graded

• Two days ahead15 pts
• The long-run share of calm days15 pts
• A calm stretch10 pts
• A stormy stretch10 pts
• A short path10 pts
• The Markov property10 pts
• Match the vocabulary15 pts
• Why stormy days come in streaks15 pts