Forward-backward smoothing

Looking back with hindsight

The forward algorithm gives filtered probabilities: our belief about day $t$ using only the data up to day $t$. After we have the whole history we can do better for the past. If day one looked calm but days two and three had large moves, those later days tell us something about day one. Smoothing uses all the data. It needs a second quantity $\beta_t(i)$: the probability of the observations after day $t$, given that day $t$ is in state $i$. We compute it backward, starting from $\beta_T=1$.

The product $\alpha_t(i)\beta_t(i)$ is the probability of all the data together with being in state $i$ on day $t$: $\alpha$ covers the past and today, $\beta$ covers the future. Dividing by the sum over states gives the smoothed probability $\gamma_t(i)$, the belief about day $t$ given the whole sample. Summing $\alpha_t\beta_t$ over the states gives the same number for every $t$, the probability of the data. Smoothed probabilities are the right tool to describe the past, for example to label historical regimes. They are the wrong tool to trade on, because on the day itself the future is not known.

• Backward values $\beta_t(i)$: the chance of all later observations given the state on day $t$.
• Smoothing $\gamma_t\propto\alpha_t\beta_t$: the belief about day $t$ given all the data, past and future.
• For describing the past only: smoothed probabilities use future data, so they cannot be used to trade.

Lab · what's graded

• A backward value15 pts
• Another backward value15 pts
• The smoothed chance of stormy15 pts
• A check on the sum10 pts
• Filtered versus smoothed10 pts
• What smoothing adds10 pts
• Match the vocabulary10 pts
• Why smoothed and filtered differ15 pts