Rolling exponential and online estimators

How much memory should an estimate have?

When the market keeps changing, an estimate needs a memory that fades. A rolling window gives equal weight to the last $W$ observations and none to older ones, so the estimate jumps when a big day enters and again when it leaves $W$ days later. An exponentially weighted estimate gives the observation $k$ days ago the weight $(1-\lambda)\lambda^{k}$, so the past fades smoothly. Its variance estimate needs only yesterday's value: $s_t^{2}=\lambda s_{t-1}^{2}+(1-\lambda)r_t^{2}$. A fully online estimate updates with the new observation only and forgets it afterwards, for example a running mean $m_n=m_{n-1}+(x_n-m_{n-1})/n$.

The same idea runs through all three: new estimate = old estimate + gain $\times$ surprise. A constant gain $g=1-\lambda$ gives the exponential estimator. A gain of $1/n$ gives the plain average of all data. The Kalman filter chooses the gain by itself from the noise levels, so it is the principled version. Memory is a trade-off: a short memory follows changes quickly but is noisy, a long memory is smooth but slow. The half-life $\ln(1/2)/\ln\lambda$ tells how many days it takes for an observation to lose half its weight, and the effective window $(1+\lambda)/(1-\lambda)$ is the equal-weight window with similar noise. The common daily choice $\lambda=0.94$ has a half-life of $11.2$ days and an effective window of $32.3$ days.

• Same recipe: new = old + gain $\times$ surprise; the gain sets the memory.
• Rolling is abrupt, exponential is smooth: a rolling window drops old data all at once, and an exponential one fades it.
• Memory is a trade-off: short memory is fast and noisy, long memory is slow and smooth.

Lab · what's graded

• An exponentially weighted update15 pts
• The half-life10 pts
• The effective window10 pts
• An old observation10 pts
• An online mean10 pts
• Which one forgets abruptly10 pts
• An online update10 pts
• Match the vocabulary10 pts
• Why exponential reacts faster15 pts