Hierarchical models and shrinkage

Letting many estimates help each other

Suppose you have the average return of $50$ stocks. The best-looking ones are partly good and partly lucky. A hierarchical model says all the true means $\theta_i$ come from one common population, $\theta_i\sim N(\mu,\tau^{2})$, and each observed average $y_i$ is the true mean plus noise with standard error $s_i$. The stock's own data are then combined with the group: the estimate is pulled toward the group mean $\mu$. This pull is called shrinkage.

The shrinkage factor $B_i$ is the share of weight given to the group. A noisy stock (large $s_i$) has $B_i$ near $1$ and is pulled strongly to the group mean. A precisely measured stock (small $s_i$) keeps nearly its own value. If you do not know $\tau^{2}$, estimate it from the spread: the variance of the $y_i$ across stocks equals $\tau^{2}$ plus the average $s_i^{2}$. So $\tau^{2}$ is that variance minus the average noise, never below $0$. If the spread is no larger than noise alone would produce, $\tau^{2}=0$ and all stocks share one estimate.

• Hierarchical model: the true means come from one common population, so every stock's data help all the others.
• Shrinkage $B_i=s_i^{2}/(s_i^{2}+\tau^{2})$: noisy estimates move a lot, precise ones barely.
• Estimating $\tau^{2}$: variance across stocks minus the average noise, never below $0$.

Lab · what's graded

• The shrinkage factor15 pts
• The shrunk mean of A15 pts
• The shrunk mean of B10 pts
• The posterior spread of A10 pts
• Estimating tau squared15 pts
• Who is shrunk most10 pts
• Match the vocabulary10 pts
• Why shrinking helps15 pts