Conjugate models: beta-binomial and normal

Updates that have a formula

Usually a posterior needs a computer. For a few pairs of prior and likelihood, though, the answer is a neat formula, because the posterior stays in the same family as the prior. Such a pair is called conjugate. The best known is for a win rate $\theta$: a Beta prior with win-or-lose data. A $\mathrm{Beta}(a,b)$ prior acts as if you had already seen $a$ wins and $b$ losses. Each new win adds $1$ to $a$, each new loss adds $1$ to $b$. The mean is $a/(a+b)$.

The second pair is for an unknown average, such as a mean return, when the noise is known. The prior is $N(\mu_0,\tau_0^{2})$ and the data give a sample mean $\bar x$ with standard error $s$. Think of precision, $1/\mathrm{variance}$, as how much you trust something. The posterior precision is the sum of the two precisions. The posterior mean is the precision-weighted average of the prior mean and the sample mean, so the more precise source gets more weight.

• Conjugate: the posterior is in the same family as the prior, so the update is a formula.
• Beta prior: counts of wins and losses already seen. Add the new counts to get the posterior.
• Normal case: add the precisions and average the means with precision weights.

Lab · what's graded

• The posterior win rate15 pts
• The spread of the win rate15 pts
• The posterior mean return15 pts
• The posterior spread15 pts
• A flat Beta prior10 pts
• Match the vocabulary15 pts
• Between the prior and the data15 pts