Ledoit-Wolf shrinkage

Pulling an estimate toward something simple

Clipping decides eigenvalue by eigenvalue whether it is signal. Shrinkage takes a smoother route: blend the noisy sample matrix $S$ with a simple, well-conditioned target $F$ , $\Sigma^{*}=(1-\delta)S+\delta F$ , where the intensity $\delta\in[0,1]$ says how much to trust the target. The classic target is a scaled identity $F=\mu I$ with $\mu=\operatorname{tr}(S)/N$ , the average variance: it says all assets have the same variance and are uncorrelated. That is surely wrong, but it is stable, and a little of it cancels a lot of noise.

\Sigma^{*}=(1-\delta)S+\delta\mu I,\qquad \mu=\frac{\operatorname{tr}S}{N},\qquad \delta=\min\!\Big(1,\frac{b^{2}}{d^{2}}\Big),\qquad d^{2}=\frac{\lVert S-\mu I\rVert_F^{2}}{N}

(1)

The intensity is not a free parameter. Ledoit and Wolf showed that the best $\delta$ for this target has a closed form: the numerator $b^2$ measures how noisy $S$ is (the average squared distance of the single-day matrices $\mathbf x_t\mathbf x_t^{T}$ from $S$ , divided by $T$ , and capped at $d^2$ ) and the denominator $d^2$ measures how far $S$ is from the target. A noisy $S$ close to its target gets a large $\delta$ . Shrinkage has three tidy properties: it keeps the trace, it moves every eigenvalue toward the mean ( $\lambda\mapsto(1-\delta)\lambda+\delta\mu$ ) so large ones shrink and small ones grow, and it makes $\Sigma^{*}$ positive definite for any $\delta>0$ , even when $S$ is singular. As $T$ grows the noise term falls and $\delta\to0$ .

A 2 by 2 exampleTake

S=\begin{pmatrix}4&1\\1&2\end{pmatrix}

, so

\mu=(4+2)/2=3

and

S-\mu I=\begin{pmatrix}1&1\\1&-1\end{pmatrix}

with squared Frobenius norm 4, hence

d^2=4/2=2

. If the noise term is

b^2=0.5

, the intensity is

\delta=0.5/2=0.25

. With

\delta=0.2

instead,

\Sigma^{*}=0.8S+0.2\cdot3I=\begin{pmatrix}3.8&0.8\\0.8&2.2\end{pmatrix}

. The eigenvalues of

S

are

3\pm\sqrt2=4.414

and

1.586

; those of

\Sigma^{*}

are

0.8(4.414)+0.6=4.131

and

0.8(1.586)+0.6=1.869

. The condition number improves from

2.78

2.21

Blend: $\Sigma^{*}=(1-\delta)S+\delta F$ with the target $F=\mu I$ , $\mu=\operatorname{tr}S/N$ .
Intensity: $\delta=\min(1,b^2/d^2)$ ; noisy matrices close to the target get more shrinkage.
Effects: trace kept, eigenvalues pulled to the mean, positive definite for any $\delta>0$ ; $\delta\to0$ as $T$ grows.

Bias on purposeShrinkage deliberately adds bias, because the target is wrong, in return for a large cut in variance. The expected squared error of the estimate falls, and for covariance estimation with many assets the trade is usually very favourable. It is the same bias-variance bargain as ridge regression, and it is the reason Ledoit-Wolf is the default covariance estimator in many libraries.

Lab · what's graded

The target level10 pts
A shrunk matrix20 pts
The intensity15 pts
A shrunk eigenvalue15 pts
The new condition number15 pts
More data, less shrinkage15 pts
The bias-variance trade10 pts

Pulling an estimate toward something simple

\Sigma^{*}=(1-\delta)S+\delta\mu I,\qquad \mu=\frac{\operatorname{tr}S}{N},\qquad \delta=\min\!\Big(1,\frac{b^{2}}{d^{2}}\Big),\qquad d^{2}=\frac{\lVert S-\mu I\rVert_F^{2}}{N}

(1)

A 2 by 2 exampleTake

S=\begin{pmatrix}4&1\\1&2\end{pmatrix}

, so

\mu=(4+2)/2=3

and

S-\mu I=\begin{pmatrix}1&1\\1&-1\end{pmatrix}

with squared Frobenius norm 4, hence

d^2=4/2=2

. If the noise term is

b^2=0.5

, the intensity is

\delta=0.5/2=0.25

. With

\delta=0.2

instead,

\Sigma^{*}=0.8S+0.2\cdot3I=\begin{pmatrix}3.8&0.8\\0.8&2.2\end{pmatrix}

. The eigenvalues of

S

are

3\pm\sqrt2=4.414

and

1.586

; those of

\Sigma^{*}

are

0.8(4.414)+0.6=4.131

and

0.8(1.586)+0.6=1.869

. The condition number improves from

2.78

2.21

Blend: $\Sigma^{*}=(1-\delta)S+\delta F$ with the target $F=\mu I$ , $\mu=\operatorname{tr}S/N$ .
Intensity: $\delta=\min(1,b^2/d^2)$ ; noisy matrices close to the target get more shrinkage.
Effects: trace kept, eigenvalues pulled to the mean, positive definite for any $\delta>0$ ; $\delta\to0$ as $T$ grows.

Lab · what's graded

The target level10 pts
A shrunk matrix20 pts
The intensity15 pts
A shrunk eigenvalue15 pts
The new condition number15 pts
More data, less shrinkage15 pts
The bias-variance trade10 pts

Pulling an estimate toward something simple

Lab · what's graded

Log in to run your code.

Ledoit-Wolf shrinkage

Pulling an estimate toward something simple

Lab · what's graded

Log in to run your code.