Eigenvalue clipping and Marchenko-Pastur

Telling signal eigenvalues from noise

If returns were pure noise with no structure, the eigenvalues of their correlation matrix would still be spread over a known interval, the Marchenko-Pastur interval $[\lambda_-,\lambda_+]$ with $\lambda_\pm=(1\pm\sqrt q)^2$ and $q=N/T$. That interval is a ruler. An eigenvalue of the sample correlation matrix that lies above $\lambda_+$ is larger than noise can explain, so it reflects a real common factor. Eigenvalues inside the interval are indistinguishable from noise: their eigenvectors are random directions and carry no information about the future.

Eigenvalue clipping acts on this verdict. Keep the eigenvalues above $\lambda_+$, and replace all the others by their own average $\bar\lambda_{\mathrm{noise}}$, which is the same as flattening the noise part of the spectrum. Averaging rather than setting them to zero keeps the trace, which for a correlation matrix is $N$, and keeps the matrix positive definite and so invertible, which an optimiser needs. Then rebuild $C_{\mathrm{clean}}=Q\Lambda'Q^{T}$ and rescale it so that its diagonal is exactly 1, because the replacement slightly changes the diagonal. Two refinements are common. The market mode soaks up variance, so the noise level is really $\sigma^2\approx1-\lambda_1/N$ rather than 1. And the Marchenko-Pastur law assumes independent observations and thin tails, so the edge is a guide, not a precise test; check the cleaned matrix out of sample.

• Edge: eigenvalues above $(1+\sqrt q)^2$ are signal; those inside the interval are noise.
• Clipping: replace the noise eigenvalues by their average, keeping the trace and positive definiteness.
• Finish: rebuild $Q\Lambda'Q^{T}$ and rescale the diagonal to 1; always validate out of sample.

Lab · what's graded

• The upper edge15 pts
• Counting signal eigenvalues10 pts
• The replacement value15 pts
• The trace stays10 pts
• The clipping algorithm15 pts
• Why the average15 pts
• What an eigenvalue above the edge means20 pts