The sample covariance matrix S=T−11XcTXc is unbiased: on average over many samples it equals the true Σ. That sounds reassuring and is misleading. A portfolio of N assets has N(N+1)/2 covariance parameters, which for N=100 is 5,050, and you estimate them from T observations of N returns. When N is not small compared with T, every entry is noisy and, worse, the eigenvalues of S are systematically spread out: the large ones too large, the small ones too small, even if the truth has all eigenvalues equal.
q=TN,λ±=σ2(1±q)2,VpromVreal≈(1−q)21(1)
If the true covariance is the identity (σ2=1, all eigenvalues 1) and N,T are large, the eigenvalues of S fill the interval [λ−,λ+] of the first formula, the Marchenko-Pastur edges. With q=0.25 that is [0.25,2.25], a spread of nine to one where the truth has none. When N>T−1, S has rank at most T−1, so it is singular and has zero eigenvalues. A minimum-variance optimiser is the worst victim: it loads up on the directions in which S shows the least variance, which are exactly the directions where S underestimates risk most. Simulations with Σ=I show that the realised variance of the sample minimum-variance portfolio is about 1/(1−q)2 times the variance it promised in-sample, and 1/(1−q) times the true minimum.
Five hundred stocks, two yearsEstimate the covariance of N=500 stocks from two years of daily data, T=504. Then q=500/504≈0.99, and the Marchenko-Pastur edges for a unit truth are λ−=(1−0.996)2≈0.00002 and λ+=(1+0.996)2≈3.98: the smallest eigenvalues are essentially zero, so S is almost singular and its inverse is dominated by noise. The matrix has 500×501/2=125,250 distinct entries but only 504×500 numbers behind them. By contrast, the twelve stocks of the capstone with 4,184 days have q=0.003, so the edges are about 0.89 and 1.11 and noise is a small issue there.
Unbiased but noisy: E[S]=Σ, yet for large q=N/T the eigenvalues of S are spread out.
Edges: for a unit truth the sample eigenvalues lie in [(1−q)2,(1+q)2].
Rank: for N>T−1 the sample covariance is singular; an optimiser exploits its smallest eigenvalues.
Optimisers are error maximisersA portfolio optimiser reads the covariance matrix as truth and concentrates where the matrix says risk is cheapest. Estimation errors are therefore not averaged out; they are selected for. The cures in the next lessons all shrink or clean the eigenvalues, because that is where the damage sits, and they are judged the only honest way: by the variance of the portfolio on data the estimator never saw.
Lab · what's graded
The upper edge15 pts
The lower edge10 pts
Counting parameters10 pts
The maximum rank10 pts
Realised versus promised variance15 pts
Why optimisers amplify the noise20 pts
Reducing the noise20 pts
0/ 7 answered
1Numeric answer15 ptsNot answered
With a true covariance equal to the identity and q = N/T = 0.25, what is the upper Marchenko-Pastur edge (1+q)2?
lambda+=accepted: ± 0.01
2Numeric answer10 ptsNot answered
For the same q = 0.25, what is the lower edge (1−q)2?
lambda−=accepted: ± 0.005
3Numeric answer10 ptsNot answered
How many distinct entries does the covariance matrix of N = 100 assets have, N(N+1)/2?
entries=accepted: ± 0.5
4Numeric answer10 ptsNot answered
A sample covariance matrix of N = 50 assets is built from T = 30 daily observations. What is the largest possible rank, T−1?
rank=accepted: ± 0.01
5Numeric answer15 ptsNot answered
For a unit true covariance and q = 0.2, the realised variance of the sample minimum-variance portfolio is about 1/(1−q)2 times the variance promised in-sample. What is that factor? Give it to two decimals.
factor=accepted: ± 0.01
6Choice20 ptsNot answered
Why does a minimum-variance optimiser do so badly with a noisy sample covariance matrix?
7Explain in words20 ptsNot answered
In one or two sentences, say two ways to make a sample covariance matrix less noisy.
Math modeKeyboard or LaTeX inputTask 1 of 7 · Ctrl ⏎ to check
