One market, a few themes, and a long tail
Standardise each column of returns (mean 0, standard deviation 1) and call the result Z, with n days and p assets. The correlation matrix is C=ZᵀZ/n. A principal component is an eigenvector vⱼ of C: the direction in asset space along which the standardised returns vary most, then most among the directions at right angles to the first, and so on. Its eigenvalue λⱼ is the variance along that direction, the eigenvalues add up to p (the total variance of p standardised columns), and λⱼ/p is the explained variance ratio. The score Z vⱼ is the time series of that component. numpy.linalg.eigh returns eigenvalues in increasing order, so sort them down; the sign of an eigenvector is arbitrary, so fix a convention (for instance the largest component positive) before you compare with anything.
Keeping only the first k components rebuilds the data as Ẑ=Z VₖVₖᵀ. The mean squared error of the rebuild is the discarded variance divided by p: 1/pΣ_(j>k)λⱼ. On the last 2,500 trading days of 61 stocks and ETFs, the first component carries 37.4% of the variance and the next four 7.3%, 6.2%, 3.3% and 2.8%. The first score has a correlation of 0.998 with the equal-weight average return: it is the market, found without being told. The next components are sector and style themes. Yet the tail is heavy: 3 components give half of the variance, but 22 are needed for 80% and 34 for 90%, and the reconstruction error is 0.626 with one component, 0.430 with five and 0.213 with twenty. PCA compresses the common part of the market; it does not make 61 assets look like three.
- Standardise, then eigen-decompose the correlation matrix; the eigenvalues add up to the number of assets.
- Signs are arbitrary: fix a convention, and compare vectors by absolute cosine.
- The first component of stock returns is the market; the tail of small components still matters.
Lab · what's graded
- Standardising10 pts
- The eigen-decomposition20 pts
- Explained variance10 pts
- Checking against scikit-learn15 pts
- How many components10 pts
- Reconstruction error15 pts
- The study20 pts