The spectral theorem

Why symmetric matrices are the nice ones

A general matrix can have complex eigenvalues and eigenvectors that are not perpendicular. A real symmetric matrix, $A=A^{T}$, cannot. The spectral theorem says that such an $A$ has only real eigenvalues, that eigenvectors for different eigenvalues are orthogonal, and that you can pick an orthonormal basis of eigenvectors. Collecting them as the columns of $Q$ gives $A=Q\Lambda Q^{T}$ with $Q^{T}Q=I$ and $\Lambda$ diagonal. Every covariance matrix is symmetric, so this theorem is the reason PCA works.

The sum form writes $A$ as a stack of simple rank-one pieces, each stretching one orthogonal direction. It makes powers trivial: $A^{k}=Q\Lambda^{k}Q^{T}$, and any function of $A$ just acts on the eigenvalues. A symmetric matrix is positive semidefinite, $\mathbf w^{T}A\mathbf w\ge0$ for all $\mathbf w$, exactly when all its eigenvalues are non-negative, and positive definite when they are all strictly positive. For a covariance matrix $\Sigma$ the quadratic form $\mathbf w^{T}\Sigma\mathbf w$ is the variance of the portfolio with weights $\mathbf w$, so it can never be negative, and it is at least $\lambda_{\min}$ times the squared length of $\mathbf w$.

• Spectral theorem: a real symmetric $A$ has real eigenvalues and orthonormal eigenvectors, $A=Q\Lambda Q^{T}$.
• Definiteness: positive semidefinite means all eigenvalues $\ge0$, positive definite all $>0$.
• Powers and functions: $A^{k}=Q\Lambda^{k}Q^{T}$; apply any function to the eigenvalues only.

Lab · what's graded

• Facts about symmetric matrices15 pts
• The larger eigenvalue15 pts
• A unit eigenvector10 pts
• A power through the eigendecomposition20 pts
• A valid covariance matrix15 pts
• A floor on portfolio variance15 pts
• Why PCA works10 pts