Covariance and correlation matrices

From one asset to many

Everything so far looked at one return series. A portfolio holds many assets, and its risk depends not only on how risky each one is but on how they move *together*. The covariance of two assets is the average product of their deviations from the mean: positive if they tend to rise and fall together, negative if one rises as the other falls. For n assets there are n * n covariances, laid out as a covariance matrix, which is symmetric and has the variances on its diagonal.

import numpy as np

# Five days of returns for two assets: shape (days, assets)
returns = np.array([[0.01, 0.02], [-0.02, -0.01], [0.03, 0.01], [0.00, -0.02], [0.01, 0.03]])

centered = returns - returns.mean(axis=0)            # subtract each column's mean
cov = centered.T @ centered / (len(returns) - 1)     # (assets x days) @ (days x assets) -> (assets x assets)
print(np.round(cov, 6))
print(np.allclose(cov, np.cov(returns, rowvar=False)))     # True

The sample covariance matrix

With a table of returns of shape (days, assets), the whole matrix is one line of linear algebra. Subtract each column's mean (axis=0, the mean over days), and multiply the centred table's transpose by itself: entry (i, j) of centered.T @ centered is the sum over days of x_i * x_j. Divide by days - 1, as for a sample variance. This is what np.cov(returns, rowvar=False) and DataFrame.cov() compute; note the rowvar=False, since np.cov expects the assets in the rows.

Correlation: covariance without the units

A covariance of 0.0004 says little: it depends on how volatile the two assets are. Dividing by the product of the two standard deviations removes the scale and gives the correlation, always between -1 and 1: corr[i, j] = cov[i, j] / (std[i] * std[j]), with std the square root of the diagonal. In code, np.outer(std, std) builds the table of products and one division rescales the whole matrix, with ones on the diagonal. Correlation tells how tightly two assets move; covariance, which keeps the scale, is what the risk formulas need.

# From covariance to correlation and annualised volatilities
import numpy as np

returns = np.array([[0.01, 0.02], [-0.02, -0.01], [0.03, 0.01], [0.00, -0.02], [0.01, 0.03]])
cov = np.cov(returns, rowvar=False)

std = np.sqrt(np.diag(cov))
corr = cov / np.outer(std, std)
print(np.round(corr, 4))
print(np.round(np.sqrt(np.diag(cov) * 252), 4))      # daily variances scaled to a year: volatility grows with sqrt(252)

Can this matrix be real?

A random table of numbers is not a covariance matrix. It must be symmetric, and positive semi-definite: for any weights w, the number w' C w is the variance of a portfolio, so it cannot be negative. Equivalently, no eigenvalue is below zero. Estimated matrices pass, but hand-edited ones, or a patchwork of correlations from different sources (for example A~B = 0.9, B~C = 0.9, A~C = -0.9), fail. Check with np.linalg.eigvalsh, which is for symmetric matrices and returns real eigenvalues, allowing a tiny tolerance for rounding.

# A validity check: symmetry and no negative eigenvalue
import numpy as np

def is_valid_cov(matrix, tol=1e-10):
    symmetric = bool(np.all(np.abs(matrix - matrix.T) <= tol))
    return symmetric and bool(np.linalg.eigvalsh(matrix).min() >= -tol)

good = np.array([[1.0, 0.5, 0.2], [0.5, 1.0, 0.3], [0.2, 0.3, 1.0]])
bad = np.array([[1.0, 0.9, -0.9], [0.9, 1.0, 0.9], [-0.9, 0.9, 1.0]])
print(is_valid_cov(good), np.round(np.linalg.eigvalsh(good), 3))
print(is_valid_cov(bad), np.round(np.linalg.eigvalsh(bad), 3))

# Few days, many assets: the sample covariance matrix is singular
import numpy as np

rng = np.random.default_rng(0)
returns = rng.normal(0, 0.01, size=(5, 8))              # 5 days, 8 assets
cov = np.cov(returns, rowvar=False)
print(np.linalg.matrix_rank(cov), "of", cov.shape[0])  # 4 of 8: some portfolios seem to have zero variance

• A covariance matrix of a (days, assets) table is centered.T @ centered / (days - 1): symmetric, variances on the diagonal.
• Correlation is cov / np.outer(std, std): unit-free, between -1 and 1, ones on the diagonal; annualise variances by periods, volatilities by its square root.
• A valid covariance matrix is symmetric with no negative eigenvalue (np.linalg.eigvalsh); estimates from too few days, or patched by hand, can break this.

Next up: portfolio return and variance, turning weights and this matrix into a portfolio's risk.

Lab · what's graded

• Build the covariance table with cov_matrix25 pts
• Rescale it to correlations with corr_from_cov25 pts
• Read each asset's volatility with annualised_vols25 pts
• Test whether it could be real with is_valid_cov25 pts