Linear regression (OLS)

Explaining one return with another

A stock rarely moves alone: when the market rises it usually rises too. Linear regression quantifies that link by fitting a straight line y = alpha + beta * x through the pairs of returns (x: the market, y: the stock). Ordinary least squares (OLS) chooses the line that makes the sum of the squared vertical gaps, the residuals, as small as possible. The result has a closed form: the slope beta is the covariance of x and y divided by the variance of x, and the intercept alpha makes the line pass through the point of means.

import numpy as np

market = np.array([0.010, -0.005, 0.004, 0.012, -0.008, 0.003, -0.002, 0.007, 0.001, -0.004])
asset  = np.array([0.013, -0.004, 0.007, 0.016, -0.011, 0.004, -0.001, 0.011, 0.002, -0.007])

dx = market - market.mean()                      # deviations of x from its mean
beta = (dx * (asset - asset.mean())).sum() / (dx ** 2).sum()
alpha = asset.mean() - beta * market.mean()
print(round(beta, 4), round(alpha, 5))           # 1.3271 0.00061   the asset moves 1.33 times as much as the market

Residuals and R squared

A residual is what the line fails to explain for one observation: y - (alpha + beta * x). OLS residuals always add up to zero and are uncorrelated with x, which is a handy check on your own code. R squared says how much of the variation of y the line explains: 1 - sum(residuals²) / sum((y - mean_y)²). It runs from 0 (the line is no better than the average) to 1 (all the points lie on the line), and in a simple regression it equals the squared correlation of x and y.

residuals = asset - (alpha + beta * market)
print(abs(residuals.sum()) < 1e-12, abs((residuals * market).sum()) < 1e-12)   # True True   the OLS fingerprints

ss_res = (residuals ** 2).sum()
ss_tot = ((asset - asset.mean()) ** 2).sum()
print(round(1 - ss_res / ss_tot, 4))                  # 0.9785
print(round(np.corrcoef(market, asset)[0, 1] ** 2, 4))   # 0.9785   the squared correlation

CAPM: beta and alpha

In finance the regression has a name. Beta is the sensitivity to the market: a beta of 1.3 means the stock tends to move 30% more than the market, up or down. Alpha is the average return left over once that market exposure is accounted for, and it is what a manager claims as skill. To keep the risk-free rate out of the picture, the CAPM regression uses excess returns: the return minus the risk-free rate, for both the asset and the market. A strategy with alpha near zero is just market exposure in disguise.

risk_free = 0.0001                                 # risk-free return of one day
x = market - risk_free                             # excess returns of the market
y = asset - risk_free                              # excess returns of the asset

beta = ((x - x.mean()) * (y - y.mean())).sum() / ((x - x.mean()) ** 2).sum()
alpha = y.mean() - beta * x.mean()
print(round(beta, 4), round(alpha, 6))             # 1.3271 0.000644   a daily alpha of 0.064%, on 10 days of data

# Points exactly on y = 1 + 2x: the fit recovers the line
x = np.array([1.0, 2.0, 3.0, 4.0])
y = 1 + 2 * x
dx = x - x.mean()
slope = (dx * (y - y.mean())).sum() / (dx ** 2).sum()
print(y.mean() - slope * x.mean(), slope)      # 1.0 2.0

• OLS minimises the squared residuals: beta = cov(x, y) / var(x) (sums of centred products) and alpha = mean_y - beta * mean_x.
• Residuals y - (alpha + beta * x) add up to zero; R squared 1 - ss_res / ss_tot is the share of variance explained.
• CAPM: regress the asset's excess returns on the market's excess returns; beta is market exposure, alpha what is left.

Next up: volatility, measuring and forecasting how much returns fluctuate.

Lab · what's graded

• Fit the line through the points with ols_fit25 pts
• Measure what the line misses with residuals25 pts
• Score the fit with r_squared25 pts
• Split an asset into market exposure and skill with capm_alpha_beta25 pts