Modelling a yes or no
Some outcomes are not numbers but events: the market closed up or down, a stock was in the top half, a firm defaulted. A straight line cannot model a probability because it can leave the range 0 to 1. Logit and probit squeeze a linear score z=α+βx through an S-shaped curve. The logit uses p=1/(1+e−z), so the score is the log-odds ln1−pp. The probit uses the normal distribution function p=Φ(z). They give nearly the same probabilities, and the probit coefficients are about 1/1.6 of the logit ones. In statsmodels you write smf.logit('up ~ lag', data).fit(), and the model is estimated by maximum likelihood, not by least squares: it picks the coefficients that make the observed ups and downs most probable.
p=1+e−z1,ln1−pp=α+βx,∂x∂p=p(1−p)β,ℓ=i∑[yilnpi+(1−yi)ln(1−pi)],RMcF2=1−ℓ0ℓ(1) Reading a logit by handA score
z=0.2 gives
p=1/(1+e−0.2)=1/1.8187=0.55. A slope
β=−0.054 means the odds are multiplied by
e−0.054=0.947 for each extra unit of
x. The effect on the probability itself depends on where you are: at
p=0.55 it is
p(1−p)β=0.55⋅0.45⋅(−0.054)=−0.0134, the marginal effect. The likelihood adds up log-probabilities of what happened: for predicted ups
0.8,0.3,0.6 and outcomes
1,0,1 the log-likelihood is
ln0.8+ln0.7+ln0.6=−1.09. McFadden's pseudo R-squared compares with a model that has only an intercept: if that model has
ℓ0=−69.3 and yours
ℓ=−60, then
1−60/69.3=0.134.
Real data keeps the lesson honest. For SPY over 5,403 trading days since 2005, 54.86% of days closed up. A logit of the next-day direction on today's return, in percent, gives α^=0.198 and β^=−0.054 with z=−2.34 (p=0.019): after a 2% fall the chance of an up day is 57.6%, after a flat day 54.9% and after a 2% rise 52.2%. The marginal effect is −0.0134 and the probit agrees (β^=−0.034, marginal effect −0.0134, same log-likelihood −3716.76). Yet McFadden's R2 is 0.0007, and predicting up whenever the probability exceeds one half is right on 54.97% of days against 54.86% for always saying up. A coefficient can be statistically detectable and still be useless for trading.
- Probabilities need a curve: logit and probit keep the fitted chance between 0 and 1, and they agree closely.
- Coefficients are log-odds: exponentiate for an odds ratio, multiply by p(1−p) for an effect on the probability.
Lab · what's graded
- From a score to a probability15 pts
- An odds ratio10 pts
- A marginal effect15 pts
- A log-likelihood20 pts
- A pseudo R-squared15 pts
- Match the vocabulary10 pts
- Significant but useless15 pts
