Logistic regression for direction

From a score to a probability

Logistic regression predicts a yes or no, such as whether tomorrow's return is positive. It first builds a linear score $z=a+bx$, exactly like a linear regression, and then turns it into a probability with the sigmoid $p=1/(1+e^{-z})$. The score is the log of the odds: $z=\ln\frac{p}{1-p}$, so $e^{z}$ is the odds. A score of $0$ means $p=0.5$. The model is fitted by minimising the log-loss, $-\ln p$ when the outcome was yes and $-\ln(1-p)$ when it was no, which punishes confident mistakes heavily. The class you report depends on a threshold, usually $0.5$, that you can move.

Accuracy alone misleads when one class is common. A model that always says up looks good if most days are up, and it is useless for trading. Judge a direction model with the log-loss against the constant-rate baseline, with the calibration of its probabilities and, above all, with the return of a strategy that acts on them after costs. In a threshold decision, the cost of a wrong buy and a wrong skip are rarely equal, so the best threshold is often not $0.5$.

• Score then sigmoid: a linear score becomes a probability between $0$ and $1$.
• Log-loss punishes confidence: a confident wrong answer costs far more than a hesitant one.
• Beat the baseline: compare with always predicting the base rate, not with $50\%$.

Lab · what's graded

• A probability from a score15 pts
• The odds10 pts
• A log-loss15 pts
• Where the model is undecided10 pts
• The constant baseline10 pts
• Why accuracy misleads10 pts
• Match the vocabulary10 pts
• Why the model barely beats the baseline20 pts