Testing a discovery honestly
This capstone puts the whole skill into one question: a trend rule that goes long when the last L days summed positive and short otherwise, tried for 24 lookbacks on twenty years of SPY. Its best version has a Sharpe ratio near 0.31. Is that an edge? First estimate the Sharpe ratio, then measure its uncertainty with the bootstrap, then correct for the fact that you looked at 24 versions and reported the best.
The first formula annualises the daily mean over the sample standard deviation. The second turns it into a z-score for the null of zero mean: the daily Sharpe ratio times the square root of the m strategy days. The third is the Bonferroni correction for N trials. Two details make the rule honest. The position on day t may use only returns up to day t-1, because a signal that peeks at the day it trades earns the absolute return every day. And the p-value of the best of N rules is not the p-value of one rule chosen in advance.
- Sharpe ratio: annualised mean over standard deviation, with the sample (n - 1) standard deviation.
- Bootstrap: resample days with replacement and read the 2.5th and 97.5th percentiles of the statistic.
- No look-ahead: a position on day t can use only returns up to day t - 1.
- Multiple testing: the p-value of the best of N tries is N times larger than it looks.
Lab · what's graded
- The Sharpe ratio15 pts
- A trend rule with no look-ahead25 pts
- A bootstrap interval25 pts
- Searching and correcting35 pts