The deflated Sharpe ratio and backtest overfitting

A great Sharpe ratio, found among hundreds

The Sharpe ratio is the mean excess return divided by its standard deviation. From daily data it is annualised by $\sqrt{252}$: a daily mean of 0.05% and daily volatility of 1% give a daily ratio of 0.05 and an annual ratio of $0.05\sqrt{252}\approx0.79$. Like any estimate it has a standard error, roughly $\sqrt{(1+\mathrm{SR}^2/2)/T}$ for $T$ years of data: even a true Sharpe of 0.8 measured over four years is only known to about $\pm0.57$, so two strategies with Sharpe 0.8 and 0.3 are statistically hard to tell apart.

Now try $N$ strategies. Even if every one has zero true Sharpe, the estimates scatter around zero with spread $\sigma_{\mathrm{SR}}$, and the best of them is positive and grows with $N$, roughly $\sigma_{\mathrm{SR}}\sqrt{2\ln N}$. With a spread of 0.5 and 100 trials, the best is expected near $0.5\sqrt{2\ln100}\approx1.52$ from luck alone. This is backtest overfitting. The deflated Sharpe ratio (Bailey and López de Prado) corrects the observed best Sharpe for the number of trials, the skewness and kurtosis of returns and the sample length. In this topic's simplified form: subtract the expected best-of-$N$ noise before believing the number.

• Sharpe ratio: mean excess return over volatility; annualise daily data by $\sqrt{252}$.
• Best of $N$: noise alone gives a best Sharpe near $\sigma_{\mathrm{SR}}\sqrt{2\ln N}$.
• Defences: record every trial, keep a final holdout, prefer simple ideas with a reason, deflate the winner.

Lab · what's graded

• An annualised Sharpe ratio15 pts
• The standard error of a Sharpe ratio15 pts
• The best of many from noise15 pts
• What to do with the winner15 pts
• Defences against overfitting10 pts
• A deflated Sharpe ratio15 pts
• Why the best backtest may be luck15 pts