Look-ahead bias

Knowing too much

Look-ahead bias is when a backtest uses information that was not available at the decision time. It is the most common reason a strategy looks brilliant on paper and loses money live. It is rarely a single obvious line; it hides in alignment (a signal from today's close traded at today's close), in statistics computed over the whole sample, in data that was revised later, and in indices chosen with hindsight. The cure is a habit: for every number a decision uses, ask when it became known.

import numpy as np

rng = np.random.default_rng(5)
returns = rng.normal(0, 0.01, size=250)             # pure noise: no strategy can really make money here

signal = returns.copy()                              # "signal" = the return of the same day
position = np.sign(signal)

biased = np.prod(1 + position * returns) - 1         # trades on the return it is about to earn
honest_position = np.concatenate([[0.0], position[:-1]])
honest = np.prod(1 + honest_position * returns) - 1
print(round(float(biased), 2), round(float(honest), 2))        # 6.0 0.01: a 600% "return" on noise, against 1%

Alignment: the one-day lag

The cure for alignment bias is a lag: a value computed at day t may be used from day t + 1. In arrays it is a shift with zeros at the start, out[t] = signal[t - periods]. The test is blunt. A noise series has no edge, so any rule that makes a fortune on noise is cheating. Compare the biased and the lagged version on the same data, and the gap is the size of the bias.

# lag_signal: delay a series, with zeros where there is nothing to look back at
import numpy as np

def lag_signal(signal, periods=1):
    out = np.zeros(len(signal))
    if periods < len(signal):
        out[periods:] = signal[:len(signal) - periods]
    return out

s = np.array([1.0, -1.0, 1.0, 1.0, -1.0])
print(lag_signal(s))             # [ 0.  1. -1.  1.  1.]
print(lag_signal(s, 2))          # [ 0.  0.  1. -1.  1.]

# trailing_mean: the mean of the last `window` values, using only today and the past
import numpy as np

def trailing_mean(values, window):
    total = np.cumsum(values)
    out = np.full(len(values), np.nan)
    out[window - 1:] = (total[window - 1:] - np.concatenate([[0.0], total[:len(values) - window]])) / window
    return out

x = np.array([1.0, 2.0, 6.0, 7.0])
print(trailing_mean(x, 2))                         # [ nan 1.5 4.  6.5]
print(trailing_mean(x[:3], 2))                     # [ nan 1.5 4. ]: adding day 4 changed nothing before it

Statistics must grow with time

The second leak is subtle. Standardising a series with its whole-sample mean and standard deviation uses numbers that only exist at the end: early values are scaled by information from the future. Use only the past: a trailing window (the last n values up to today) or an expanding window (everything up to today). The test for any such feature: compute it on the first k days only, and it must equal the first k values of the full computation.

# Whole-sample z-scores leak the future; expanding ones do not
import numpy as np

x = np.array([1.0, 3.0, 2.0, 8.0, 4.0])
whole = (x - x.mean()) / x.std()

n = np.arange(1, len(x) + 1)
mean = np.cumsum(x) / n
var = np.cumsum(x ** 2) / n - mean ** 2
with np.errstate(divide="ignore", invalid="ignore"):
    expanding = (x - mean) / np.sqrt(var)
expanding[0] = np.nan

print(np.round(whole, 3))
print(np.round(expanding, 3))
# Prefix test: computing on x[:3] gives the first three expanding values, but not the first three whole-sample ones
print(np.round((x[:3] - x[:3].mean()) / x[:3].std(), 3))

• Look-ahead bias is any use of information before it was known; alignment is the classic form: a day-t signal can trade from day t + 1.
• Features must be built from the past only: trailing or expanding windows, never the whole-sample mean or standard deviation.
• Prefix test: a feature or backtest run on the first k days must equal the first k results of the full run; a noise series must not make money.

Next up: transaction costs, the first thing a clean backtest has to pay for.

Lab · what's graded

• Delay a signal with lag_signal25 pts
• Measure the cheat with biased_vs_honest25 pts
• Average only the past with trailing_mean25 pts
• Standardise without peeking with expanding_zscore25 pts