Purged and embargoed cross-validation

Cross-validation that respects the clock

Cross-validation reuses the data: it cuts it into folds, tests on one fold and trains on the rest. For independent rows that works, but financial labels overlap in time. If the label of row $j$ is the return from $j$ to $j+H$, its window is $[j,j+H]$. A test fold that covers rows $a$ to $b-1$ has labels covering $[a,b-1+H]$. Any training row whose window touches that range, which means $a-H\le j\le b-1+H$, shares returns with the test labels. Keeping such rows lets the model see part of the answer.

Purging removes the overlapping training rows. An embargo removes a further stretch of $e$ rows after the test fold, because returns are autocorrelated and volatility clusters, so the first rows after a test block still carry information about it. In a walk-forward scheme training only comes before the test, and the same idea becomes a gap: train on rows up to $a-H-1$, so the gap is $H$ (plus $e$). scikit-learn's TimeSeriesSplit has a gap argument for this. On the monthly stock panel with $H=3$, the walk-forward information coefficient is $0.0326$ with no embargo and $0.0327$ with $3$ months of embargo: the cleaning changes little because the label overlap is short, but you cannot know that without doing it.

• Purge, then embargo: drop training rows whose labels touch the test labels, then a few more after the test.
• A gap in walk-forward: train only on rows at least $H$ before the test.
• Do it even if it changes little: a clean evaluation costs a few rows, a leak can cost a strategy.

Lab · what's graded

• Rows purged before the fold10 pts
• Rows purged after the fold15 pts
• The training size that remains15 pts
• A walk-forward fold with a gap10 pts
• Which rows are purged10 pts
• A purged split10 pts
• Match the vocabulary15 pts
• Why an embargo as well15 pts