Filtering versus smoothing and the look-ahead trap

Skill positionModule 3 of 3 · Topic 3 of 4

3.3 Filtering versus smoothing and the look-ahead trap

—Lab score— / 100Total XP—Streak—

A signal you cannot trade on

Regime models are popular in trading: reduce exposure when the market is stormy. But there are two ways to get the probability of a state, and only one of them is honest. Filtered probabilities $\alpha_t$ use the data up to day $t$ . Smoothed probabilities $\gamma_t$ use the whole sample, including days after $t$ . For describing history, for example to colour a chart of past regimes, smoothing is better. For trading it is a trap: on the day you must decide, the future days are not available, so a rule built on $\gamma_t$ trades on information from the future. This is the look-ahead trap.

\hat p_{t+1}=\alpha_t\,A\ \ne\ \gamma_{t+1}

(1)

The honest recipe is to look one step ahead from what you know. After the close of day $t$ you have the filtered probabilities $\alpha_t$ . The predicted state probabilities for tomorrow are $\alpha_t\,A$ : move the belief one step with the transition matrix. Then decide tomorrow's exposure from these numbers and apply it to tomorrow's return. Predicted probabilities slowly forget today and drift toward the stationary distribution. One more source of cheating is easy to miss: fitting the model's numbers on the whole sample also leaks the future, so a careful test refits on past data only and rolls forward.

What the real data showsIn our small example the filtered chance of Stormy today is

0.903

, so tomorrow's predicted chance is

0.903\times0.8+0.097\times0.1=0.732

, and two days out

0.613

, drifting toward the stationary

1/3

. On SPY with a three-state model, staying in the market only when the high-volatility state is unlikely gives an annual Sharpe ratio of

0.66

with filtered, lagged signals, against

0.52

for buy-and-hold. The same rule on smoothed probabilities shows

1.12

. That extra

0.46

is not skill. It is the future leaking into the signal.

Filtered uses data up to today. Smoothed also uses later days and is for describing the past.
Honest signal: predict tomorrow with $\alpha_t A$ and apply it to tomorrow's return.
Also leaks the future: Viterbi paths and model numbers fitted on the whole sample. Refit on past data only.

Lab · what's graded

Tomorrow's chance of stormy15 pts
The day after tomorrow15 pts
Where predictions drift10 pts
How much the future inflates10 pts
Which signals use the future15 pts
An honest regime backtest10 pts
Match the vocabulary10 pts
Why the smoothed backtest looks too good15 pts

0/ 8 answered

1Numeric answer15 ptsNot answered

Today the filtered chance of Stormy is $0.903$ and of Calm $0.097$ . With $A_{SS}=0.8$ and $A_{CS}=0.1$ , what is tomorrow's predicted chance of Stormy, $0.903\cdot0.8+0.097\cdot0.1$ ? Give three decimals.

p =

accepted: ± 0.003

2Numeric answer15 ptsNot answered

Tomorrow's predicted chance of Stormy is $0.732$ . What is the predicted chance two days from today, $0.732\cdot0.8+0.268\cdot0.1$ ? Give three decimals.

p =

accepted: ± 0.003

3Numeric answer10 ptsNot answered

Far in the future the predicted chance of Stormy settles at the stationary value $A_{CS}/(A_{CS}+A_{SC})$ . What is it for $A_{CS}=0.1$ and $A_{SC}=0.2$ ? Give three decimals.

p =

accepted: ± 0.003

4Numeric answer10 ptsNot answered

On SPY the Sharpe ratio is $0.66$ with filtered, lagged signals and $1.12$ with smoothed ones. By how much does the look-ahead inflate it? Give two decimals.

g =

accepted: ± 0.01

5Choice15 ptsNot answered

Which of these quantities use data from after the day they describe? Pick all that apply.

6Order the steps10 ptsNot answered

Put the steps of an honest regime backtest in order.

Drag to reorder, or focus an item, press Space to lift it, the arrow keys to move it and Space to drop it.

1Apply the exposure to tomorrow's return
2Fit the model on past data only
3Move forward one day and refit from time to time
4Predict tomorrow with the filtered belief times A
5Filter forward to the close of today

7Match10 ptsNot answered

Match each term with its meaning.

Pick the partner of each item on the left.

Filtered probability

Smoothed probability

Predicted probability

Look-ahead trap

8Explain in words15 ptsNot answered

In one or two sentences, explain why a trading rule built on smoothed regime probabilities looks better in a backtest than it can ever be in practice.

Graded on:Smoothed probabilities use data from after the trading day, the futureSo the backtest is inflated and cannot be reproduced in real time0 / 400 characters

Math modeKeyboard or LaTeX inputTask 1 of 8 · Ctrl ⏎ to check

Filtering versus smoothing and the look-ahead trap

Skill positionModule 3 of 3 · Topic 3 of 4

3.3 Filtering versus smoothing and the look-ahead trap

—Lab score— / 100Total XP—Streak—

A signal you cannot trade on

\hat p_{t+1}=\alpha_t\,A\ \ne\ \gamma_{t+1}

(1)

What the real data showsIn our small example the filtered chance of Stormy today is

0.903

, so tomorrow's predicted chance is

0.903\times0.8+0.097\times0.1=0.732

, and two days out

0.613

, drifting toward the stationary

1/3

. On SPY with a three-state model, staying in the market only when the high-volatility state is unlikely gives an annual Sharpe ratio of

0.66

with filtered, lagged signals, against

0.52

for buy-and-hold. The same rule on smoothed probabilities shows

1.12

. That extra

0.46

is not skill. It is the future leaking into the signal.

Filtered uses data up to today. Smoothed also uses later days and is for describing the past.
Honest signal: predict tomorrow with $\alpha_t A$ and apply it to tomorrow's return.
Also leaks the future: Viterbi paths and model numbers fitted on the whole sample. Refit on past data only.

Lab · what's graded

Tomorrow's chance of stormy15 pts
The day after tomorrow15 pts
Where predictions drift10 pts
How much the future inflates10 pts
Which signals use the future15 pts
An honest regime backtest10 pts
Match the vocabulary10 pts
Why the smoothed backtest looks too good15 pts

0/ 8 answered

1Numeric answer15 ptsNot answered

p =

accepted: ± 0.003

2Numeric answer15 ptsNot answered

Tomorrow's predicted chance of Stormy is $0.732$ . What is the predicted chance two days from today, $0.732\cdot0.8+0.268\cdot0.1$ ? Give three decimals.

p =

accepted: ± 0.003

3Numeric answer10 ptsNot answered

Far in the future the predicted chance of Stormy settles at the stationary value $A_{CS}/(A_{CS}+A_{SC})$ . What is it for $A_{CS}=0.1$ and $A_{SC}=0.2$ ? Give three decimals.

p =

accepted: ± 0.003

4Numeric answer10 ptsNot answered

On SPY the Sharpe ratio is $0.66$ with filtered, lagged signals and $1.12$ with smoothed ones. By how much does the look-ahead inflate it? Give two decimals.

g =

accepted: ± 0.01

5Choice15 ptsNot answered

Which of these quantities use data from after the day they describe? Pick all that apply.

6Order the steps10 ptsNot answered

Put the steps of an honest regime backtest in order.

Drag to reorder, or focus an item, press Space to lift it, the arrow keys to move it and Space to drop it.

1Apply the exposure to tomorrow's return
2Fit the model on past data only
3Move forward one day and refit from time to time
4Predict tomorrow with the filtered belief times A
5Filter forward to the close of today

7Match10 ptsNot answered

Match each term with its meaning.

Pick the partner of each item on the left.

Filtered probability

Smoothed probability

Predicted probability

Look-ahead trap

8Explain in words15 ptsNot answered

In one or two sentences, explain why a trading rule built on smoothed regime probabilities looks better in a backtest than it can ever be in practice.

Graded on:Smoothed probabilities use data from after the trading day, the futureSo the backtest is inflated and cannot be reproduced in real time0 / 400 characters

Math modeKeyboard or LaTeX inputTask 1 of 8 · Ctrl ⏎ to check

Filtering versus smoothing and the look-ahead trap

A signal you cannot trade on

Lab · what's graded

Log in to run your code.

Filtering versus smoothing and the look-ahead trap

A signal you cannot trade on

Lab · what's graded

Log in to run your code.