Walk-Forward Validation, Properly

A backtest over your full history tells you one thing reliably: how good you became at fitting that history. It cannot distinguish a strategy that captures something durable from one that memorised the particular sequence of moves in your sample, because both produce the same beautiful equity curve. Walk-forward validation exists to force that distinction.

How walk-forward differs from a single backtest

In a single backtest you choose parameters using all available data, then report performance on that same data. The result is guaranteed to flatter you, because the parameters were selected precisely because they performed well on those observations.

Walk-forward instead moves a pair of windows through history. Parameters are fitted on the in-sample window, then applied unchanged to the immediately following out-of-sample window. The windows advance, the fit is repeated, and the out-of-sample results are concatenated into a single series. Every point on that series was produced by parameters chosen without seeing it.

The performance gap that tells you you have overfitted

Some degradation from in-sample to out-of-sample is normal and expected. The size of the gap is the diagnostic. A strategy that returns a Sharpe of 2.4 in-sample and 0.3 out-of-sample has not been validated; it has been shown to be curve-fitted, quite precisely.

There is no universal threshold, but a workable heuristic: out-of-sample performance below roughly half the in-sample figure is a strong signal that the parameters are fitting noise. Consistency across windows matters at least as much as the average. Out-of-sample results that alternate between excellent and awful describe a strategy whose behaviour you cannot predict, even if the mean looks acceptable.

This is the same trap the common backtesting mistakes piece describes as multiple-testing bias, seen from the validation side. Each iteration through the loop of tune, test, adjust makes the reported figure less meaningful, and nothing in the output warns you that it is happening.

Choosing window lengths

Window length is the one parameter of the validation procedure itself, and it needs to be set from the strategy's trade frequency rather than from a calendar convention.

The binding constraint is trade count, not elapsed time. An in-sample window needs enough trades for the fitted parameters to mean something — a few hundred at minimum — and each out-of-sample window needs enough trades for its result to be more than noise. A strategy taking two trades a week cannot be validated on one-month windows no matter how many windows you use.

from dataclasses import dataclass  @dataclassclass WindowResult:    is_start: int    oos_start: int    oos_end: int    params: dict    is_metric: float    oos_metric: float    oos_trades: int  MIN_IS_TRADES = 300MIN_OOS_TRADES = 40  def walk_forward(data, strategy, is_len: int, oos_len: int, step: int, anchored=False):    results: list[WindowResult] = []    cursor = 0     while cursor + is_len + oos_len <= len(data):        is_slice = data[0 : cursor + is_len] if anchored else data[cursor : cursor + is_len]        oos_slice = data[cursor + is_len : cursor + is_len + oos_len]         params, is_metric, is_trades = strategy.optimise(is_slice)        if is_trades < MIN_IS_TRADES:            raise TooFewTrades(f"in-sample has {is_trades} trades, need {MIN_IS_TRADES}")         oos_metric, oos_trades = strategy.evaluate(oos_slice, params)        results.append(            WindowResult(                is_start=0 if anchored else cursor,                oos_start=cursor + is_len,                oos_end=cursor + is_len + oos_len,                params=params,                is_metric=is_metric,                oos_metric=oos_metric,                oos_trades=oos_trades,            )        )        cursor += step     return results  def degradation_ratio(results: list[WindowResult]) -> float:    """Out-of-sample performance as a fraction of in-sample. Below ~0.5 is a warning."""    usable = [r for r in results if r.oos_trades >= MIN_OOS_TRADES]    if not usable:        raise TooFewTrades("no window had enough out-of-sample trades")    is_mean = sum(r.is_metric for r in usable) / len(usable)    oos_mean = sum(r.oos_metric for r in usable) / len(usable)    if is_mean <= 0:        return 0.0    return oos_mean / is_mean  class TooFewTrades(Exception):    pass

The exceptions do the real work there. Raising when a window has too few trades stops the procedure from producing a confident-looking number derived from a handful of observations, which is the most common way walk-forward gets misapplied — and the resulting figure looks no different from a sound one.

How many strategies survive each stage

It is worth setting expectations about attrition before starting, because the rate at which ideas die is high enough to feel like a mistake rather than the normal operation of the process.

Most ideas fail at the in-sample stage. Of those that pass, most fail walk-forward. Of those that survive walk-forward, several fail once realistic costs and slippage are applied, because a strategy with a small per-trade edge and high turnover is exactly the kind that looks best before costs. Of the survivors, some still fail in paper trading for operational reasons that have nothing to do with the signal.

A validation protocol you can actually follow

Walk-forward validation does not tell you a strategy will make money. It tells you whether the evidence you have is consistent with an edge that exists outside your sample. That is a weaker claim than most backtest reports imply, and it is the strongest claim the data can actually support.