Why a backtest alone is not enough: overfitting, out-of-sample and walk-forward

A backtest is a simulation of rules against historical data. It is useful, cheap and necessary, but it is not proof. The main reason is simple: if you try enough variations, one of them will look excellent on the exact data you used to choose it.

Overfitting in plain terms

Overfitting means describing the noise of the past rather than a repeatable behaviour. Signs include a strategy that needs many conditions, parameters tuned to unusual precision, or performance that collapses when you change a small setting.

A useful discipline is to ask, for every rule: could I explain why this should work to another trader without pointing at the equity curve? If the only justification is the result, the rule is probably fitted.

In-sample and out-of-sample

Split history into two parts before you optimise. Design and tune on the in-sample period. Then run the frozen strategy once on the out-of-sample period. If the result is very different, you have learned something valuable about fragility.

Do not reuse the out-of-sample period repeatedly; it slowly becomes another in-sample period. Keep a final hold-out that you touch rarely, and be honest with yourself when you decide to look.

Walk-forward

Walk-forward testing repeats a cycle: optimise on a window, trade the next window untouched, move forward, repeat. It approximates how a strategy would have been maintained in reality and shows whether parameter choices remain stable across regimes.

Record the chosen parameters for each window. If they jump wildly, the strategy is sensitive to the sample.

Costs and data realism

Spread, commission, swap, slippage and execution delay all reduce a simulated result. Test with the costs you actually expect. Check how the engine handles gaps, missing bars and the spread during volatile periods.

Small samples produce impressive-looking statistics by accident. Prefer more trades, more years and more than one instrument before drawing conclusions.

Metrics with caution

Profit factor, win rate, average trade and drawdown describe different things. A high win rate can hide rare large losses; a smooth curve can hide a dependency on a handful of trades. Look at the distribution, not only the totals.

What to do instead of trusting one number

Use the backtest to remove bad ideas, not to certify good ones. Combine out-of-sample results, walk-forward stability, cost sensitivity and Monte Carlo ranges. Then observe the strategy in a demo account before risking capital.

This article is educational and is not financial advice.