Every trader eventually notices the same pattern: strategies look brilliant in the backtest and ordinary — or worse — the moment real money follows them. The usual explanations are vibes (“the market changed”) or conspiracy (“they hunt my stops”). The real explanation is more boring and more useful: a backtest and a forward test answer different questions, and almost everything published online only ever shows you the first one.
What a backtest actually is
A backtest asks: if I had traded these rules in the past, what would have happened?Done honestly, it’s the most useful tool in trading — it kills bad ideas in seconds and shows you a strategy’s character (how it loses, how long it sits flat, what its worst stretch felt like) before you pay tuition for the lesson.
But a backtest has a structural weakness no amount of data fixes: it is graded on an exam whose answers existed before you wrote it. Four leaks follow from that, and they account for most vanishing edges:
- Selection.You tested twenty ideas and kept the best one. The best of twenty random strategies also looks good — that’s what “best of twenty” means. The published winner carries the invisible luck of its dead siblings.
- Overfitting.Every parameter you tuned — the 20-day lookback, the 8% stop — was tuned against the same history you’re now citing as evidence. The more knobs, the more the “edge” is just memorized noise.
- Look-ahead and survivorship.Subtler leaks: rules that peek at data the bar hasn’t produced yet, universes built from stocks you already know survived. (Our engine evaluates entry and exit rules only on closed bars, which closes the worst of the first — though every simulator’s fill-price conventions involve judgment calls, ours included. Picking today’s megacaps as your test universe — which our own templates do — is a form of the second, and we label it wherever it applies.)
- Costs.Slippage, spreads, and missed fills are small numbers that compound into the whole edge at higher trade frequencies. A backtest that ignores them isn’t optimistic — it’s fictional.
What a forward test changes
A forward test asks a harder question: do these exact rules make money on data that did not exist when the rules were written? Nothing about it is sophisticated — it’s just a backtest with the one thing money actually requires: the future.
Three properties make it the referee:
- The rules are frozen first. No quiet re-tuning after a bad week. The spec is locked before the data arrives, so overfitting has nothing left to fit.
- Every result counts.There is no drawer to hide the losers in. Selection bias needs a selection step, and forward records don’t have one.
- Time does the auditing. A 20-day live record is modest evidence; a 200-day one is a reputation. Either way it accumulates in public, one closed candle at a time.
How to use both without fooling yourself
- Backtest to reject, not to believe.The backtest’s honest job is falsification: most ideas should die there, cheaply.
- Stress the survivor.Shift the date window, change the universe, fatten the costs. An edge that only exists in one decade on five stocks isn’t an edge — it’s an anecdote. (We wrote up a strategy of ours that failed exactly this kind of scrutiny — and published it anyway.)
- Then freeze it and go live on paper.Deploy the exact spec to a forward test and stop touching it. The waiting is the test. Anyone unwilling to wait is telling you which question they’re afraid of.
The uncomfortable summary: backtests are where edges are found, and forward tests are where they’re proven — and the entire strategy-selling industry is built on hoping you never insist on the second step. Insist on it. You can build and backtest a strategy free in the lab, and when you believe it, put it on the record.