A backtest is a claim about a strategy you never actually traded. Three biases show up in almost every one of them, and all three push in the same direction: they make the curve look better than the strategy was. Here's how each gets in, and what to check.
1. Survivorship bias
Survivorship bias is testing on a universe that was chosen after you knew which instruments made it. The classic case is equities: backtest a screen on today's S&P 500 members over the last twenty years and you have quietly excluded every company that went bankrupt, got delisted or was acquired at a discount. The survivors are, definitionally, the ones that did well.
Index futures are usually presented as immune to this. They mostly are — ES has been continuously listed since 1997, and there is no bankruptcy risk in a front-month contract. But two versions still bite:
- Contract-level survivorship. Not every futures product survives. If you screen "liquid CME products" as of today and test back ten years, you have excluded the contracts that got delisted for lack of volume. Any conclusion of the form "this strategy works across futures markets" inherits the bias even if each individual series is clean.
- Strategy survivorship — the version that actually costs people money. You test forty parameter combinations, three of them look excellent, you publish those. Nobody deleted any data, but the reported result is still a maximum over forty draws rather than a typical outcome. This is the same statistical error wearing a different hat, and in discretionary strategy research it is far more common than the data kind.
2. Look-ahead bias
Look-ahead bias is using information at bar t that wasn't knowable until after bar t. It is the most damaging of the three because it doesn't degrade a result — it manufactures one. A strategy with a look-ahead leak can show an equity curve no real strategy would ever produce, and the tell is usually that the result looks too good.
Where it gets in
- Using the close of the signal bar as the fill. A rule like "if this bar closes above the opening range high, buy at this bar's close" cannot be executed: at the moment you know the close, the close has happened. Realistically you fill at the open of the next bar, or intrabar on a stop order — and on a breakout strategy the gap between those two is a material part of the edge.
- Indicators that peek. Centred moving averages, any smoother with a forward-looking window, and pivot/swing labels that need n bars to the right to confirm. A swing high is only a swing high in hindsight; using it as a live signal is a leak.
- Whole-sample normalisation. Z-scoring a feature against the mean and standard deviation of the entire history bakes the future into every early bar. Any scaling must be rolling and backward-looking.
- Revised or restated data. Less of a problem in futures than in fundamentals or macro, but relevant the moment you join in an economic release: you need the number as first published at its release timestamp, not today's revised value.
- Session and timezone boundaries. Index futures trade nearly around the clock. If your "daily" bar is built to a timezone that doesn't match the session you're trading, the bar can contain ticks that hadn't printed when you claim to have acted.
3. Slippage and costs
The third bias isn't subtle, it's just skipped. Commission, exchange and clearing fees, the spread, and slippage against your stop all come out of every round turn — and their effect scales with how often you trade, not with how much you make. A strategy taking one trade a week barely notices; one taking six a day can be entirely cost.
The arithmetic is worth doing explicitly, because the result surprises people. Take a strategy whose gross average is a quarter of a point per ES round turn, and vary only the cost assumption:
| Cost per round turn | Gross avg / trade | Net avg / trade | Net over 1,000 trades |
|---|---|---|---|
| $0.00 (ignored) | $12.50 | $12.50 | +$12,500 |
| $4.50 (commissions only) | $12.50 | $8.00 | +$8,000 |
| $17.00 (commissions + 1 tick) | $12.50 | −$4.50 | −$4,500 |
| $29.50 (commissions + 2 ticks) | $12.50 | −$17.00 | −$17,000 |
Two further details that a flat per-trade cost hides:
- Stops slip asymmetrically. A resting limit order fills at your price or not at all. A stop fills at the market after it's touched — and it gets touched precisely when the market is moving against you fast. Modelling stop exits at the exact stop price is a systematic overstatement, and a strategy with a tight stop and a wide target is the most exposed to it.
- Costs concentrate where volume doesn't. The spread you'll actually pay at the cash open is not the spread at 3am, and it is not the spread on a half-day session. If a strategy's trades cluster at one time of day, that time's real liquidity is the one that matters — an average is optimistic.
The short version
Before you believe a backtest, run these four:
- Shift entries one bar later. Does the edge survive?
- Add a cost per round turn that's larger than you think is fair. At what cost does the strategy break even? If that number is close to real-world costs, there is no edge, only a margin of error.
- Count the variants you tested. Then check the chosen one against data you held out from the start.
- Look at the trade distribution, not just the total. If removing the best five trades kills the result, you have five lucky trades, not a strategy.
None of this makes a backtest true. It makes it honest — which is the most a backtest can be. For what to expect from the numbers that survive these checks, see what a realistic profit factor looks like, and for how much data you need before any of it means anything, how many trades a backtest needs.