What is backtesting, how is it done, and what does it not prove?
Explaining the steps of a backtest, the assumptions it rests on, typical traps like lookahead and overfitting, and what the results do and do not prove.
Backtesting is applying a defined rule to historical data and measuring how it performed. Its purpose is to see whether the rule behaves sensibly and to extract rough statistics (such as the win/loss distribution and maximum drawdown). It is not a tool to find a “profit machine” but a tool to eliminate bad ideas.
Why it matters
A good backtest lets you discard bad ideas before risking live money. But a bad backtest does the opposite: you fit the rule to the past so tightly that it looks great and then collapses in the future. So knowing what a backtest does not prove matters as much as knowing what it does.
The backtesting lifecycle
- 1Fix the rule
All parameters are written and frozen before testing.
- 2Prepare the data
Clean, realistic data covering a sufficient period.
- 3Add costs
Commission, spread, and slippage are included.
- 4Run and measure
Drawdown, distribution, and trade count are recorded.
- 5Interpret
Read the result as the strength of a hypothesis, not as proof.
Every backtest carries hidden assumptions: that orders always fill, that costs are constant, that past liquidity will exist in the future. If these are not stated, the result looks more reliable than it really is.
Typical traps
- Lookahead bias
Using data that could not be known at the time (e.g. using the day’s close intraday).
Use only the data available at that moment.
- Overfitting
Fitting parameters perfectly to the past; mistaking noise for signal.
Use few parameters; validate on out-of-sample data.
- Ignoring costs
Omitting commission, spread, and slippage.
Model realistic costs and the worst case.
Interpreting the result
A good backtest result does not say “this rule wins”; it says “this rule behaved sensibly in the past and is worth testing further.” A single shiny equity curve is usually a sign of overfitting; consistency and robustness are worth more than brilliance.
In the Tradora context, a backtest is one step and not sufficient on its own. A rule that behaved well in the past is not yet ready for live use — the next stage is to observe it forward, under real-time conditions.
Past performance is not a guarantee of future results — a core warning regulators repeat constantly. Even a good backtest only lowers the odds; it does not remove the risk.
Once you see the limits of a backtest, learn forward validation and how to read result metrics honestly. Move on to paper trading and overfitting.