Backtest, forward test, live

12 minutesLesson 6 of 9
Femi, the HustlCraft strategy mascot

Three stages, in this order, every time. Each answers a question the one before it cannot. People who skip to the last are not saving time. They are paying for the answer with money instead of patience.

Three stages, three different questions

StageThe question it answersWhat it still cannot know
BacktestWould these rules have survived past prices at all, and how often do they trade?Your broker's real fills, the spread at a release, or anything about the future.
Forward test on demoDoes the program behave in real time the way the report said?How the broker fills real size, and how you behave when the money is yours.
Live, at the smallest sizeDo the fills, the costs and your own nerve match what you were shown?The long run. This is the start of the evidence, not the end of it.

Each stage catches a different family of mistakes, and no stage catches the family below it. That is why there are three.

What each stage catches

Six weeks on demo, one discovery

  1. A rule tests well over two years, so instead of going live you put it on a demo account at the same broker.
  2. In week three the terminal restarts overnight after a Windows update. The program comes back, finds nothing in its own memory, and opens a second position alongside the one already open.
  3. Two positions on the same symbol, double the intended risk, and a daily loss limit that now means nothing.

No backtest on earth reports that, because in a backtest nothing ever restarts. It costs six weeks of demo to discover, or one bad night of live money.

Why people skip straight to live

The reasons are always the same and none of them are stupid. The tester report looked convincing, a seller said the results were proven, and demo money feels unreal.

But the parts that make demo unlike live are exactly the parts stage three exists to measure, the argument of why backtests and demos mislead. Skipping a stage does not remove the question, it moves the answer to a more expensive place.

The most expensive sentence in automated trading is: the backtest looked good, so I went live.

A sequence you can actually follow

  1. Write the rules down until a stranger could follow them without asking a question.
  2. Backtest over at least a year and more than one kind of market, with your broker's real spread and commission entered rather than the defaults from the three costs.
  3. Change one setting at a time and watch how quickly the result falls apart. A rule that survives at only one exact value is a warning.
  4. Keep back a stretch of history you never used while tuning, and run the finished rule over it once.
  5. Forward test on demo at the same broker long enough to include a news week, a quiet week and a restart.
  6. Only then go live at the smallest size the account allows, compare every fill against what the test promised, and let size follow live evidence.

The picture to keep

Three gates stand before the money. The backtest asks whether the car starts, the forward test asks how it handles traffic, and live asks what happens when the fuel is yours.

PreviousNext: Over-fitting, explained simply