Truth in Backtesting (Part 12)

I will complete (for now) this blog mini-series by detailing a few additional backtesting pitfalls likely to misrepresent potential trading system candidates according to Kevin Davey.

The third backtesting pitfall involves limit orders that fill when price is touched rather than penetrated.  Davey writes:

> One of the tricks unscrupulous system vendors play is to assume that all limit
> orders are filled as soon as the price is touched.  You can recognize this… If the
> method shows trades… bought at the exact low of a bar, and/or sold at the exact
> high…you can bet this game is being played… Of course, the reality is that it is
> hard to buy the low and sell the high.  My experience is that… you can probably
> do this 5-20% of the time.  The other 80-95%… price has to trade through your
> price to get you a fill at the limit price… This can be an issue…  If your back-test
> engine assumes… limit orders are filled when touched, the results will be…
> optimistic.  If the back-test engine assumes price must be penetrated to get a
> fill, then the back-test results will… be… pessimistic.  I always go with the
> pessimistic approach.  My actual results can then only be better
> than the back test [emphasis mine].

The final pitfall Davey describes involves strategies that exit on same bar as entry or have tight stops or profit targets such that a profit and loss exit could occur on the same bar:

> My experience is that it is easy to trick a strategy engine… when entry and
> exits occur on the same bar.  This is due to the assumptions the strategy
> engine must make regarding price travel.  Usually, the results will be overly
> optimistic when compared to real live trading.

A first goal should be to backtest in a way that closely models live trading.

A second goal should be to trade like I backtest.

If these are done then I have a better chance of developing a trading strategy that will work just as well in real time.