Truth in Backtesting (Part 2)

Backtesting must be done the same way that live trades are executed to prevent results from being contaminated with immeasurable error due to differences in trading method.  The question at hand addresses the implicit assumption with end-of-day (EOD) backtesting that transactions will take place at the closing print.  A detailed look will make it evident that getting the closing price in live trading is at best more complicated than one might think and at worst not even realistic.

Market on Close orders would seem like the obvious choice to trade based on EOD backtesting.  As explained on the Interactive Brokers web site:

> [Suppose] You want to buy 100 shares… and decide that the closing price…
> has… proven to be the best price of the day. Create a BUY order, then select
> MOC… to specify a market-on-close order… You must submit the MOC order
> at least 15 minutes prior to the close. The order will be submitted at the close
> as a market order.

A significant caveat to MOC orders is explained on the Orion Futures web site:

> …this order will be executed on the market close. The fill price will be within
> the closing range which may in some markets be substantially
> different from the settlement price. [emphasis mine].

With a MOC order, I don’t know how far off the closing price I will be filled!  Only through experimentation with different securities and different lot sizes can I really get a feel for how big the “closing range” is.  For all I know, getting filled at the closing price is a rarity.  As a prudent system developer, I should ultimately include the average deviation between closing price and actual fill as slippage into my backtesting because slippage can substantially affect the equity curve.

In my next post, I will discuss an alternative to MOC order placement.