Challenges to Option Backtesting (Part 2)

Earlier this year I had an e-mail correspondence about difficulties with option backtesting. My pasted excerpts made points about how stale quotes may produce misleading data, how software mechanics may lead to unreliable backtesting, and how proficiency (or lack thereof) with trade execution may affect backtesting results.

I had one particularly memorable experience with the latter point about slippage. I backtested a calendar trade over two years with and without slippage. In the former case, the trade made huge money. In the latter case, the account was ground into minced meat. This was a night/day difference between a trade I would jump to put on every single month versus a trade I would never even wish on my worst enemy.

I try to be particularly cognizant of transaction fees when I backtest. I would rather estimate transaction fees too high than too low when looking at a trade I might actually do with real money.

One thing I have learned in the years since studying that calendar trade is the profound effect of sample size. I believe minor inaccuracies (also known as “random error”) here and there will be averaged out with a large number of occurrences. Put a different way, the more occurrences I have to study, the larger will be the signal-to-noise ratio. If a trade has worked in the past then I will be more likely to detect it.

As long as I am reasonable with regard to slippage, backtested results will give me the confidence to trade with real money. No methodology to trading system development will ever be perfect. At the end of the day, the ultimate goal must therefore be having the necessary confidence to trade it live and to stick with it during expected drawdowns to emerge profitable on the other side.