Challenges to Option Backtesting (Part 1)

I had an exchange with another trader some months ago about option backtesting. I wanted to copy and paste parts of my last response because they are good things to keep in mind:

    “Greeks do lie sometimes… I was just on the phone
    with OptionVue (OV) discussing a position I was
    studying… between 10:30 AM and 11 AM yesterday.
    Greeks suggested a market rally and volatility
    contraction would increase cash flow when cash
    flow significantly decreased. The back month
    option in the spread had no volume change over
    that 30 minutes so without any volume that may
    have been a stale quote. This is another
    potential contaminant to accurate backtesting…

    One thing that has frustrated me over the years
    with OV is that I can open a matrix and get
    identical prices but different greeks simply by
    refreshing the data, which I suppose forces the
    program to recalculate modeled values. If I’m
    looking to sell nothing greater than a 10 delta,
    for example, then a 10.4 versus 9.8 delta
    simply as a result of refreshing the data means
    the backtest is not reliable: one option may
    land a winner and the other a loser. This is
    another argument in favor of a large sample
    size because if small then the difference
    between a winning trade and a losing one can
    be very significant in the totality of results…

    Just this morning I was reading a trading forum
    and I found this post:

     > I have found backtesting options strategies
     > to be very difficult mainly due to the large
     > bid/ask spread.

     > If, in your backtest, you always use the
     > worst price, the strategy will never
     > make money. If you use the best price
     > then your probability of success is
     > artificially inflated.

     > Backtesting options strategies is very
     > hard, and the results given by most
     > websites and/or software seems to always
     > show the best case scenario. Most of the
     > time this is not happening

     This really hit home. I’ve encountered this for
     years and I totally agree.”

I’ll finish up with the next post.