Backtesting Conundrum with S&P 500 Stocks

I concluded http://www.optionfanatic.com/2012/11/12/position-sizing-implications-of-multiple-open-positions-part-3 by affirming the validity of backtesting S&P 500 member stocks as a proxy for results obtained with SPY.  Today I want to address two data challenges with backtesting S&P 500 member stocks.

The first challenge that must be overcome is survivorship bias.  Wikipedia explains:

> In finance, survivorship bias is the tendency for failed companies to be excluded
> from performance studies because they no longer exist. It often causes the results
> of studies to skew higher because only companies which were successful enough to
> survive until the end of the period are included.

Bankrupt companies are the quintessential example of survivorship bias.  Stocks of bankrupt companies get delisted from the exchange on their way to hitting $0.00/share.  Many bankrupt companies were once members of the S&P 500.  If I run a backtest on “S&P 500 stocks,” then bankrupt stocks are not going to be included since they are no longer in the database.

The second challenge of backtesting S&P 500 stocks is to accurately manage changes in index composition.  Stocks are added and deleted from the S&P 500 on an irregular, but not infrequent basis.  If I run a backtest on “S&P 500 stocks” then my database will look at those stocks currently in the S&P 500 folder. That folder is current as of right now but not accurate for historical dates.

To truly serve as a proxy for backtesting SPY, the individual S&P 500 stocks can therefore be used if:  1. the database includes delisted stocks to avoid survivorship bias; 2. The database tags stocks with both index membership and defined time interval(s) during which that index membership took place.