Trading System #1–SPY VIX (Part 2)

In http://www.optionfanatic.com/2012/09/28/trading-system-1-spy-vix-part-1/ (9/28/12), I introduced the concept of evolutionary operation and began to talk about the different variables present in this trading system.  Today I want to continue that discussion from the perspective of optimization.

Investopedia.com defines optimization as:

“…the process of adjusting one’s trading system in an attempt to make it more effective. These adjustments include changing the number of periods used in moving averages, changing the number of indicators used, or simply taking away what doesn’t work.”

In the system development process, optimization is going to allow me to look at performance metrics for every combination of specified parameter values.  In the present context, I will allow x to vary from 3-7 in increments of one [e.g. 3 (days), 4, 5, 6, or 7].  I will allow y to vary from 5-25 in increments of five [e.g. 5 (percent), 10, 15, 20, or 25].  I will allow z to vary from 6-15 in increments of one.  The result of this process will be summary statistics for 5 * 5 * 10 = 250 trading systems.

The goal of optimization is not to find out which set of parameter values performed the best in order to trade those going forward.  This is called “curve-fitting” and the reason optimization has gotten a bad name in some circles.  The more variables you have and the greater the number of total parameter value combinations, the more unlikely you are to see similar performance in the future as seen in the most profitable backtest.

Rather, the goal of optimization should be to assess the performance landscape from one combination of parameter values to the next.  Going back again to http://www.optionfanatic.com/2012/09/28/trading-system-1-spy-vix-part-1/ (9/28/12), I want to know if these performance graphs have spike regions or plateau regions.  If they have spikes then I will discard them, considering performance to be lucky rather than good.  If they have plateau regions then I want to choose parameter values near the middle of those plateaus (margin of safety).

In my next post I’ll start running the numbers.

Trading System #1–SPY VIX (Part 1)

With the “motion to dismiss” now denied, the next step is to lay the groundwork for developing this strategy into a trading system.

As a review, the trading strategy is based on two claims.  First, five days after the CBOE Volatility Index (VIX) closes 5% or more below its 10-day simple moving average (SMA), the S&P 500 index (SPX) has lost money on average. Second, when the VIX closes 5% or more above its 10-SMA, SPX has outperformed the average week better than 2-1 over the next five trading days.

The current system in development offers three variables to be backtested:  x-bar stop y% extended from its z-SMA.  In this case, we have used x=5 (five-day trades), y=5, and z=10 (VIX closing 5% above/below its 10-SMA).  What we want to see in backtesting is solid results with these values.  We also want to see solid results when we slightly change these values, though.

The process of “evolutionary operation” looks to keep one variable constant while changing another to see its effect on system performance.  Graphically speaking, we want to plot some measure of performance (e.g. net profit) vs. the one variable being changed:

This does not have the appearance of a robust system because slight changes in the variable drastically affect net profit.  Whether a moving average is four periods vs. five or 14 periods vs. 15 should not matter.  If one value appears magically improved in the midst of mediocre results then those backtested profits are likely a matter of luck rather than the system taking advantage of consistent opportunity (and subsequent future profits).

This is what we hope to see.  All values from 6-15 produced solid profit for this trading system.  Rather than the spikes in the previous graph we see a plateau region in this one.  This should be a necessary (but not sufficient) condition for a system we might trade in the future with real capital.

These graphs were adapted from Tomasini and Jaekle (2009).

In my next post I will continue this discussion in terms of optimization.

Trading System #1–Brief Notes on Statistical Interpretation

In addition to generating profit and loss numbers, sophisticated backtesting software will provide a whole slew of statistics each designed to address a different aspect of the performance results.

Without question, one of the most complicated matters in developing a system is how best to evaluate it.  Different aspects will be more or less important to different people.  One person may not care about the average win vs. average loss as long as most trades win.  Another person may not care about how few trades win and how large the drawdowns get as long as net return is maximized over time.  Younger people can afford to lose more because time is on their side.  Retirees may be risk-averse because what they lose they don’t have so much opportunity to regain.

From a graphical point of view, many backtesting statistics describe the equity curve itself.  Supposing a given initial value ($1M in our case), the equity curve shows the value of the account over time.  A curve sloping up (down) is making (losing) money.  Drawdowns are the difference (in dollars or percent) from an equity value peak to valley.  Flat times describe time intervals between new equity highs.  Most people prefer shallow drawdowns and shorter flat times to sleep easier at night.  In theory, this may be achieved at the expense of total profit and exponential (sloping up more and more over time) equity curves.  A lower total profit often means a safer system, however, that has a lower Risk of Ruin:  probability of the account going to zero (“bust”) from a string of losses or just a couple large losses.

Like trading itself, much of system development is about tradeoffs.