To Optimize or Not to Optimize?

In the quest for consistent trading profits, backtesting is generally regarded as useful but optimization is viewed by some as a four-letter word.

If a system captures profits from consistent human behavioral characteristics reflected in the market then the past should reasonably approximate the future.  This will never be perfect but it may be profitable.

Optimization is the determination of what trading parameters applied to past market action would have resulted in maximal returns.  After all, if it worked well in the past then shouldn’t be likely to work in the future if the past is at least some small reflection of the future due to those consistent behavioral characteristics?  This is called curve-fitting.

One of the biggest scams on Wall Street is to present a system with brilliant historical returns (anyone can run an optimization procedure on backtested data to do this) and sell it as a system to be used in live trading.  Trading an optimized system into the future is almost guaranteed to underperform the past.  Selling an optimized system can be like selling snake oil, and charlatans abound in financial circles who aim to do just that.

To avoid excessive curve-fitting, some think that if you don’t optimize then you can’t fall prey to the illusion.  Choose only one parameter value and backtest it; if you come up with impressive results then they’re likely to persist into the future, right?

Not necessarily!

Suppose a system has one parameter (e.g. period for a moving average).  Imagine backtesting the system repeatedly over the historical time period while changing the period each time.  When you plot performance, does it look like this…

…or like this?

In both cases, using a period of A would have resulted in the best historical performance. However, in the latter case, if the period varied just a tad higher or lower then system performance was devastated. In the former case, trading with a period higher or lower than A would still have generated solid performance. Since the past is never identical to the future, which do you think has a better chance of generating profits going forward?

The bottom line is that “curve-fitting” and “optimization” are not four-letter words if used to know what you do not know.  Optimize not with the intent of finding the perfect parameter to trade going forward. Optimize with the intent to determine whether encouraging past results are likely to persist into the future.