The Essence of Trading System Development (Part 2)

Today I continue describing my overall concept of trading system development.

We want to steer clear of overfit models. A model that backtests well and does not perform well going forward is overfit. “Steer clear” means do not trade this strategy. It may also mean abandon further attempts to develop the strategy. There are an infinite number of possible models to test; rather than trying to tweak one that doesn’t work [well] to make it better, toss it and move on to something else.

Trying to tweak a subpar strategy for improvement is a form of curve fitting that can result in overfit models. We can always find a model that better fits historical data by making it more complex.* In mathematics, more terms are added to a polynomial to increase its degree and more closely fit a curve to the data points. In trading, this is done by adding more rules or choosing more specific variable values. The more specific the model, the greater its historical performance and the more curve fit it becomes. Such models are less likely to perform well on future [unseen] data, though, which is why this poses a risk. In this context, “curve fit” is almost like a bad word.

Regardless of what development approach is implemented, I need to be careful not to overfit the model. An overfit strategy will be profitable on historical data while losing money when traded live.

As previously discussed, the signal is constant from some historical date through some future date. Understand the importance of the word some. A viable strategy may profit in live trading… until it doesn’t! Signals change and strategies break. We must constantly monitor our strategies in an attempt to turn them off before catastrophic loss results.

I’ll make one final comment on use of the word “strategy” versus “system.” I mostly agree with my comments from seven years ago. I generally think of “system” to mean the finished product. As I pointed out in the last paragraph, though, it’s never really finished since continuous monitoring should be done to detect if and when it breaks. In the linked post, I said vendors sell “strategies.” I would agree with this because while the developer may regard it as a finished product, for the customer it’s not. First, position size needs to be tailored to the individual using it. Second, each individual should ideally test it for confirmation (rather than taking someone else’s word that it is, in fact, viable). Third, time has passed since the precise completion date [which may not even be given]. We should therefore do further testing to make sure it isn’t broken.

* — I will discuss this in a future post.