Trading System Development 101 (Part 3)

Choice of subjective function is just the tip of the iceberg. As the last two posts have made clear, trading system development can be a very individual process.

Granularity of the variable grid is another important detail that has no right answer. A coarse (less granular) grid has fewer potential variable values whereas a fine (more granular) grid will have more. Testing from 10-30 by 10 gives me three values to test (10, 20, 30) whereas testing from 10-30 by 2 gives me 11 values to test. A more granular grid will result in more iterations (third-to-last paragraph here) and more processing time.

If I want to differentiate between spike and high plateau regions, then I really need increased granularity.* Recall the diagram in the last blog post linked. If one [variable] value tests well, then a high plateau region means an adjacent one should too. This makes more sense for 10 or 14 relative to 12 than it does for 10 or 30 relative to 20. Ten or 14 are only 17% away from 12 whereas 10 or 30 are 50% away from 20. Qualitatively, 17% away may be “similar” while 50% away is apples and oranges rendering exploration of the surrounding space impossible. Signal can fall right through the cracks with a coarse grid.

Feasibility testing is where I can tweak the strategy to get a good result. This is curve-fitting on a small percentage of the total data. If this results in an overfit model, then the strategy will be subsequently rejected when I test on a larger portion of data.

I found number of trades to be a recurring challenge with feasibility testing. Basic statistics (see third-to-last paragraph here) dictates preference of larger sample sizes when evaluating test results. I don’t feel confident in the outcome when one or two routine trade results can drastically skew the result. I’m not looking for an enormous number of trades since this is just a 2-year feasibility test, but if I get too few then I’ll be concerned. What minimum number to accept—all together now!—has no right answer.

I have a few ideas on how to increase number of trades. A strategy that closes or reverses direction when the opposite trigger occurs is fine as long as the triggers occur with some degree of regularity. If only 1-2 triggers occur per year then over two years I may get a tiny number of trades. I’ve tried implementing profit targets, stop losses, and m-day exits to increase number of trades. Feasibility testing is the time to do this; if I get a tiny sample size of trades over the full dataset then I must junk the strategy and move on to avoid curve fitting.

* — Of course, if subsequent steps in the development process don’t do this then why be concerned at all?