Mining for Trading Strategies (Part 5)

Today I want to present and discuss results from my initial testing of 2-rule versus 4-rule strategies.

The methodology is given in the third-to-last paragraph here.

I basically developed the strategies with random entry signals and simple exits from 2011-2015 and tested OOS from 2007-2011. I sorted by best performers over the entire 8-year period. I took the top (+/-) 60 (two simulations of ~30 each) and tested from 2015-2019. As the focus of my study, I then tested these strategies from 2015-2019 (incubation).

I applied $25 plus $3.50 / contract-trade to account for transaction fees.

The analysis consisted of T-tests assuming unequal variance in Microsoft Excel. I chose unequal because I had no reason to assume equal variance. I tested on five parameters: PNL, PNLDD, number of trades during incubation, average trade, and PF. Number of trades is not a performance metric and could probably have been left out.

Since I did five statistical comparisons, I adjusted the significance value from 0.05 per Holm Method. I think Holm is better than Bonferroni in this instance because the hypotheses are not independent: all measures of four rules versus two.

Here are the results:

No differences were significant after application of the Holm method.

As I reviewed the study, I found that I made a few mistakes. First, the starting date was 2001 instead of 2007. Second, I had OOS set to 40% of the data instead of 50%. Third, I had cash set to be invested in the Biotech index instead of nothing. The latter probably affected strategies across the board and is nothing to be concerned about. It may have differentially affected the groups if one tended toward more/less time in the market during a period when Biotechs performed significantly better/worse than large-cap US equities.

I would rather do the study correctly before making too much of it. I will look to build on this with later studies.