Lingering Quandaries about System Development (Part 2)

This series details different aspects of nebulous understanding that prevent my advancement along the System Development learning curve.  In http://www.optionfanatic.com/2013/01/18/lingering-quandries-about-system-development-part-1/,  I began by describing confusion over the chosen subjective function RAR/MDD.

As mentioned, one possibility is to select a different subjective function and for me, Profit Factor (PF) leads the pack of contenders.  PF describes the number of dollars made per $1 lost.  This does not vary by number of trades or position size–two factors that affect exposure and therefore affect RAR/MDD.  With PF, I can study one ticker or multiple tickers and generate comparable statistics regardless of initial equity and position sizing.

One downside to PF is that it does not account for distribution of profits and losses.  Having one big winner among many fractional profits could generate a large PF where the equity curve would tell a different story.  This argues for inclusion of standard deviation (SD) to the subjective function.

Another downside to PF is disregard for DD.  For example, AAPL had a 50% return from 2008 to 2011, but there was also a 60% drawdown in the middle.  While this spanned several years, it could certainly occur over any time frame and would be more than I could tolerate despite a decent overall PF.  Perhaps some metric like PF / (MDD * SD) would be useful.

Another possibility is to stick with RAR/MDD and backtest just one ticker at a time.  The cumulative effect of trading multiple tickers would likely be a lower RAR/MDD than reflected by any single ticker alone but that is unimportant because consistent comparisons would be achieved.  Besides, whether by different tickers or different systems, I want multiple trades with the potential of concentrated profit (large RAR/MDD) on each one.

I believe the bigger issues with multiple tickers have to do with backtesting validity, portfolio heat, and what exactly it takes in order to be a robust finding.

To be continued…