Walking it Forward with System Validation (Part 6)

My blog series “Lingering Quandaries about System Development” concluded by discussing a paradox with regard to Howard Bandy’s WFA discussion.  In http://www.optionfanatic.com/2013/02/07/lingering-quandaries-about-system-development-part-9/, I concluded by suggesting a new and improved WFA that can achieve system development goals.

I recommend not following Bandy’s advice to select periods for IS and OOS data early in the process and retaining them throughout development.  Besides effectively burying your head in the sand as described by Example 1 (http://www.optionfanatic.com/2013/01/29/walking-it-forward-with-system-validation-part-1/), you really can’t know what values may or may not work until you actually perform the WFA.  Validation is the final step of system development.

Rather, look to perform WFA by optimizing the periods and studying the entire parameter space.  Perhaps I will vary IS period from one year to three years by increments of two months.  Perhaps I will vary OOS period from one month to six months by increments of one month.  I then need to plot values of the subjective function in a three-dimensional space (or in two dimensions with color coding to represent subjective function ranges) to get a feel for where the high plateaus exist.  I should then select the time ratio to coincide with the middle of a high plateau and use trading parameters coincident with that combination.

Every time I do a WF iteration, I am selecting WF parameter values and subsequently trading parameter values.  The two differ, in effect, by an order of differentiation.  That is, the subjective function for the WF optimization coincides with a value for the concatenated equity curve to date.  The subjective function for the iteration coincides with a value for the equity curve of the preceding IS period only.

WFA serves to both validate a trading system and to direct trading at the right edge of a chart.