Walking it Forward with System Validation (Part 5)

In http://www.optionfanatic.com/2013/02/01/walking-it-forward-with-system-validation-part-4/, I introduced the process of Walk-Forward Analysis (WFA).

The pictorial representation of the WFA process for Example #4 is as follows:

Note how WFA achieves 13 full years of OOS testing and validation.

Howard Bandy considers WFA to be the gold standard of trading system validation, and I see multiple reasons to support his claim.  First, the largest risk for curve-fitting has been eliminated by using OOS data to test a system developed using IS data.  To find the best combination of trading parameters and then to advertise said system to potential customers is unrealistic at best and criminal at worst.  Second, with WFA my system will better adapt to changes in market behavior over time.  Change in market behavior is responsible for systems working well until they don’t.  WFA provides the means to adapt.  Provided my OOS data is extensive enough to sample all market environments, a WFA equity curve sufficient to meet my personal criteria (i.e. subjective function) should give me the confidence necessary to trade the system live.  WFA is not just a nifty backtesting tool; it offers a process that may be done at any time to resync trading parameters with recent market activity.

In this blog series, I have studied four general approaches to system development.  In Example #1, I backtested one set of system parameters to trade live if results impressed.  In Example #2, I optimized a trading system over historical data to subsequently trade live.  In Example #3, I optimized a trading system over 13 of 15 years of historical data and used the last two years to validate the system.  In Example #4, I used WFA to generate 13 full years of OOS validation on a trading system that periodically aligns itself to recent market activity.

Because the system parameters may adapt, WFA results in a dynamic trading system that is qualitatively different from what most people conceptualize when discussing system development.

This writer believes it makes a lot of sense.