Does Technical Analysis Work? Here’s Proof! (Part 3)

Today I continue with commentary and analysis of Janny Kul’s TDS article with the same title.

I left off at the point where I think Kul’s article gets really special:

     > Now if by some miracle this does work, just to prove it was all one
     > big fluke, we should be able to roll forwards another 3 months to
     > produce positive P&L again.

The main takeaway from this article is right here. It’s one thing to train a model, which by definition is going to demonstrate good performance, and then follow through with more good performance. I became disillusioned when I was unable to accomplish this repeatedly, which is basically what walk-forward optimizmation does. I then became disillusioned again when I incorporated one additional incubation period as Kul mentions here. I wrote about OOS2 in the third bullet point of this post.

     > Note… the average across all these indicators across all instruments
     > is 0.095% per month so I think it’s reasonable to deduce that the
     > indicators used by themselves without any filtering have pretty much
     > zero Alpha…
     >
     > Now… we want to… run [the winners]… for 1 month forward… the
     > average of these is -1.92% for this 1 month period so if anything we
     > might be able to deduce that filtering positive indicators is actually
     > mean-reverting. Annualised performance here would be 23.0%.

I’m guessing he meant negative 23.0%, here.

In what follows, Kul falls apart a bit. I will do my best to tie things together:

     > If you look back to our train period MACD on Boeing stock… is the
     > best performing and here it’s the worst performing so instead of
     > filtering above 0 P&L we may actually find more Alpha filtering above
     > some +ve threshold (feel free to do this yourself!).

It’s not clear to me how he arrives at this conclusion. Maybe he’s saying not to take the best performers because they could subsequently revert and be the worst performers? Simply raising the threshold above zero would not resolve this, though.

     > The way we’d be able to deduce if this relationship holds is just to
     > roll our train/test period forwards one month and run again. If we do
     > this (i.e. use Feb 18 to Jul 18 as the train period and Aug 18 as the
     > test period) we get 15% annualised returns.

Initially, Kul trained on Jan 1, 2018, to June 30, 2018, and tested on July 1, 2018, to July 31, 2018. How does rolling forward one month start on Feb 18? It should be Feb 1 through July 31, 2018. Maybe he got the year confused with the date?

In addition to being uncertain about what dates he’s addressing, I also don’t know if the 15% annualized returns are positive or negative since he made that mistake just above. I’m a bit confused overall.

I will continue next time.