Tasty Statistics

In December 2015, I posted on statistics and trading. In March 2017, I critiqued Tasty Trade’s (TT) “Market Measures” (MM) segment for omitting critical information. Today I present an August 2015 e-mail correspondence I had with the TT research team about a failure to include statistics when presenting backtesting results.

My initial e-mail was as follows:

     > I’m a numbers guy, which is one reason I love the
     > MM segment. One thing I would like to see are tests
     > of statistical significance.
     >
     > For example, in a screenshot from 8/12/14, the
     > takeaway is printed on the slide (16% and 3%). My
     > training suggests if these numbers aren’t statistically
     > significant then they aren’t very meaningful. I’d like
     > to see how significant they are (e.g. p-value).

They responded the next day:

     > I completely understand what you mean about adding
     > statistical significance to studies… this is something
     > we are trying to work into future studies. We have
     > several members of our team who are very capable and
     > have a strong history with… statistical analysis. I believe
     > the major positive to including these numbers would be…
     > more validity to our methodologies and studies… As for
     > a potential negative, our main concern… would be
     > barriers to entry. We want our MM and other segments
     > to be accessible and, while complex, understandable to
     > new and seasoned traders…
     >
     > Thank you so much for the feedback!

I responded a few days later:

     > [With regard to the potential negative, you have] a
     > very legitimate point. Statistics is difficult for many.
     > I believe it provides essential information for those who
     > understand, though. When a study compares two
     > results, one is almost always going to be greater
     > than the other… without the p-value, it’s impossible
     > to know whether the difference is meaningful. Small
     > mean differences and large standard deviations are not
     > significant and only statistics can show us that.
     >
     > I totally get what you’re saying about being audience
     > friendly… it doesn’t take much statistical discussion
     > to get many people to gloss over and tune out.
     >
     > At the same time though, consider Tom Preston’s
     > recent praise of work done by the research team:
     >
         > When they put results out there, we can
         > stand by them both from a data accuracy
         > and conceptual point of view.
     >
     > Can you without providing the statistics? Omitting
     > statistics misrepresents the reality, in some cases,
     > by suggesting meaningful differences exist when in
     > fact they may not.
     >
     > Scientific analysis can always be scrutinized. I
     > believe statistics should at least be presented.
     > Omitting them may substantially undermine the Tasty
     > Trade mission (to combat misinformation and lack of
     > information provided by the financial industry), which
     > Tom Sosnoff literally pounds the table to support.

Their final response:

     > Thanks again for the feedback.
     >
     > Once again, I 100% agree with you that adding hard-
     > hitting statistical numbers will add to studies for those
     > who understand them. We are trying to implement this
     > going forward… I have passed your ideas onto the team
     > and emphasized that there are viewers… that want to
     > see this kind of analysis.

I felt that was a promising e-mail exchange.

Nearly 30 months have now passed and I have yet to see a p-value reported on MM, though. I have seen every single episode.

I had one other controversial idea not shared in the e-mail:

     > Maybe people who don’t or can’t understand statistics
     > should not be trading. Or perhaps part-time trading in
     > in small/hobby size is okay as opposed to trading full-
     > time as a business.

I did mention critical thinking and statistical background in my post on prerequisites for trading as a business.