Debugging the Missing Time Spread Backtrades (Part 3)

I will continue with the debug journey after explaining the complexity of date formats.

As discussed near the end of this post, the date field is in “number of days since Jan 1, 1970.” Although unreadable on its own, I keep this format to check the data file. This can also be subtracted to get difference in days.

While the Python in the link converts these dates to a common format, I have also needed to convert in reverse. I came up with the following for this purpose, which [I think] needs time and datetime modules to work:

      int( time.mktime ( datetime.datetime.strptime ( summary_results [ “Trade End” ].iloc [ -4 ], \
      “%Y-%m-%d” ) . timetuple ( ) ) / 86400 )

The code takes YYYY-mm-dd as input and outputs days since 1/1/70. I also started to log row numbers in btstats to better identify what rows are matching should I need to refer back to the data file.

I think I am better prepared to approach this debug now that I can convert date forward and backward.

Let’s return to the end of Part 2 where I discovered no apparent problem is being had with the number of matching options.

My next thought was that maybe the particular spread is not being selected from the encoded data. I therefore moved the debug line just below the block where short option variables get assigned:

     for i in range ( len ( dte_list ) ):
         if dte_list[ i ] >= ( 30 * mte ):

Again, dates_without_enough_matches is empty. So far, things are seeming to work just fine when I know they’re not.

I then added these lines near the top before the conditional block begins to determine control branch:

    if current_date != current_date_prev: #debug
        print ( f ‘ current_date is { current_date }, common date is { datetime.utcfromtimestamp ( current_date * \
        86400 ).strftime( “%Y-%m-%d” ) }, trade status is { trade_status }, and control flag is { control_flag } . ‘ )
        current_date_prev = current_date

The idea here is to print out a trace line every time current_date changes (and to update current_date_prev to current_date).

Take a close look at two anomalies that stick out in the following output segment:

First, I am not sure why I get the occasional blank line as indicated by the red arrow. Current date is always five digits, common date is always the same format, trade_status is at most nine characters, and control_flag is always 11 characters. I get roughly 3500 total trace lines (from 2007 through 2021). In case the trace may occasionally exceed one line in length (thereby showing up as a blank second line), I added ‘123456789’ to the end expecting that I might see many more blanks (or second lines with some of those extra digits). This has no effect; the output looks the same.

While the first anomaly remains unresolved, the second anomaly (yellow) is more relevant. Per this this fourth-to-last paragraph, I am getting some consecutive trace lines printing out with ‘INCEPTION,’ which may indicate missing trade dates. 26 days are skipped from 2010-04-08 to 2010-05-04, though, when I would expect a trace line for each date in the data file.

What is going on here?