Questions about CMLViz Trade Machine (Part 3)

I recently viewed a 2019 presentation from Ophir Gottlieb on the CMLViz Trade Machine PRO (TM) automated backtester. Continuing from last time, this is a laundry list of questions, suggestions, and discussion ultimately aimed toward getting clarification about what TM offers and what its limitations are.

I like that TM displays risk on the backtested trades. “Risked” is clickable (look back to the results screenshot in Part 2), and I’ll be curious to see exactly what gets displayed. I want to know the most relevant calculation of “max capital required,” “cost,” or “max risk.” Transparency is important because this will be the basis for total return percentage.

I like that we can select particular start/end dates for the backtest in addition to canned time intervals. I would also like the ability to compare results from the same strategy over different time intervals or different [tweaked] strategy over the same time interval side-by-side on the screen. This was done in the presentation, but I do not think it’s an actual feature.

I appreciate the fact that individual trade results may be viewed and exported to .csv.

I wondered if “pre-earnings” trade could be combined with other option trades like calendar, IC, [long/short] straddle, etc. The answer appears to be yes. Delta values may be specified for different legs and I should be able to incorporate directional biases by setting delta values accordingly. Unfortunately, I did not see the ability to customize percentages (ITM or OTM).

I would like to be able to focus on a particular backtrade and view the option chain as well. This is a premium product and as such, I would hope the data are clean. However, I’ve seen other premium products fail to deliver (see this about OptionNet Explorer). I want to verify option chains are complete and consistent with regard to strike arbitrage, etc.

I like that a low-liquidity warning is displayed when a TM backtest encounters such options; are we able to filter by open interest and/or option volume relative to some customizable average?

Here are some other questions about TM that I was not able to glean from the presentation:

• Can we close trades at X DTE or after Y DIT?
• Can we manage winners at Z% of initial credit or of max potential profit?
• What settings are available for studying 0-4 DTE (Weeklys) trades?
• Can we set max loss exits in terms of delta [relative to initial]?
• Can we apply a customizable IV rank filter?
• Can we apply a customizable VIX filter?

It appears the first bullet above is available as “days.” I would also like to be able to set as a percentage of original DTE for daily backtests or trades triggered by technical criteria (that may occur regardless of DTE).

Daily backtesting is the ability to open a new overlapping position every trading day. This maximizes number of occurrences and enhances trade statistics at the cost of [a] realistic equity curve [statistics]. I addressed serial and overlapping/daily backtesting approaches here and here. For me, lack of this feature would be a major disadvantage.

I did not see the ability to backtest multi-component positions. An example of this is a condor with different entry criteria for the put credit spread vs. the put debit spread, different wing widths for the two, maybe something like a call butterfly and/or another long option added as a hedge, etc. Each component may have its own set of entry/exit criteria but the whole position is maintained or closed based on the net PnL of all components or position net greeks.

Finally, are we able to backtest trades with adjustments? I did not see any examples of this, either.

TM offers a one-month free trial. If the trial includes full functionality, then I can easily answer all these questions.

Questions about CMLViz Trade Machine (Part 2)

Continuing from last time, this is a laundry list of questions, suggestions, and discussion ultimately aimed toward getting clarification about what the CMLViz Trade Machine PRO (TM) offers and what its limitations are.

In the 2019 presentation mentioned in Part 1, Ophir Gottlieb’s response to a question about forward testing is a curve-fitting alert. Someone asked if forward testing could be done and he responded:

> What is forward testing? If I knew the future then I’d certainly trade it (followed by laughter).

Forward testing is a form of OOS validation I described in paragraphs 2-3 here. One of my concerns about Gottlieb’s examples is the possibility of curve fitting to generate overly optimistic results. Remember: this is still a sales presentation.

At one point, Gottlieb presents the table to follow. Is it susceptible to curve fitting?

Cherry picking this model over many others that did not show good performance would be duplicitous. I would like to know how many moving average (MA) models did not perform well. To screen out a spike in favor of plateau region (see last two paragraphs here, I would also like to know how MA models with similar parameter values perform.

Curve-fitting caution aside, the table suggests this model works better than baseline in most periods except recent history, which itself was still better than average baseline. I would like to see numbers of trades and, ideally, inferential statistics with correction for multiple comparisons (sixth paragraph here).

The ability to backtest strategies on S&P 500 stocks and to count how many meet customizable performance criteria would be interesting. This might be data mining, though, and unlikely to reveal persistent edges (it usually doesn’t).

Getting back to TM, I would like to be able to customize the trade statistics from a wider variety of potential calculations. What I saw in terms of output is not impressive:

Thinking off the top of my head, I would like to be able to include/exclude:

• Distribution of losses
• Maximum drawdown (MDD)
• Biggest/smallest winner/loser
• Max consecutive winners/losers
• Average days in trade
• Standard deviation (SD) for all, winning, and losing trades
• Average trade
• Average win : average loss ratio
• Profit factor
• Compound annualized growth rate
• PnL/contract-day
• Exposure %
• Risk-adjusted return (perhaps by MDD, SD, or exposure)

A complete listing of potential trade statistics is given in Part 1 and Part 2 of my Automated Backtester Research Plan.

I don’t think it should take a whole lot of programming to make a plethora of trade statistics available. Make us happy and let us play. Too much can never be offered in this area. I would say the same for “export to .csv” buttons: put them anywhere TM generates output because you never know when someone will want to do further processing (e.g. with spreadsheets).

I will finish up next time.

Questions about CMLViz Trade Machine (Part 1)

I recently saw a 2019 presentation on CMLViz Trade Machine PRO (TM) given by Ophir Gottlieb. The presentation is compelling, but I am not yet sold on this as my first automated backtester. Please excuse me in advance for the laundry list of questions, suggestions, and discussion that is to follow. More than anything else, I just want clarification about what the product offers and what its limitations are.

The presentation is ~75 minutes long and filled with examples that answer many questions. The backtest results are impressive, but I know better by now than to take impressive results at face value without establishing proper validation.

TM comes with a few technical analysis (TA) filters built-in. One viewer asked if this would be expanded. Gottlieb said the reason more TA indicators have not been added is because additional edge has not been found. While he will be looking to add, his criteria to do so are stringent. I found this answer to be very consistent with some of my writing on the subject.

As Gottlieb presents one technical setup I find very encouraging, I wondered whether customization is possible as discussed in the fifth paragraph here. A later screenshot shows this to be the case. Unfortunately, I don’t see that TA can be used for exits.

I wondered whether slippage and commissions are customizable. Slippage has three settings [none, market (natural pricing), or halfway between the two], which is acceptable. Ideally, I would like to also be able to customize slippage as a percentage and/or have dynamic slippage that increases with market speed. Commissions are customizable.

Although some examples showed impressive results, it’s unclear what the monthly fee includes. TM and CML Stock Research PRO are two separate products. The former includes some scans and write-ups, but the latter offers much more research. Gottlieb seems to have taken ideas from the latter as backtesting examples for this presentation, which deceptively makes TM seem more powerful than it actually is. I personally think Research PRO should be included for as much as TM costs.

Small sample sizes in the presentation limit generalizability of results that would otherwise be more meaningful. Many backtests were single stocks with few trades. No backtest of a single stock earnings trade over the last 2-3 years will ever qualify as a large sample size. Filtering entries to further cut down the number of trades is less than impressive.

Whether I can backtest the same strategy on a basket of stocks to increase sample size is discussed here. Part of me believes different people trade different stocks and drivers of one stock will not be drivers of another. Then again, perhaps the biggest money managers trade them all. Alternatively, if all stocks are in the S&P 500, then perhaps all overriding stock movements are dictated by traders of the index in the form of /ES or other tracking ETFs. Avoiding assumption is probably best practice.

I will continue next time.

0 DTE Iron Condors (Part 3)

I want to spend some time reviewing the second-to-last bullet point from Part 2.

As a future direction for research, I suggested comparing mean realized loss in excess of stop-loss trigger on either side to study directional edge on a more granular level. What exactly do I mean by that?

I wrote a blog post in 2017 that spells out many of the points I wish to make here.

When done properly, I believe backtesting can provide a good idea how a strategy will perform in real-time. Particular attention must be paid to ensure the plan can be backtested accurately (this blog mini-series explores such facets). A big limitation is data granularity. OptionNet Explorer (ONE) is good to offer 5-minute intraday market data for the last several years, but in live trading stop orders are monitored much more frequently (e.g. every three seconds).

The 0 DTE trade presentation suggested buy stop-market or buy stop-limit orders to mange risk. Once the stop is triggered, the stop-market order will be filled at the next available price. Since I may not like the fill if the market gaps big at that point, a buy stop-limit order may be used to avoid high prices. This becomes a limit order when triggered but will not fill if price gaps beyond the limit value. For example, “buy at 3.00 stop limit 3.30” will be triggered when price hits or exceeds 3.00 but will not offer more than 3.30. If the price sequence is “2.90, 2.95, 2.98, 3.50,” then this stop-limit order will not fill.

ONE does not provide intrabar data, which would be necessary to determine if/when during the period a stop order would get filled. We can see the open, high, low, and close for each period, but we don’t know the sequence of prices in between. We must therefore look at the open or close (be consistent) to make that determination. As five minutes is 100x longer than three seconds, results from backtesting may differ significantly from live trading where stops are used because the market has much more time to run past the stop.

Whether such deviation is good or bad is discussed in the blog post linked above. Today I just highlight the difference.

If I am developing a strategy through backtesting, then I must trade like I backtest to get similar results: 5-minute monitoring of opens/closes and no stop-market/stop-limit orders.

From the top I mentioned the suggestion to compare summed excess losses between put and call sides. Over six months, I found a roughly equal number of losses on each side. If one side therefore generates a much larger excess loss than the other, then perhaps that side would make for a better trade in real-time since stops would mitigate the excess losses.

I now retract this suggestion for two reasons. First, I do not have a PnL comparison between call/put credit spreads since I backtested both together as IC’s. Second, if I don’t trade like I backtest then even comparing excess loss in a credit-spread backtest will not be meaningful because additional trades stopped out in live trading will not be reflected.

0 DTE Iron Condors (Part 2)

Last time, I detailed backtesting for an intraday option strategy I recently viewed online. Today I will discuss results.

The initial backtest covers every trading day from 12/30/2020 through 5/10/2021: 89 trades total. This is a relatively small sample size studied over a limited time interval.

Over this period, the average trade is -$5.90 with a standard deviation of $203. The largest winner (loser) is $338 (-$679). 62% of trades are winners, but the average win (loss) is $134 (-$232). The maximum number of consecutive wins (losses) is 7 (5). Profit factor (PF) clocks in at 0.93.

An inauspicious beginning…

Remember from Part 1 that 0 DTE trades are different from 1 DTE trades. 1 DTE trades take place on Tuesdays and Thursdays. Despite the theoretical consideration given last time, one thing we know for sure is they cannot experience near-100% decay as intraday trades. Comparing the two:

With three days per week instead of two, the number of trades is expectedly larger for 0 DTE. While winning percentage is about the same, average trade and PF both show 0 DTE trades to be profitable whereas 1 DTE trades are not. With a larger sample size, I would perform hypothesis testing to verify statistical significance.

While 0 DTE trades seem to be better than 1 DTE, a PF of 1.13 implies only the slightest of edge. Kudos to me for including (some would say liberally) transaction fees (see third-to-last paragraph here), but average trade (just slightly ahead of commissions + exchange fees) is still close to a serious debacle being barely positive over a short period.

Although this is an IC backtest, I can still extract some information about vertical spreads.

Vertical spreads have a greater probability of profit than iron condors (IC). Theoretically at expiration, a 10-delta IC has ~80% probability of profit while a 10-delta put or call credit spread has ~90% probability of profit. As a potential tradeoff, the credit received for constant-delta verticals will be less because only one side is sold. Risk will be roughly equal since spread width is determined the same way for verticals and ICs.

I would expect more trades to be stopped out on the call than put side for two reasons. First, the market generally has an upward bias. Second, same-delta options are NTM in terms of points (or percentage of underlying) on the call versus put side as a result of vertical skew. Altogether, 16 (18) trades are stopped out on the put (call) sides. For 0 DTE, nine (11) trades are stopped out on the put (call) sides. For 1 DTE, seven trades each are stopped out on either side.

Despite a small sample size, these are balanced distributions offering no hint that trading just the puts or calls should be more advantageous than the other.

And based on PF, it’s hard for me to be overly encouraged by any of these short-term trades.

Potential directions for further study include:

• Selling farther OTM (e.g. NTM option with delta 5 or less) to decrease frequency of stop-loss (SL)
• Selling farthest OTM over threshold premium to normalize influence of any single trade on the average trade
• Selling naked shorts rather than spreads, which limit margin exposure at the cost of long-option premium
• Varying the SL trigger from 3x to 4x, 5x, 2x, or 1.5x
• Comparing mean realized loss in excess of SL trigger on either side to study directional edge on a more granular level
• Backtesting a longer time interval and/or different market environments
Categories: Backtesting | Comments (0) | Permalink

0 DTE Iron Condors (Part 1)

I recently viewed a webinar on 0 DTE credit spreads and iron condors (IC) that I found quite enticing. Today I will present my backtesting approach to studying this strategy.

Why did the presentation pique my interest?

• 75-85% winners
• Available discretion with regard to stop-loss and exit (mechanical or mental)
• Tradeable mechanically from the [near] open or using discretion to determine market direction before selecting side(s)
• Subscription software available for $250/month providing built-in indicators capable of improving success rate to 90-95%

Promises of your run-of-the-mill Holy Grail, pretty much: gently disguised, perhaps.

I conducted the backtest in OptionNet Explorer (ONE) as follows:

• At 9:40 AM, open one contract IC in front expiration (monthly expiration on Th with 1 DTE else Weeklys 0-1 DTE).
• Sell short put at 5-10 delta to receive at least $0.75 – $0.90 (opt for closer to 10 delta on Tu/Th).
• Sell short call at 5-10 delta to receive at least $0.60 (opt for closer to 10 delta on Tu/Th).
• Buy long put and long call at the NTM strike available for $0.05 or 50 points away from short strike (use NTM strike when multiple options are displayed with identical premium).
• For opening trade, assess transaction fee of $11 for premium approaching/exceeding $1.00 or $6 otherwise.
• If intraday distribution of PnL (graphically displayed in ONE!) mostly in the center then allow to expire worthless unless check of intraday high or low (pne click each in ONE) reveals breach of max loss threshold.
• Max loss for a vertical spread is 2x initial credit received for the short option: short option premium * 3.
• At the earliest 5-minute increment when premium exceeds net loss, close the distressed vertical spread leaving long option only if priced at $0.05 or less.
• For closing trades, assess transaction fee as follows: $21 for losers priced over $5.00 in a fast-moving market, $11 when priced between $0.80 – $5.00, or $6 otherwise.
• Any remaining options DOTM at 3:55 PM may be allowed to expire worthless by closing for zero cost.*
• BTC any remaining options at 3:55 PM per transaction fee guidelines described.

I try to follow presented guidelines presented, but some details are left vague. 5-10 delta is vague (the presentation mentioned “under 5 delta” or “under 10 delta”). Defining transaction cost to account for slippage is vague (but present, which I don’t believe was the case in the presentation. See second paragraph here). 0-1 DTE is vague compared to only 0 DTE. If the trade is robust and not just good marketing, then these specifics should not matter over a large sample size of trades.

The biggest deviation from presented guidelines is inclusion of Tu/Th trades. The presentation focused on 0 DTE. 100% decay is only possible for these. The Tu/Th trades decay up to ~50-70% even when DOTM with 1 DTE. I therefore tried to get a bit more premium by opening these trades closer to 10 delta. The risk to selling NTM is a lower winning percentage, but shouldn’t that be offset by the additional DTE to keep shorts farther OTM? All trades lasted exactly one day, which in theory should normalize magnitude of market movement. Whether a better control exists may be an empirical question, but always remember past performance is no guarantee of future results.

I will continue next time.

* — An Expire Options button to do so with two clicks (including confirmation) would be a nice software addition.