Trading System #3–Naked Puts (Part 4)

In http://www.optionfanatic.com/2012/12/10/trading-system-3-naked-puts-part-3/, I showed how DD can limit position size and annual return despite a high profit factor.  The next thing I want to do is conduct a thorough DD analysis.  I want to see the distribution of DDs to see if the max DD is typical.  I then want to look closer and see if I might be able to minimize DDs.

Before I expend too much energy, I need to boost the sample size to enable statistically valid conclusions.  These 113 wins and 16 losses (http://www.optionfanatic.com/2012/12/06/trading-system-3-naked-puts-part-1/ ) compose just one study.  I generated every data point by taking account values at Friday’s close (Thursday in case of holiday).  I could rerun the backtest and take data points on any other day of the week, which would generate five studies.  I could also run Monte Carlo analyses to randomize the trade order.  With hundreds to thousands of Monte Carlo simulations, I can get distributions of performance statistics and study values that encompass 95% of the runs to feel more confident about my conclusions.

I must also flush out the arbitrary and make sure these results weren’t simply a matter of luck.  Rather than only backtesting a weekly stop-loss of $30K and staying out of the until two positive weeks are seen, I should backtest a range of values (e.g. weekly stop-loss of $20K-$40K in increments of $5K; remaining on the sidelines until one, two, or three consecutive weeks of market gains have been printed).  I could study the subjective function (RAR/MDD) and look for plateau regions rather than spike regions across the graphs.

Another thing I might consider aside from using a weekly stop-loss could be a cumulative stop-loss.  That is, if DD reaches a certain value then exit whether it takes one week, two weeks, or more to hit that DD.

Trading System #2–Consecutive Directional Close (Part 11)

Back in http://www.optionfanatic.com/2012/10/31/trading-system-2-consecutive-directional-close-part-1, I stated the initial trading claims upon which the CDC trading system is based.  I will now assess those claims.

The claims are as follows:

> From 1995-2007, after the SPX has dropped three days in a row, it has risen more
> than 4 times its average weekly gain over the next five trading days.
>
> And, after the SPX has risen three days in a row, it has on average lost money over
> the next five trading days.

My backtesting dates are from 1/29/1993 – 9/30/2012.  I have included $8/trade commissions and delays for buy orders (i.e. buy at the next open).  The backtesting includes long trades only.

Here are the data for SPY:

Here are the data for QQQ:

Here are the data for IWM:

The first claim is true.  After falling on three consecutive days, the index rose over four, 22, and five times its average weekly gain for SPY, QQQ, and IWM respectively.

The second claim is also true:  after rising on three consecutive days, SPY lost money on average over the next five days.  This was also the case for IWM.

Truth in Backtesting (Part 5)

I am discussing different ways of representing trading rules to understand the truth in backtesting:  what allows for consistency with live trading and what does not.

Trading rule #1 was discussed in http://www.optionfanatic.com/2012/12/3/truth-in-backtesting-part-4.

Trading rule #2 is to buy at the close if today’s close is greater than yesterday’s 20-MA.  Yesterday’s 20-MA is known after yesterday’s close but today’s close will not be known until after today’s close.  If price heading into the close is near yesterday’s 20-MA then tomorrow’s open would be the soonest I could execute this trade with certainty.

Trading rule #3 is to buy at the close if today’s open is greater than yesterday’s 20-MA.  Both yesterday’s 20-MA and today’s open are known by today’s close.  No problems here.

My observation has been that most people who backtest trading rules write code to match trading rule #1.  This applies to crossovers of indicators such as Bollinger Bands, MACD, or RSI.  This also applies to crossovers of indicators above or below a critical value.  In all cases, I truly do not know indicator values or the closing price until after the close, which leaves great opportunity to trade when I shouldn’t have (or vice versa) if I trade without delays.  This will result in performance departure from backtesting results if the backtesting was not performed correctly.

Trading rule #1 is easiest to write and seems to scream loud and clear “buy on a close above the moving average,” but once again I need to stop and analyze at the moment of execution to see if it makes sense.  In this case it does not.  Taking the easy way out is never in my best interest (see http://www.optionfanatic.com/2012/10/18/laziness-dissected).  The only one who stands to be hurt is me when I implement a trading system that looked great in backtesting only to realize much fewer profits or significant losses when I start trading it live.

Truth in Backtesting (Part 4)

In http://www.optionfanatic.com/2012/11/30/truth-in-backtesting-part-3, I finished discussing Market on Close orders and raised the possibility that execution at the closing price is doubtful if not impossible to achieve on a routine basis.  This may artificially skew backtesting results to the upside.  Today, I want to discuss the truth about end-of-day (EOD) trade triggers.

EOD backtesting typically uses daily or weekly time frames with each bar having an open, close, high, and low price.  For daily studies, the vast majority of backtesting computes trades based on the day’s close.

Consider a basic moving average crossover system where a long trade is executed or held if the close is greater than the 20-day moving average (20-MA).  I am going to ask a question that may itself seem very shocking:  is trading such a system even possible?

The trading rule can take three forms.  Trading rule #1:  buy at the close if today’s close is greater than today’s 20-MA.  This is riddled with uncertainty of the sort described in my last post.  Today’s closing price is not known until after the close.  Today’s 20-MA is not known until after the close because today’s closing price is used in the calculation.  I can project what today’s 20-MA will be depending on closing price and monitor price as the close approaches.  In some cases it might be clear whether a trade should be made.  In other cases it may be impossible to make this determination if price is near the projected 20-MA and the closing ticks of the day become decisive.  The soonest I could execute this trade with certainty would then be at the next open.

I will discuss other forms of this trading rule in my next post.

Truth in Backtesting (Part 3)

In http://www.optionfanatic.com/2012/11/29/truth-in-backtesting-part-2/, I described how market on close (MOC) orders will likely not be executed at the closing price.  This presents a challenge to the trade like you backtest, or “truth in backtesting” mandate.

As an alternative to using MOC orders, I might be better off manually submitting a market order just before the close.  I believe only live trading experience of placing orders x seconds before the close for different values of x can give me a true idea about resultant slippage off the closing print.  In theory, the closer to 4:00 PM ET I place an order, the smaller the closing range relative to MOC closing range, which remains constant.  I must understand the risk in trying to execute closer to 4:00 PM ET as the greater possibility of not being filled at all due to Internet blips/delays or phone tie-ups, however.  Missing a trade altogether, which in trading jargon is called an “unable,” and having to execute at the next market open could result in a much different trade price especially if the market gaps.

Application of supply/demand logic corroborates the proposition that execution at the actual closing price is uncertain.  If a large number of people were trading my system and buy (sell) orders were submitted near the close in heavy volume then the stock price would move higher (lower) with speed.  This would make execution at the closing price even less likely unless the order were submitted proportionally closer to 4:00 PM ET.  Because the increased risk of unables is very difficult, if not impossible to quantify in monetary terms, I might have to accept a minimum amount of slippage at a latest possible order submission time and incorporate that into the backtesting model.

I will introduce and discuss more truth in backtesting with my next post.

Truth in Backtesting (Part 2)

Backtesting must be done the same way that live trades are executed to prevent results from being contaminated with immeasurable error due to differences in trading method.  The question at hand addresses the implicit assumption with end-of-day (EOD) backtesting that transactions will take place at the closing print.  A detailed look will make it evident that getting the closing price in live trading is at best more complicated than one might think and at worst not even realistic.

Market on Close orders would seem like the obvious choice to trade based on EOD backtesting.  As explained on the Interactive Brokers web site:

> [Suppose] You want to buy 100 shares… and decide that the closing price…
> has… proven to be the best price of the day. Create a BUY order, then select
> MOC… to specify a market-on-close order… You must submit the MOC order
> at least 15 minutes prior to the close. The order will be submitted at the close
> as a market order.

A significant caveat to MOC orders is explained on the Orion Futures web site:

> …this order will be executed on the market close. The fill price will be within
> the closing range which may in some markets be substantially
> different from the settlement price. [emphasis mine].

With a MOC order, I don’t know how far off the closing price I will be filled!  Only through experimentation with different securities and different lot sizes can I really get a feel for how big the “closing range” is.  For all I know, getting filled at the closing price is a rarity.  As a prudent system developer, I should ultimately include the average deviation between closing price and actual fill as slippage into my backtesting because slippage can substantially affect the equity curve.

In my next post, I will discuss an alternative to MOC order placement.

Truth in Backtesting (Part 1)

I came across an advertisement for a trading service that provides a good educational point about backtesting.  The lesson to learn is that I must always attempt to trade like I backtest.  This is the “truth in backtesting” mantra, and if this is not possible then I can expect my live trading performance to be much worse than backtested models.

The first thing to understand is that I should always expect live trading performance to be somewhat worse than backtesting would suggest.  This can occur for many different reasons:  transaction fees (slippage), curve-fitting, loss of system effectiveness over time, etc.   Even if I can steer clear of any unintended factors that might exaggerate backtesting results, I should still expect the live trading results to disappoint.

This also suggests I should avoid any system that generates marginal profit in backtesting.  These systems may not profit at all in live trading.

One common way to generate exaggerated backtesting results is to couple trade signals and trade execution.  The advertisement mentioned above is for a trading system package that allegedly performed as follows over the last three years:

Each row represents performance of a different trading system (10 total).  Trades are triggered by end-of-day signals.  The first set of numbers shows performance when buy/sell signals are taken at the very same close.  The second set of numbers shows performance when buy/sell signals are taken at the following open.

Is it realistic to couple trading signals with immediate trade execution at the closing price as shown in the first set of data, above?

I will continue this discussion in the next post.

Trading System #2–Consecutive Directional Close (Part 8)

In http://www.optionfanatic.com/2012/11/20/trading-system-2-consecutive-directional-close-part-7, I found the CDC trading system to perform better without trade delays.  Today I want to put this conclusion into perspective.

Even with trade delays, the CDC system is worth pursuing.  A mean RAR/MDD of 16.50 and mean profit factor (PF) of 1.90 are worthy enough statistics to shake a stick at.

Furthermore, while it may be necessary in some cases to trade at the next open, in other cases no trade delay will be needed.  As an example, consider the CDC system with x = 4 where four consecutive up or down closes triggers a trade.  If SPY has closed higher each of the last three days and is currently up 3% with just minutes left in the trading session then I can reasonably proceed with a trade.  SPY would have to catastrophically tank in the remaining time to close lower and I know in my experience of watching this market that such an occurrence would truly be a Black Swan.  On the other hand, if the market has been choppy all day moving back and forth between positive and negative territory then heading into the close I truly may have no clue whether the final print will be up or down.  In this case I will be forced to wait until the following open to trade.

As mentioned above, the CDC system is worth pursuing even with trade delays.  At least sometimes if not often, however, trade signals will be clear heading into the close and no delay will be needed.  I can therefore interpret these backtesting results to be better than the minimal, yet acceptable, performance numbers mentioned above (i.e. RAR/MDD > 16.50 and PF > 1.90).