In the last couple of posts I’ve beat up on the idea of “sum of the trades” approach to system development, so today is the day we engage in a little self-deprecation. That’s right, no system is perfect, and I think it’s important to understand even your own warts.
The truth is, “sum of the trades” can be an effective way of demonstrating an edge on an individual trade, but a trading system is much more than that. It should include position sizing and portfolio management.
At CalculatedReturns we take this more holistic approach to evaluating performance. The results posted on the website are determined by simulating the correct position sizes within an overall portfolio. If the system generates a trade for a day, but the system is already fully invested, we don’t take that trade. This way we can a more “real world’ view of the system performance during back-testing.
That being said, there are still many reasons that the results of a trading system can’t be duplicated. If you are a subscriber to CalculatedReturns, or have been following along, you might wonder why your results don’t match the back-tested results. I've mentioned - several times - that actual results may vary significantly from the historical model. So, why is that?
I’ve traded this system for several years, and there is no question that my results don’t match exactly. One of the ways I monitor my execution is to run the model at the end of each month. Then I take the model trades (the ones produced by the back-tester for the past month) and compare them with my actual trades. Why wouldn't they match perfectly? Well, they don't always and there are a host of reasons... here are a few:
Commissions: I normally include commissions when I run my back-tests, particularly when I have a system at the final stages of development. My commission rate through Interactive Brokers is $0.005 per share (half penny per share). That's the rate I use, but you may have a completely different rate.
Sequencing: A few times of year something happens that almost assures variance from the model. Each day I scan for signals and enter orders accordingly. Normally, it's just a few orders and even if they all triggered - a rare occurrence - I would have enough equity in my account to cover the orders. On rare occasion, MANY signals are returned... sometimes 40 or 50 or more. When this happens it is likely that I would not have enough equity to cover all the trades, so I will buy the first few that trigger in the order that they trigger. The back-tested results may vary from this. The back-tester I use (AmiBroker) is looking at end-of-day data. So it doesn’t know the order in which trades trigger throughout the day. Thus, there is some variation.
Liquidity: Every once in a while there is a very fine line between reality and modeling (aka. backtesting). Trades that look like they should work in a backtest, don't quite work in the real world. For example, I might have an order to buy at a price... call it $20.00. The low of the day was indeed $20; right at the price of my limit order. The problem is that it was the extreme low for the day, didn't stay there long, and didn't trade much volume at that level. I had a similar circumstance about two weeks ago. My order triggered and partially filled, but that was it. The price never went back and I never got the complete fill. In backtesting it looks just fine, but in the real world it didn't work out that way.
Liquidity – Part 2: There is another aspect of the system which will also cause massive variance from the system results, but only over the long term. If you examine the chart in our results table with the month-by-month returns, or even the general performance data, you will see some amazing results. The MSv9 Trading System has averaged in excess of 200% returns over a 10 year period. Once you do that for a while your account gets so big, and position sizes are so big that they are no longer realistic. For example, recently I ran a test where the last trade of the test was a trade on IDIX. After 10 years of these amazing returns, the simulated account size was so huge that the 15% position size was $88 Billion. Oops! IDIX total market cap is only $1B. Simulation has it’s limitations and it’s good to know what they are. The flip-side is that this was a profitable trade. And if you were just starting out, that trade, and even that year’s results, could have been replicated. It just isn’t realistic over the long run once your account gets that large. I figure once mine is that large I’ll figure out another approach <grin>
Mistakes: While I make every effort to follow the trading parameters precisely, occasionally I just mess it up. Mistakes are tracked and measured to determine their impact, and how to avoid them in the future. Trading automation is employed for several steps of the trading process, which reduces many errors, but mistakes still happen. Each month I compare my actual trades to the trades of the model. Rarely are there variances, but in the event of a mistake it shows me exactly where I didn’t follow the model precisely. There are also mental mistakes that happen. I’ve steeled myself, to a degree, against these biases, but early on I second-guessed the system quite often. A few losing trades in a row will really test your resolve and stress the psychological aspects of the trader. Those mistakes usually cost money.
Generally, these are the four types of variances, that I see, and they do happen. Real world results don't match exactly, but they should always be directionally correct. Over time I’ve been able to minimize the variation and get results very close to the back-tested system.
One final episode coming next time, with a video of what we look for in trading system results.
Good Trading…
No comments:
Post a Comment