Quantitative Trading: Part 7 - Analyzing Results

I hope I haven't overly simplified this area of quantitative analysis in trading. It's a topic that is extensive, detailed, and frankly one that I'm probably not qualified to treat in depth. What I do know is what works for me, and sometimes what doesn't work, primarily learned over many years of experience and trial-and-error. I say all this because I've moved quite quickly and in seven very short articles am at the point of looking at results... and here we go...

Results of quantifying a trading strategy can be measured in many ways. In fact, that's the first question you have to answer when thinking about testing. What exactly are you looking for? How do you decide what a good result is? It's very tempting - especially when we start drooling over all the big money we can make - to simply go for the trading system with the highest return. In real life, however, that's not always the easiest system to trade without losing a lot of sleep. There are many ways to measure performance, so it's up to the system developer to decide what characteristics you are looking for in a trading system.

You might remember that my system development philosophy began with objectives for my system. Specifically, there were two:

1. "We're here to make money"; and
2. "I never want to be more than a few days away from an all time high in my account"

In other words, what I am looking for in system results is consistent profits, and quick recoveries from drawdowns. I might sacrifice large trends and HUGE profits, in return for consistently above average profits. It's important to figure this out in order to analyze results efficiently. What are you looking for? Pure Growth Rate, or growth rate that is adjusted for risk. I tend to gravitate toward metrics that reflect growth, but penalize large drawdowns and volatility. Sharpe Ratio is a good measure for this, but it also has to be balanced with CAGR. It's possible to have a very high Sharpe Ratio without a big return.

Amibroker has a couple of nice metrics in their default reports. One is CAR/MDD... that's Compounded Annual Rate divided by Maximum DrawDown. That's a metric that benefits from larger returns and penalizes drawdowns. Another metric you might like is K-ratio, which is a metric that reflects consistency in the equity curve. In other words, if you have big drawdowns, your equity curve consistency will be lower and the K-ratio will go down.

There are many different performance metrics to evaluate trading systems, and each one is a little different. There are many with high correlations, but in the end what is right for you depends on what your personal trading objectives are. Generally speaking, I use Sharpe Ratio for optimization and testing, while keeping an eye on CAR to make sure I have adequate overall return.

I'll leave you with a snapshot of the Amibroker backtesting for one of my systems. This is a long/short system, which trades highly liquid stocks. The test you are seeing here is from the start of 2002 to the end of 2003 - a period of two years. The SP500 started and ended this period just about in the same spot (around 1150). It saw a strong bear trend followed by a strong bull trend. This is a good example of testing across long periods to take in many different market types. As you can see, over that time there were 791 trades.

I don't pay a lot of attention to the 276.46% return, as there are many reasons this isn't attainable, but I pay a LOT of attention to the average return per trade... in this case 3.45%. That number includes all wins and losses and shows me that I have a nice edge on every trade. I also like to take a look at the exposure and the Max. system % drawdown.

Exposure tells me how much of the time I have money committed to the market. With this system exposed at only 14.62%, there might be some things I could do the other 85% of the time to make more money elsewhere. The drawdown number shows me what the worst dip in my account was during this time. I have to be honest at this point and decide if I can stomach that drawdown. 11% would be ugly, but I can probably handle it without abandoning ship.

There are many more statistics here, so take a look. And of course, you can always right custom code to insert your own or other metrics that might be more meaningful to you. As I mentioned, I like to watch the Sharpe. Anything above 3 is in my comfort zone. For many people 1+ is not bad, but I have many systems over 3 so I usually set that as a minimum. I have to sacrifice returns a bit for it, but as you can see they aren't suffering too much.

Good Trading...