Numbers Aren’t Always Answers

11/17/2011 7:30 am EST

Focus: MARKETS

Just because you can quantify something doesn’t mean you should buy into models blindly, notes John Sartz of Investor’s Digest of Canada.

Lovers of British humor may recall Peter Cook, the English satirist, writer, and comedian who died in 1995. He had a routine in which he contrasted the rigorous exam required to become a judge with the less rigorous exam required to become a coal miner.

For his miner’s exam, Cook was required to provide his name, scoring 75% as a result. This is the kind of guy I am. So, I often think of Cook when I read dissertations on investments where I believe the research lacks the requisite rigor.

On several occasions, I’ve warned readers to be highly skeptical of statistical models. I’ve emphasized that statistical correlation shouldn’t be confused with causality and that observed correlation may in fact be random.

Think of it this way. Were you to do a statistical analysis of the market’s behavior over the past five years by sorting alphabetically all the names listed on the Toronto Stock Exchange, I’d wager that stocks beginning with certain letters would be out-performers, while others would be laggards.

But would you as an investor act on those results? I hope not. Yet, had I used sorting criteria that sounded slightly more investment-respectable, you might have bought into my selection system.

Here’s an example: Not long ago I read a piece that claimed to prove categorically that investment in lower-risk equities outperforms the market over the long term. And this, as Chief Dan George said in Little Big Man, “made my heart soar like a hawk.”

After all, what could be better than proving that the lower a stock’s investment risk is, the higher its return will be?

According to this piece, those of us who’ve actually believed that risk and reward are positively correlated have clearly had it wrong all these years. Go figure: “My bad,” as my younger daughter might say.

But this is where rigor comes in. And a study which purports to show that stocks with lower risk do better over the long term is based on less-than-vigorous research.

For example, as a proxy for the long term, the authors use a time span of 11 years. And 11 years would be perfect if one were a dog. But for us ordinary humans, one-eighth of our life span doesn’t a long term make.

Moreover, a cynic might wonder why the authors chose 11 years. It’s not exactly a round number.

Of greater importance, the 11-year period in question went from 1999 until 2010. If one were to try, it would likely be impossible to find a less representative period for the analysis of investment returns.

Consider this. The beginning of the period under study constitutes the peak of the tech bubble and then its collapse. Moreover, the end includes the 2008 commodities bull and, after that, its collapse.

Now, I fully agree that weird things happen in the stock market. But keep in mind that since the 1930s, there have only been three bears in which the Canadian stock market declined more than 40%, with two of the bears contained within the 11-year period. No period of, say, a decade nor any period of fewer than 20 years could have accomplished this feat.

As a result, absolute returns during the period were obviously low. That being the case, risk-taking wouldn’t have been rewarded.

In spite of all this, I fearlessly predict that gravity will continue to be a factor in our lives. Likewise, any study conducted with reasonable academic rigor will conclude that risk and reward are positively correlated.

Finally, we’ll continue to have to put up with investment dissertations which, while not rigorous, will still make illogical claims. After all, there are no laws preventing people who’ve successfully passed their coal miner’s exam from writing about investing.

Subscribe to Investor’s Digest of Canada here…

Related Reading:

Related Articles on MARKETS