Beware the hidden flaw in some exchange-traded funds that can make mutual funds look good by comparison, says The Globe and Mail columnist Rob Carrick.

Exchange traded funds (ETFs) ideally provide the return of a specific stock or bond index, minus fees. But in the global and bond categories, some ETFs are lagging their indexes by much more than that.

The problem is called tracking error and it should be a top consideration when you’re researching ETFs for your portfolio.

An example of an ETF with significant tracking error is the iShares MSCI EAFE Index Fund (Toronto: XIN), which offers a way to invest in stock markets outside North America with the benefit of currency hedging. The expected tracking error for this ETF would be the return of the underlying Morgan Stanley Capital International Europe Australasia Far East Index in Canadian dollars, minus the 0.5-percent management fee and a little extra to cover other costs.

The real-life gap between the index and the fund is a fair bit larger, though. Data on the iShares Web site (ca.ishares.com) show the index lost 1.4 percent annually over the five years through Sept. 30, while the ETF itself lost an average annual 2.6 percent.

ETF investing hinges on the idea that the returns of a large number of mutual funds at any given time will fall short of benchmark stock and bond indexes. But this assumes tracking error isn’t a spoiler.

In fact, tracking error for some ETFs has reduced their returns to the point where they look uncompetitive with mutual funds. For example, international equity funds outperformed XIN on average over the past five years by holding their losses to just below 2 percent annually.

Analyst Pat Chiefalo of National Bank Financial has been working on creating model ETF portfolios recently and he’s concluded that in the Canadian equity and bond categories, ETFs are very competitive against mutual funds. But thanks in large part to tracking error, he’s less positive about global and emerging markets ETFs.

“From a foreign equity perspective, there may be an argument to think about mutual funds,” he said.

Technically, tracking error measures the amount by which an ETF’s returns are likely to fluctuate above or below the performance of the index it tracks. But in the real world, the term is used more loosely to describe the actual gap between an ETF’s return and that of its index.

An example of an ETF with inconsequential tracking error is the massive $11.7-billion iShares S&P/TSX 60 Index Fund (Toronto: XIU), which according to the iShares.ca website has underperformed its target index by 0.19 percentage points on an annual basis since May, 2000. The management fee for XIU is 0.17 percent.

Another ETF with negligible tracking error is the iShares DEX Short Term Bond Index Fund, which has a management fee of 0.25 percent and has lagged its index by an annualized 0.28 points.

The more severe tracking error seen in XIN can be explained in part through currency hedging, where financial instruments called options, futures, or forward contracts are used to minimize the impact of currency fluctuations on returns from investments outside Canada. The cost of hedging adds 0.05 to 0.15 of a percentage point to an ETF’s fees (these fees are on top of the management fee).

The way in which hedging is carried out further contributes to tracking error. Hedges are reset every month, which means they can’t quickly be adjusted to reflect big market moves. If an index surges 10 percent in a month, an ETF could be under-hedged and thus partly vulnerable to the currency fluctuations it’s supposed to suppress. “Over time, you can see this starts to make an ETF’s performance deviate,” said John Gabriel, a specialist in Canadian market ETFs with Morningstar in Chicago.

Another contributor to tracking error is the way in which an ETF firm goes about replicating the returns of an index. In some cases, funds will hold all the same stocks as the index. In others, like the iShares MSCI Emerging Markets Index Fund (Toronto: XEM), some stocks are omitted.

“There are something like 21 countries and 1,500 stocks in the benchmark, and it just doesn’t make sense to hold 1,500 stocks,” said Oliver McMahon, director of product management for iShares ETFs at BlackRock Canada.

This approach is called index optimization and it balances lower costs for investors against the potential for returns to deviate from the index and create tracking error. BlackRock’s own data show that XEM made 11.2 percent for the 12 months to

Sept. 30, compared to 14.9 percent for its underlying index. The management fee for XEM is 0.82 percent.

The iShares emerging markets ETF is a Canadianized version of the US-listed iShares MSCI Emerging Markets Index Fund (NYSEArca: EEM), which according to Morningstar’s Mr. Gabriel, has been criticized by investors for its tracking error. He said money has flowed out of this ETF and into the competing Vanguard Emerging Markets ETF (NYSEArca: VWO).

Tracking error for bond ETFs can also be affected by index optimization. Consider the iShares DEX All Corporate Bond Index Fund (Toronto: XCB), which has a management fee of 0.4 percent and has underperformed its index by 0.82 of a percentage point since mid-2006.

To Mr. Gabriel, that’s a reasonable price to pay for the ability to quickly and cleanly trade in and out of a diversified package of bonds. Buying corporate bonds individually could be a lot tougher, given the relatively small size of the Canadian market.

The most important criteria for choosing the right ETFs: fees, trading volumes, the diversification and transparency of the underlying index and tracking error.

“Tracking error is extremely important for us,” said Mr. Chiefalo of National Bank Financial. “Active managers get paid to meet or beat their benchmark, ETFs get paid to track it. To us, that’s the measure of quality. The closer you are to your index, the better the quality.”