Featured Image

ETF Due Diligence: Chasing Quality, Not Performance

Companies and Markets

By Elisabeth Kashner, CFA  |  November 2, 2017

The Wall Street Journal lit a fire in the mutual fund due diligence world last week, publishing The Morningstar Miragea 19-page analysis of Morningstar’s five-star rating system. In it, the WSJ charged that “funds that earned high [Morningstar] star ratings attracted the vast majority of investor dollars. Most of them failed to perform.” The WSJ continued, “On the average, five-star funds eventually turn into merely ordinary performers.”

The cost of performance chasing is staggering. ETF investors are not immune. Just ask the crowd that invested $8.3 billion in WisdomTree Europe Hedged Equity Fund (HEDJ-US) between January 1, 2015 and June 30, 2016. 

The cumulative net return of HEDJ during that time was a mere 0.11%, while AUM increased from $5.6 billion to $10.6 billion.

The cumulative net return of HEDJ during that time was a mere 0.11, while AUM increased from $5.6 billion to $10.6 billion.jpg

Over that 18-month period, buy-and-hold investors earned .07% per year. On a dollar-weighted basic, investors lost 21.58% per year. Dollar-weighted returns show the average experience of the “hot money”—the investors who created and redeemed shares of the fund during the period. Dollar-weighted returns take into account the rate of return to every new investment (inflows) and sale (outflows).

That 21.65% performance gap between buy-and-hold and tactical trading in HEDJ shows just how painful performance chasing can be.

In March of this year, I compared fund returns vs. investors’ returns for 60 ETFs in the U.S. total market, U.S. large cap, and Developed Ex-U.S. total market segments, for a five-year period. That’s all the “smart beta” and a few representative vanilla funds that had a five-year track record, with no change in investment strategy that might cloud the results. 

The results were clear. Buy-and-hold won out in 53 of the 60 funds. Dollar-weighted investors underperformed their buy-and-hold counterparts by an average of 1.7% per year, per fund.

How Performance Chasing Falls Short

Performance chasing often ends badly, because few funds can outperform quarter after quarter, year after year. As Morningstar explained to The Wall Street Journal in a written response to its article, “Reversion to the mean is a powerful force that can affect any investment vehicle.” Performance history, even when risk-adjusted, is not a reliable predictor of future results.

Morningstar CEO Kunal Kapoor also wrote in the rebuttal, “Our research finds that the star rating points investors toward cheaper funds that are easier to own and likelier to outperform in the future. These are qualities that correspond with investor success.”

We agree that fund qualitylow cost and low operational riskare predictable factors that influence long-term fund performance, because every penny that leaks out of an investment account is a penny that can’t provide future returns. 

Fund quality is mostly a function of costsboth operational and tradingbut it also includes some risk assessment that measures the probability of rare, high-impact events that potentially have severe consequences. The biggest operational costs are tracking difference and trading charges, which can be measured by the bid-ask spread. ETF closure is an example of a severe negative event, as is ETN issuer default.

Analysis of these costs and risks allows us to differentiate between funds with similar mandates and equal expense ratios, such as iShares TIPS Bond ETF (TIP-US) and PIMCO Broad US TIPS Index ETF (TIPZ-US). Both funds cost 0.20% per year, but the iShares product tracks its index more closely, and trades with higher volumes and much lower spreads. That’s why FactSet ETF Analytics gives TIP an A letter grade, and TIPZ a B. 
Both funds cost 0.20 per year, but the iShares product tracks its index more closely, and trades with higher volumes and much lower spreads.jpg

Costs can add up over time. An investor who chose A-rated ETFS Physical Platinum Shares (PPLT-US) over F-rated iPath Bloomberg Platinum Subindex Total Return ETN (PGM-US) for a one-year holding period would have held onto an average of 2.05%, based on the median tracking difference and average spreads. That’s a huge performance boost, with no market risk.

An investor who chose A-rated ETFS Physical Platinum Shares over F-rated iPath Bloomberg Platinum Subindex Total Return ETN.jpg 
Performance analysis simply is not a part of an assessment of operating and trading costs. Instead, we look at fund performance and market risk as a separate section, called Fit.

ETF Analytics’ Fit score ranks funds on the basis of active risk. A score of 100 indicates that the fund completely reflects its opportunity set; a score of zero points to extreme bets against the market. Fit measures relative risk, without extrapolating returns.

Why not returns? It turns out that historical performance—even rigorously measured, statistically significant outperformanceis an unreliable predictor of future performance. 

This second point is worth taking first. In the ETF universe, risk-adjusted outperformance that passes even the most basic tests of statistical significance is hard to come by. And it’s not just because vanilla ETFs are built to mimic the market rather than outperform. It’s worse. Any time I’ve tested the risk-adjusted returns of “smart beta” ETFs that promote their potential for outperformance, the vast majority perform in-line with risks taken. Not over, not under. Last time I did this, with a five-year look-back as of March 31, 2017, only 8.8% of the 53 funds I tested delivered positive alpha at the generous 90% significance level. In layman’s terms, most of the raw over- and under-performance was attributable to risk and statistical noise.

Even at the simple levelraw performancepersistence of returns turns out to be vanishingly rare. Just ask Aye Soe and Ryan Poirier, authors of the semi-annual SPIVA study, who explained, in their June 2017 Persistence Scorecard, ”no large-cap, mid-cap, or small-cap funds managed to remain in the top quartile at the end of the five year measurement period. This figure paints a negative picture regarding long-term persistence in mutual fund returns.” They went on to say, “The data show a stronger likelihood for the best-performing funds to become the worst-performing funds than vice versa.”

In other words, performance chasing is often quite costly.

ETFs that offer broad-based, cap-weighted coverage of a well-defined market segment are built to reflect the market rather than outperform it. In order to outperform the market, a fund has to offer exposure to risks that are different from the market baseline, aka relative risk. No relative risk means no outperformance, and no underperformance. There’s nothing to chase. 

ETF Analytics’ Fit score gives high marks for minimizing relative risk.

For example, iShares Core MSCI Europe (IEUR-US) scores a 99 for Fit, because it provides broad-based, unbiased coverage of the developed European equity markets. By contrast, WisdomTree Europe Hedged Equity Fund (remember the hot money flowing into HEDJ?) holds a narrow portfolio of export-oriented firms that pay dividends, weighted by annual cash dividends paid with specific caps, with a currency hedge overlay. Investors have to get four bets right to win with HEDJ: Europe, exporters, dividend emphasis, currency hedge. That’s why HEDJ scores 39 in Fit.

These portfolio differences make for vastly different returns. While HEDJ had an excellent month (as of 10-27-17), it trailed badly over the six-month lookback, even against other currency hedged or “smart beta” funds like DBEU and EUMV. 

While HEDJ had an excellent month it trailed badly over the six-month lookback, even against other currency hedged.jpg

ETF Analytics’ Fit score rolls up portfolio comparisons and the returns difference analysis. Investors who prefer to ride the mean, rather than watch their investments surge and fall back in potentially costly mean reversion need only search out a high Fit score. While these investors will bear market risk that comes with exposure to the segment, they can at least avoid relative risk. For those who prefer to make tactical bets, the Fit score allows for a quick assessment of the active risk chosen.

Analyst Pick, a designation granted to one fund per segment, combines classic ETF due diligence and a straightforward investment philosophy that minimizes active risk. FactSet’s ETF Analyst Pick recommends ETFs that investors can hold for an extended timefor as long as they want exposure to a market segment. Analyst Pick is the antidote to performance chasing.

By applying basic cost and liquidity thresholds, Analyst Pick narrows each segment’s offering to the funds that are cheap and low risk to hold, and easy to trade. Next, the methodology selects the simplest vanilla investment strategies, favoring those with the highest Fit score.

Here’s how it works within the US Total Market Equity segmenta favorite for long-term equity exposure. ETF Analyst Pick, Vanguard Total Stock Market ETF (VTI-US), posts a headline expense ratio that is 0.01% higher than the segment low, but executes as if it were even cheaper, with a median -0.01% annual tracking difference. VTI is cheap to trade at a 0.01% spread. Most importantly, VTI’s Fit score of 99 assures investors that the fund does what it claims togives exposure to the US Total Marketwithout making bets against the market.

The table below, from FactSet’s new ETF screener, makes it clear why VTI stands out in this crowded segment.

Most importantly, VTI’s Fit score of 99 assures investors that the fund does what it claims to – gives exposure to the US Total Market – without making bets against the mark.jpg

ETF due diligence works best when costs and risks are foremost. Low cost, liquid, well-run funds preserve investor capital for taking desired risks, such as exposure to a particular market segment. Pointing to the odds-on bet of broad-based, cap-weighted exposure helps position long-term investors to avoid active risk and costly performance chasing.  

demonstrating the value of active management

Elisabeth Kashner, CFA

Vice President, Director of ETF Research and Analytics

Ms. Elisabeth Kashner is Vice President, Director of Exchange-Traded Fund Research and Analytics at FactSet. In this role, she develops tools and methodologies for all aspects of ETF and mutual fund classification and analysis with a focus on costs, risks, trading issues, and performance. Prior, she served as director of research at ETF.com and published extensively on the classification, efficacy, and persistence of strategic beta strategies and robo-adviser portfolio exposures. Ms. Kashner earned a BA from Brown University and an MS in financial analysis from the University of San Francisco. She is a CFA charterholder.


The information contained in this article is not investment advice. FactSet does not endorse or recommend any investments and assumes no liability for any consequence relating directly or indirectly to any action or inaction taken based on the information contained in this article.