Evidence used in developing new funds may not be so reliable, a new study suggests that
Smart-beta mutual funds, which take the middle ground between active and passive management, have been on a tear. An analysis of Morningstar figures shows that they’re on track to reach US$1trn in assets under management by the end of the year.
But one study suggests that smart-beta funds should be examined with a more critical eye. Research led by Antti Suhonen, a finance professor at Aalto University in Finland, indicates a possible problem with the “robustness” of smart-beta strategies, according to the Financial Times.
Smart-beta strategies are different from passive indexing because they use factors other than market capitalization, such as dividend payout levels, price-earnings ratios, and price-to-book ratios. Before a strategy gets launched, academics and index providers do “back tests” on factors, essentially examining how well the factor would have done if it were used in the past.
Suhonen’s study, which looked at 215 strategies across five asset classes that were developed by investment banks, raises concerns about these back tests. According to the study, companies that back-test tend to do data mining — testing different figures to get a desired result.
The study also found a median deterioration rate of 73% in Sharpe ratios among all the funds studied when comparing back-tested and live post-launch performance, suggesting a “lack of robustness in trading strategy performance.”
“The joke in the industry is that you’ve never seen a bad back test and you will never see one,” Hortense Bioy, director of passive fund research in Europe at Morningstar, told the Times. “The reality is that there is a fair amount of data mining, testing many things to obtain a desired result.”
“Computing power is cheap, and it is easy to perform millions of back tests and find which one works best or looks best to clients,” said Peter Sleep, senior portfolio manager at UK wealth-management firm Seven Investment Management. He advises investors to look at back tests as a “marketing document.”
The average back-test period for the funds in Suhonen’s study was 10.7 years, and the average live period was 4.6 years. Rob Arnott, a smart-beta pioneer who more recently became a critic of the industry, has described the live timeframe in the study as short. However, he said the research provides “empirical evidence of ‘factor fishing’ and back-test overfitting.”
If Arnott is correct, many of the factors that promise investment outperformance — currently totalling more than 400, according to former American Finance Association President John Cochrane — are likely to disappoint.
Francois Millet, product line manager for ETFs and indexing at French asset manager Lyxor, looks for the best smart-beta funds, which combine “good back tests,” an easy explanation for expected outperformance, and transparency about what drives the outperformance.
“If it is too complicated and too difficult to explain, it is usually a good reason for staying away from the strategy,” he said.
For more of Wealth Professional's latest industry news, click here.
Related stories:
What are the hallmarks of a winning mutual fund?
Most smart-beta ETFs performing worse than the market
But one study suggests that smart-beta funds should be examined with a more critical eye. Research led by Antti Suhonen, a finance professor at Aalto University in Finland, indicates a possible problem with the “robustness” of smart-beta strategies, according to the Financial Times.
Smart-beta strategies are different from passive indexing because they use factors other than market capitalization, such as dividend payout levels, price-earnings ratios, and price-to-book ratios. Before a strategy gets launched, academics and index providers do “back tests” on factors, essentially examining how well the factor would have done if it were used in the past.
Suhonen’s study, which looked at 215 strategies across five asset classes that were developed by investment banks, raises concerns about these back tests. According to the study, companies that back-test tend to do data mining — testing different figures to get a desired result.
The study also found a median deterioration rate of 73% in Sharpe ratios among all the funds studied when comparing back-tested and live post-launch performance, suggesting a “lack of robustness in trading strategy performance.”
“The joke in the industry is that you’ve never seen a bad back test and you will never see one,” Hortense Bioy, director of passive fund research in Europe at Morningstar, told the Times. “The reality is that there is a fair amount of data mining, testing many things to obtain a desired result.”
“Computing power is cheap, and it is easy to perform millions of back tests and find which one works best or looks best to clients,” said Peter Sleep, senior portfolio manager at UK wealth-management firm Seven Investment Management. He advises investors to look at back tests as a “marketing document.”
The average back-test period for the funds in Suhonen’s study was 10.7 years, and the average live period was 4.6 years. Rob Arnott, a smart-beta pioneer who more recently became a critic of the industry, has described the live timeframe in the study as short. However, he said the research provides “empirical evidence of ‘factor fishing’ and back-test overfitting.”
If Arnott is correct, many of the factors that promise investment outperformance — currently totalling more than 400, according to former American Finance Association President John Cochrane — are likely to disappoint.
Francois Millet, product line manager for ETFs and indexing at French asset manager Lyxor, looks for the best smart-beta funds, which combine “good back tests,” an easy explanation for expected outperformance, and transparency about what drives the outperformance.
“If it is too complicated and too difficult to explain, it is usually a good reason for staying away from the strategy,” he said.
For more of Wealth Professional's latest industry news, click here.
Related stories:
What are the hallmarks of a winning mutual fund?
Most smart-beta ETFs performing worse than the market
