Everyone knows that past performance does not guarantee future success. But when it comes to choosing where to invest their institution’s cash, selectors still rely on rating systems, which award funds stars, crowns or marks out of five.
Most analysis is quantitative, crunching data and statistics to evaluate funds based on net asset value and performance. Quantitative systems include Morningstar’s star rating, FE Crown fund ratings, Scope Mutual fund rating (formerly Feri) and Thomson Reuters’ Lipper leader rating system.
In addition, there are a handful of qualitative systems, which carry out in-depth analysis of funds’ strategies such as types of asset chosen and when a manager makes changes. These systems include Morningstar Qualitative Rating, and Sauren Fund Manager Rating (see table, below).
Despite the widespread use of rating systems, the UK’s Financial Conduct Authority has expressed concerns over their effectiveness and ability to identify funds that will outperform the market.
PA sister title Expert Investor decided to take the pulse of European fund selectors on this issue. Do rating systems work? Are they effective? And which is better – quantitative or qualitative?
The numbers game
Qualitative analysis pips quantitative in the eyes of Stephan Arnold, executive director for fund research and investment at Vontobel Asset Management.
He says quantitative analysis of funds is a good first step to identify strategies that could outperform, and then qualitative analysis can be used to drill down and find the winners. However, he finds rating system categories “too heterogenous”.
“The ratings are not part of our process,” he says. “We define our own peer groups that are cleaner and therefore have better comparisons on similar strategies.”
Frankfurt-based Thomas Romig, head of multi-asset portfolio management at Assenagon Asset management, does not rely on rating systems either, but still mines platforms like Bloomberg, Lipper and Morningstar for data.
Quant ratings provide a “first impression”, Romig says, adding that if a fund rated poorly, it did not mean it should be dismissed.
“Specific reasons could be responsible for a poor quant result,” he says. “A fund with a one or two star rating [out of five] could still be interesting to us.”
“We do our own quantitative analysis to get a view on past risk-taking practices. We look at how the fund manager reacted during a period of market stress – whether they decreased or increased risk. This is the basis for our qualitative assessment.”
Romig adds that he did not rely on quantitative measures because a fund’s asset weighting meant it could underperform over a certain period of time but might outperform in the future.
“When we look at the fund’s quant, we want to know how the manager performed during a drawdown period, on a relative or absolute basis depending on the fund. We also look at total performance over the long term relative to acceptable asset classes or indices,” he says.
Rome-based Ilaria D’Ascenzio, a fund analyst at BNL BNP Paribas Private Banking, shares the view that quantitative analysis represented a “first step” to choosing a fund and says she “does not care” about a funds’ ratings because she prefers to analyse them herself.
“I don’t like Morningstar very much,” she says, “because I don’t like how they manage their asset classes.”
However Juan Hernando, head of fund selection at Morabanc, is a fan of Morningstar’s research. Its star rating system assigns a one to five-star ranking to funds based on past performance relative to peers.
“I like the Morningstar research team in Spain and the way they analyse funds, the market and trends,” Andorra-based Hernando says. Nonetheless, he points to a Morningstar report from June last year advising that funds awarded five stars are not guaranteed to perform better than the average in the future.
“Often funds awarded five stars perform worse over the subsequent five years compared to their lesser-starred ounterparts,” he says.
The report notes: “The star rating is a retrospective rating – it analyses past profitability and risk, not future profitability and risk. Given this warning, it is curious that many institutional investors continue to rely on this metric to select their funds.”
The star systems’ value is found in helping identify funds that performed significantly below their competitors in the past, Morningstar says, adding that if a fund systematically obtained the lowest rating over successive years, it generally indicated problems such as poor management or high fees.
Qualitative or quantitative
Hernando says he uses Morningstar qualitative ratings in tandem with his firm’s own inhouse research.
“Sometimes other people’s research sees things that we haven’t or they have interviewed a manager and have drawn different conclusions,” he says.
“I find it useful when there is a discrepancy with quant and qual results, but it doesn’t happen often. If a fund has a good quant score but not a good qual rating, I look at if the fund only had good performance in one market.”
“On the other hand, there are funds that haven’t performed well but the qual rating has been high due to the process, the manager, price a t nd philosophy of the fund,” he says. “It is interesting to note when there is a discrepancy.”
Vontobel’s Arnold does not give too much credence to qualitative ratings relying instead on his own qualitative analysis.
“It is important to know the detail of the fund process, how it acts during different stages of the process, restrictions, attributions, and whether it has a defensive approach in a cyclical environment,” he says.
