Last autumn, the Saïd Business School at the University of Oxford published a survey on fund manager selection that came to some controversial conclusions.

Picking Winners? Investment Consultants’ Recommendations of Fund Managers analysed the selections of long-only US equity managers by US consultants for US pension plans. It found no evidence that those selections added value to the plans. In fact, in certain circumstances, the reverse was true.

“On an equal-weighted basis, the performance of recommended funds is significantly worse that that of non-recommended funds,” asserted the authors, Tim Jenkinson, Howard Jones and Jose Vicente Martinez. “The underperformance of recommended products on an equal-weighted basis can be explained by the tendency of consultants to recommend large products which perform worse.”

They went on to say that on a value-weighted basis the performance is “mixed”, and the recommended and non-recommended products do not perform significantly differently from each other.

While the headline findings provided the press with easy targets, some of the subtle questions raised by the research were unanswered. For example, the study looked only at US managers running US long-only assets, an area where it is difficult to outperform the market consistently. Furthermore, it did not analyse performance related to actuarial and asset allocation decisions, which are arguably more important for the health of a pension scheme than portfolio-manager selection decisions.

Perhaps most interesting of all, while the consultants’ selections were influenced by past investment performance, they were significantly more influenced by what the researchers call “soft investment factors” and “service factors” – particularly the presence of a “capable investment manager” and a “consistent investment philosophy”. These seem to be sensible criteria against which to select managers, and suggest something different from the crude return-chasing that advisers are often accused of.

So is the survey an accurate reflection of the added value of consultants’ input in manager selection particularly, and pension-scheme advice in general?

Andy Green, CIO at UK-based Hymans Robertson takes issue with the non-representative asset-class universe chosen for the Saïd Business School research. “The US equity market is regarded as one of the most efficient capital markets, so we would be wary of extrapolating the study’s findings beyond the market in which it was conducted,” he says. “While we believe it is still possible to find managers who can add alpha consistently in efficient markets, undoubtedly the opportunities to do so are greater in less efficient regions. All else being equal, we would have expected some improvement if the study had focused on less efficient markets.”

It’s difficult to judge all consultants on one asset class, agrees Anton Wouters, head of customised and fiduciary solutions at BNP Paribas Investment Partners. “Furthermore, the alpha potential of asset classes can change over time,” he reasons. “Some consultants can find managers which offer good potential in different classes at different times, also managers, and classes may be in or out of favour over time.”

While Chris Jones, managing director and head of public markets and alternatives at Bfinance, agrees that a restricted dataset limits the significance of the results, he insists that this does not invalidate the research.

“Given the study incorporated the less efficient and less well-covered space of small and mid-cap US equities, there was plenty of scope for managers and consultants to add value through skill and process,” he says.

Moreover, he doubts that more favourable results would have been found had the study been broadened. “The flaws inherent in the world of large, buy list-focused investment consultancy apply equally across markets,” he says.

Howard Jones, senior research fellow in finance at Saïd Business School, and co-author of the study, says the team would be happy to run a similar analysis on other asset classes that are considered to be less efficient, but had not so far been able to access such data.

However, he says: “Even in this asset class, investment consultants continue to make recommendations, and plan sponsors follow them, as shown by our flows results – and yet these recommendations do not add value. It is therefore surprising that consultants have continued to make recommendations in this class.”

Aon Hewitt, however, says the study’s results are “less negative” than portrayed in the press.

“The study finds that both recommended and not-recommended products outperformed their benchmarks in aggregate when equally weighted – although non-recommended products outperformed by a greater margin,” Frank Driessen, chief commercial officer, Aon Hewitt Netherlands reminds us. “On a value-weighted basis, recommended products earned greater excess returns than not-recommended, relative to manager and style benchmarks.”

In other words, the results of the study suggest that the subsequent relative performance of managers was negatively affected by their size, and that consultants tended to pick larger managers and funds.

“Excessive scale or rapid growth is well-recognised as a potential problem for active managers,” concedes Green. “It can lead to problems with market liquidity and organisational complexity which can have an adverse impact on the manager’s investment decision-making.”

He says that, understandably, some clients prefer the knowledge that they are investing with a recognised name.

“However, in preparing long-lists of managers, we will canvass a large and diversified range,” he continues. “While there are many excellent individual strategies managed by larger institutions, we typically consider smaller, often newer, firms where managers have a stake in the business to be more focused on performance.”

For Bfinance there is no best manager for its client base as a whole – just the best for each client, whether a large or small manager. The firm has no buy lists, so smaller, nimbler managers favoured by its clients do not get deluged with assets as a result of being on the list.

“An investment manager getting on to a large consultant’s buy list is often a negative factor for us, as the negative effect on performance once they get on to a buy list is well-documented,” says Chris Jones. “But, we are in no way anti-large manager. In certain circumstances, size can even be a competitive advantage.”

Jones says an important factor other than size is the pace of growth. Too fast, and investment teams may struggle to source enough good investment opportunities for new cash inflow and be slow in recruiting the right talent to seek them out.

But in any case, faced with the headlines generated by the research paper – ‘Billions wasted on advice’ is a representative example – readers might be forgiven for believing that manager selection was the be-all and end-all of a consultant’s advisory role. Is it really more important than asset allocation in terms of influencing relative performance? Shouldn’t consultants concentrate on the big decisions such as long-term strategy and asset allocation? And indeed, when it comes to manager selection within that, if outperformance over any given period is fairly random, does it not make sense for consultants to spend less time and resources on ‘picking winners’, and more time focusing on identifying the ‘safest’ managers with the most consistent philosophies – essentially what the Saïd Business School paper finds them doing?

Consultants are prepared to go some way down this line of defence.

“Asset allocation is more important than manager selection,” agrees Ciaran Mulligan, global head of manager research and selection, Buck Global Investment Advisors. “We would look first to get the client to establish the right modelling framework in order to develop asset allocation, then build on that to select the right managers.”

He also agrees that operational robustness and ‘soft’ investment factors are important: these would be assessed by the due diligence process for each fund manager, which would also be carried out on the individuals responsible for generating fund performance.

“We would also maintain that, while you can’t predict with complete certainty a manager’s performance over time, performance is not random, and you can increase ability to predict future performance by having a structural process in place,” Mulligan adds.

In particular, he says, consultants who do well with regard to manager research are typically those who employ separate manager research teams and think of manager research as a distinct career path.

Assessing an investment manager is like assessing the strength of a chain, concedes Bfinance’s Jones – a weak link can cause the chain to snap.

“However, recommending a good manager is not just about picking the ‘least worst’,” he adds. “Edge and differentiating factors must also be identified. It is in this aspect of analysis that the skill, context and experience of the investment consultant can add value. We believe every aspect of an investment manager needs to be analysed, be it how investment ideas are generated, through to more infrastructural issues, including client servicing and transparency.”

Hymans Robertson takes the opposite view to Buck Global Investment Advisors, regarding asset allocation and implementation as being increasingly linked. All of its manager researchers are also asset class researchers. Aon Hewitt also believes that asset allocation and management are linked.

“The prospects for active management are stronger in more flexible, less constrained alternative strategies such as hedge funds,” says Driessen. “Our manager selection record reflects this, with five-year average value added in hedge funds of +2.09%. We recommend that clients shift towards higher allocations in these areas.”

Wouters says: “In general, 80% of performance comes from strategic asset allocation and more dynamic longer-term asset allocation strategies. The other 20% is outperformance by managers, or other alpha sources.

“But even if consultants need to spend more time on the strategic side of things in relation to manager selection, I don’t think spending time and money on the selection process is wasted,” Wouters continues. “If that buys you an additional 1% of alpha, over 20 years that is an awful lot of money.”

And he points out that picking winners is not a one-off decision. “If we pick specialist managers with a specific way of managing money and who take a lot of risk, the universe is limited,” he says. “Moreover, the winner now can also be a loser over time. You should take time and effort to find a decent universe and a group of managers who are able to create added value over time.”

Nick Greenwood, pension fund manager for the Royal Borough of Windsor and Maidenhead, says simply: “Our focus is strategy first, then manager, and it has worked.”

He is a strong advocate of high-conviction portfolios with a longer-term view, but says: “I would suggest that risk aversion causes consultants to focus on low-tracking-error, low-conviction managers to restrict downside risk against the index.”

If this is indeed the case, is there a principal-agent problem? Is this excessive focus on ‘safety’ and ‘consistency’ partly to manage the consultant’s own business risk, and does it lead to opportunity costs in terms of client investment performance?

“This is the nub of the problem – hence why we do the work ourselves, and not via a consultant,” says Greenwood.

Jones at the Saïd Business School acknowledges the issue – and not just between asset owner-principals and consultant-agents. His paper How Institutional Investors Form and Ignore their Own Recommendations, co-written with Martinez, shows that plan sponsors form opinions about the future performance of asset managers, only to ignore these in favour of the past performance of funds and the recommendations of investment consultants.

“This suggests that plan sponsors make decisions that they feel are most defensible towards their stakeholders,” says Jones. “It is possible that investment consultants are appealing to plan sponsors’ preference for easily defensible decisions by themselves recommending funds which have done well in the past.”

Unsurprisingly, the consultants themselves take issue with that idea.

 “We would trust our research process to pick managers we can have confidence in,” insists Mulligan. “Consultants should ensure their clients benefit from developed expertise. If you don’t have confidence in your research, why are you offering your services?”

Green adds: “We have no incentive to produce a sub-optimal outcome for the client. A low-risk approach, if this suits the client’s objectives and risk is measured in terms of predictable outcomes, is more likely to involve greater indexation than ‘safe’ active manager names.”

The truth, as ever, is probably somewhere between the extremes of the ‘billions wasted’ headlines and the well-honed defences of the consultants themselves. Pension fund managers clearly feel that there are commercially-driven biases in their consultants’ practices – but they must recognise that they face similar pressures, and introduce similar biases, in their own work for scheme members. Awareness of these biases, and governance structures designed to manage and mitigate them, are important to both the adviser and the client.