Active Management: Understanding investment skill

Rather than outcomes-oriented measures, Michael Ervolini argues that to assess active managers’ skills they need to be isolated by comparing their portfolios with alternative, ‘adjusted’ portfolios

The debate about skill is taking a new twist, with the investigation into ‘active share’ by KJ Martijn Cremers and Antti Petajisto and ‘best ideas’ by Randy Cohen et al, indicating that more equity portfolio managers have skill than previously believed. Most, however, do not possess sufficient skill to outperform their benchmarks, and that is the problem. This means the often-asked question, “Do excess returns reflect manager skill or luck?” now shifts to the more pertinent inquiry: “Do you have the ability to identify managers with skill, or are you just hoping to get lucky?” This article explains why the question of skill has vexed investors for so long and how it is being answered with the use of a new framework for quantifying investment skill, processes, and behavioral tendencies. 

Fool’s gold 
Capital sources and manager selection consultants rely extensively on traditional portfolio analytics in order to understand the ability of a manager. Measures such as return, relative return, information ratio, alpha, tracking error, hit rates, win/loss ratios, attribution analysis, and value at risk all have their place in manager evaluations. None, however, offers the slightest insight into skill. These metrics measure one thing only, and that is outcome. When they are used as proxies for skill, they lead, predictably, to weak understanding, incorrect conclusions, and actions that undermine success.

Consider two active managers. The first has a hit rate of 35 and the second 45. The first manager underperforms his benchmark while the second outperforms. Which has more skill at buying? It is impossible to know from this information. One reason why is that hit rates do not describe the quality of the manager’s buys, only which fraction of them is sold with a gain. The first manager might generate huge levels of alpha from his buys but give that and more back through weak selling. The second manager may generate only a modest amount of alpha from her buys, while producing most of her excess performance from strong selling and sizing skills. Neither is able to learn anything meaningful about their buying or other skills from hit-rate analysis.   

New framework  
The identification and quantification of skill requires a new analytic approach that is both rigorous and granular. Similar to the way a golfer knows how skilled she is at drives off the tee, fairway shots, and putts, the manager must know how effective she is at buying, selling, and sizing of positions. No one expects a golfer to be able to improve without the proper feedback, while active managers have struggled to improve for decades using outcomes to measure skill. This is the equivalent of asking a golfer to lower her handicap, when all she is provided are the scores of each round played.  

A framework developed by Cabot Research uses the comparison of adjusted portfolio histories to isolate and measure skills. The basic idea is to begin with the actual history of a portfolio and then modify one or more elements in order to construct adjusted portfolios. The comparison of adjusted portfolios to the actual portfolio and to each other supports the precise investigation of skills. Here is a simple example. 

The manager of portfolio ABC is interested in understanding the effectiveness of his sizing decisions. To investigate this, an adjusted portfolio is created that, on each and every day, owns the same names as the manager’s actual portfolio, but all of the manager’s sizing decisions are ignored, and all positions are instead sized using a passive sizing rule. From here the comparison of the actual portfolio to the adjusted portfolio can be based on total return, relative return, or alpha.  

Whichever performance measure is used, if the results for the actual portfolio are greater than the adjusted portfolio, then the manager’s sizing decisions added value – he has sizing skill.  

Adjusted portfolios like these are used to quantify the three basic investment skills (buying, selling, sizing), to investigate more granular skills – for example, effectiveness of adding to losing positions – and to identify and measure the impact of behavioural tendencies (such as the impact of holding winners too long). The essential point is that this new analytic framework measures skills directly, providing both managers and their clients with the feedback needed to make knowledgeable decisions.  

Clarifying  
The new analytic framework also provides rigorous and objective insight into a manager’s investment process. An example of how buying process can be better understood is presented in the figure. This plot describes the typical new buy of the manager at the time of initial purchase. It shows that new buys tend to have very low momentum and low price-to-value, while possessing relatively high return on equity (ROE), trailing earnings growth, and slightly above-average debt. These attributes are consistent with the manager’s value approach of finding great companies whose price has been beaten up.  

Even more interesting is that the chart shows which of his picks go on to become winners (dark grey) and which become losers (light grey). The analysis indicates that while this manager makes buys that are consistent with his intent or style, his best picks are those stocks that are modestly cheap and not heavily discounted. Feedback like this can help managers become more self-aware and improve, and it can also enable the manager to objectively explain his process to clients and demonstrate that it is actually followed. 

The next time you meet with a money manager, try asking one or more of these questions: 
• Which of your skills is strongest and how do you know? 
• How are you going about improving your weakest skill? 
• Do you engage in any behavioural tendency? What are you doing to eliminate it? 
• What is your buy process and how closely is it followed? 

As an institutional investor and a fiduciary, you deserve rigorous and objective answers to questions like these. You need facts, not hunches and educated guesses. A growing number of equity managers now have answers to these and other questions that really delve into skill, process, and behaviours. Armed with this type of rigorous and granular feedback, investors are in a stronger position to identify skilled managers who are most likely to outperform. The choice is clear. You can become more skilled at identifying and working with top portfolio managers, or you can continue with your current approach and hope that you get lucky.  

Michael Ervolini is CEO of Cabot Research, a specialist behavioural finance consultancy, and author of Managing Equity Portfolios: A Behavioral Approach To Improving Skills and Investment Processes (forthcoming, MIT Press)