Investors face a quandary as analytical tools depart from standard academic definitions of factors

Key Points

  • Despite academic evidence, a lack of consensus on factors persists at industry level 
  • Different factor analysis tools use different definitions of factors
  • Factor exposure of stocks and portfolios is measured through distinct approaches

At first glance, the latest controversy surrounding factor investing appears to be nothing more than an academic quibble. Scientific Beta, the smart beta index provider set up in 2012 by the EDHEC-Risk Institute and linked with the EDHEC Business School, is advocating the use of “academically-validated factors”. In doing so, it has warned investors against relying on “proprietary factor definitions and analytic toolkits” provided by other commercial organisations. 

If this sounds too technical, remember that how one defines factors and analyses factor portfolios can determine the success of a factor strategy. According to Scientific Beta, definitions of factors and analytic tools that deviate from the academic literature produce “non-standard factors” and therefore defeat the purpose of factor investing.

The controversy was sparked by a two-part paper published by Scientific Beta last October, which presented two main arguments. The first relates to the definitions of factors. The authors argued that the definitions used by commercially popular analytic tools are “completely inconsistent” with those that have been identified by academic research. By way of example, peer-reviewed academic studies use stock book value-to-price as a measure of the value factor in equities. Instead, commercial analytic tools often use a combination of valuation ratios including price-to-book-value, price-to-forward-earnings and price to cashflow. This, according to the authors, exposes portfolios to risks that are not rewarded systematically, ultimately betraying the essence of factor investing.

The second argument is that ‘factor betas’ should be preferred to ‘factor scores’ as a way to measure the factor exposure of stocks and portfolios. The first approach consists of estimating the correlation of a portfolio with a factor, based on past returns. It is arguably the most rigorous approach from a statistical perspective. Factor scores approaches, used by popular analytics tools such as MSCI Factor Classification Standard (FaCS) and Style Analytics, measure the factor exposure of a portfolio based on the characteristic of the stocks in that portfolio. The authors of the paper argue that this approach ignores the correlation across factors, leading to double counting of exposures. In other words, a value-tilted portfolio as measured using the factor scores approach could in fact be heavily exposed to other factors, such as size or quality.

Scientific Beta reiterated the arguments in a paper published earlier this year. It concluded: “Factors used in investment practice show a stark mismatch with factors that have been documented by financial economists. […] Therefore, many factors that appear in popular investment products and analytic tools are likely false.” The authors argued that factors as defined by commercial analytic tools are unlikely to persist and that relying on those definitions can lead to unintended exposures. “Available factor products thus do not deliver on the promise of factor investing,” they said. 

Such strong language is arguably justified but it was always likely to cause outrage. Felix Goltz, research director at Scientific Beta, one of the authors of the papers, uses a softer tone when he describes the thought process behind the studies. “The argument we are making is not that the factors as they are defined in academic literature are the only true ones, and that therefore everyone should be using those. Nobody actually knows what the true factors are. We are just trying to find out what a reasonable definition of factor should be, by studying the data and seeing how things work in practice. We are also saying that you can wrongly define factors,” he says.

The big question, Goltz says, is whether research results can be replicated. In the academic literature, research results using standard definitions of factors have been replicated countless times. 

Conversely, the results of tests that use non-standard definitions have not been replicated to an extent that makes them scientifically acceptable, Goltz argues. “Given the risk of ending up with the wrong factors, it is much safer for investors to stay with the standard ones, on which there is lots of consensual evidence”, he says.

What about the argument that using standard academic factors creates an implementation challenge? Goltz says: “It is true that in many academic studies implementation aspects are not addressed. The problem is that if you were to use an academic factor definition with the rebalancing frequency that it implies, you could face quite significant transaction costs. But there are mitigation techniques that do not require changing the factor definition. These factors offer a premium even when implementation techniques are used.” Examples of mitigation techniques are focusing on a liquid universe or using a staggered rebalancing method. 

Goltz points out that there are potential drawbacks with using a factor-beta approach for measuring factor exposures, too. He says: “Factor betas need to be measured using statistical methods, which introduces the possibility of measurement errors. Furthermore, it is difficult to pick up sudden changes in factor-beta exposures, while factor scores change every time there is accounting information available.” 

Scientific Beta’s uncompromising position on this topic puts it at odds with more established providers of factor-analysis tools which are less strict about applying academic research. The organisations on the receiving end of Scientific Beta’s criticism maintain that their frameworks are meant to reflect investment practice as well as theory. 

“Nobody actually knows what the true factors are. We are just trying to find out what a reasonable definition of factor should be, by studying the data and seeing how things work in practice. We are also saying that you can wrongly define factors” - Felix Goltz

Bernie Nelson, president of Style Analytics North America, says: “We all drink from the same well, which is the academic literature where, for instance the value factor is defined as book value to price. But the academic factor models are not tuned into the real world of practitioners. The fact is that many investors at the moment are questioning the relevance of the book to price ratio as a proxy for value. Investors are sensing that something has changed and weighing the growing importance of things like intangible assets, R&D, brand value in determining the value of a stock. If you hang your hat just on book-to-price, I think you are detuning yourself from the discussions that are currently taking place among investors.”

screen shot 2019 03 29 at 14.42.38

Furthermore, Nelson points out that Style Analytics’ framework, which focuses on style tilts, is forward-looking, as opposed to factor betas-based frameworks. “We are looking at how managers are positioning their portfolios with respect to factors. We don’t like using only backward-looking, regression-based techniques because they are too period-specific and do not allow that flexibility,” says Nelson. Style Analytics also shows ‘academic’ factors separately, he adds.

Dimitris Melas, global head of equity research at MSCI, takes a similar view. He says: “We see our mission as creating a common language for investors to classify stocks. With our FaCS framework, we are describing the main equity factors to reflect the way a fundamental manager would look at a stock. Our approach has a strong foundation in academic research but also reflects industry practice. Our aim is to take academic research and make it useful and actionable for investors.”

Melas also argues that book-to-price may no longer capture the true value of a stock, due to the growing importance of characteristics that are not captured by accounting measures. “Book to price is only one of many different ways to capture value. Practitioners look at different valuation ratios that offer additional, complementary information on the value of a company,” he says.  

Ultimately, Melas attributes Scientific Beta’s somewhat confrontational style to its legitimate desire to emerge as a provider of factor-analysis tools and factor indices. He says: “This happens in every industry. If you are a new entrant, you must raise awareness on how you differ from more established players, and that’s fine. Competition is good. It ensures we keep our tools and our data up to date.” 

It could be argued that Scientific Beta’s academic approach and the ‘real-world’ approach of Style Analytics and MSCI simply serve different purposes. The former attempts to apply academic research to the real world while minimising the risks of any errors seeping in but is highly prescriptive. The other approach aims to offer investors flexibility, but investors must be aware that flexibility can lead to unintended, or diluted, results. Think of it like the difference between a prescription and an over-the-counter (OTC) drug to cure the same symptoms, such as a headache. The prescription drug is potentially more powerful but taking it requires all sorts of precautions. The OTC medication can be taken more freely but is not necessarily fit for the specific purpose. 

However, Scientific Beta makes some valid points that should be carefully considered by investors. This leads to the question: how should investors choose which factor analysis tools to use?

Axioma, a provider of risk-management tools, leans on the side of Scientific Beta. Melissa Brown, managing director of applied research, says: “We provide factor-risk models, including ‘factor-mimicking portfolios’ that have pure exposure to factors. In order to do that, we have adopted very specific definitions of what we would call factors. We would advocate purity, but we also provide tools to our customers to use their own definitions in a risk model.

“Our view is that if you combine different definitions of value, you are diluting the impact of that factor on your portfolio. Different stock valuation ratios are not that correlated. If you build risk models, you want some parsimony, otherwise you are double counting or getting bad statistical properties in your model. We would be particularly cautious with factor definitions that do not limit the exposure to other factors,” she says. 

Brown, however, points out that if the goal of a factor-analysis tool is just to calculate the performance attribution of a portfolio, then it is helpful to have the factor definition match what the portfolio is trying to achieve. “It can help you explain whether a manager did what they said they were going to do, and how it worked out,” she says.

screen shot 2019 03 29 at 14.44.20

When it comes to factor betas versus factor scores, Brown prefers the latter. She says: “The problem with beta is that it is very unstable. A portfolio might have high sensitivity today, but roll forward and for whatever reason that sensitivity is no longer there.”

Pragmatic Approaches
Joop Huij, head of equities factor investing and index research at Robeco, says his firm has conducted extensive research that confirmed only standard academic factors are truly relevant. He adds: “However, we would not go as far as Scientific Beta with saying that other analytic tools may result in nonsense outcomes. In defence of those approaches, they originate from a different background, which is risk management. They focus on a broader set of variables, other than factor premia, that are relevant to understand where risk comes from.” 

Robeco has developed specific tools for analysing factor-investing portfolios but Huij does not dismiss the use of other tools. He adds: “Running a factor analysis is more of an art than just the pressing of a button. The person that performs the analysis is way more important than the tool that is used. It is comparable with a doctor coming up with an interpretation of an x-ray scan. Using tools grounded in risk management rather than pure factor analysis is tricky and can lead to misinterpretation. However, if a knowledgeable person uses that too, they will not come to a different conclusion from using a pure factor-analysis tool.”

As far as the difference between factor betas and factor scores approaches for analysing factor exposure, Huij compares the first with a low-res film while the latter is akin to a high-res photograph. Both can provide useful information on portfolios but have a different purpose. However, he argues that factor betas approaches are better suited for pure factor-analysis exposure. 

“As a practitioner that pays particular attention to academic insights, we aim to help our clients understand the differences between the two approaches to factor analysis and get the best perspective on the exposures of their portfolios,” Huij says.

NN Investment Partners has a different approach to the problem. Stan Verhoeven, senior portfolio manager for factor investing, explains that the company applies factor investing to multiple asset classes. This is in line with a belief that single-factor strategies, and factors applied to individual asset classes, do not provide significantly higher risk-adjusted returns. 

Verhoeven says: “Factors themselves do not shoot the lights out. There are no factors with amazing risk-adjusted returns on a standalone basis. The real power is when you combine them, because they generally have low correlations between each other. Furthermore, a real test of robustness is applying the factor to different asset classes.” 

When it comes to value, of course, it is impossible to use a single definition across asset classes. Verhoeven favours the standard academic definition of value in equities owing to its simplicity. “Complexity sells better but we believe that often the best way is the simplest way. That is why we focus on factor definitions that require as few assumptions as possible. Of course, we take into account implementation problems, but our focus is on simplicity,” he says.

screen shot 2019 03 29 at 14.44.41

Using as many different tools as possible may be a good starting point, according to Michael Hunstad, head of quantitative strategies at Northern Trust Asset Management. But when the goal is maximising risk-adjusted returns, consistency and transparency are key, he argues. It is important, therefore, not to get bogged down on any individual risk model.

The lack of consensus around the definition of factors has led to great differences between factor-investing strategies and some sub-optimal allocations, according to Hunstad. In his view investors should be wary of taking shortcuts to getting factor exposure. “Naive factor tilts can often lead to a higher exposure to unintended risks,” he says. “One example are low volatility strategies, which in recent times performed well despite being exposed to many non-low-volatility stocks.”

“We need analytical tools to measure unintended risk and minimise it. One simply has to work hard to build to a truly efficient factor portfolio.”