Many smart beta ETFs are bought with the expectation of long-term market outperformance. The factors that many are based on have been proven both academically and empirically to produce excess returns. 

However, especially in the short term, many of these funds may be vulnerable to different market, factor and economic events. Because factors are correlated, even funds that tilt on one factor (for example, momentum) may be impacted by a big move in a different factor (for example, value). Stress testing is a tool that can help managers and fund investors better understand the potential impact when history repeats, or of plausible moves in economic variables or factors. 

In this article, we show results of stress tests on sample smart beta ETFs. This analysis allows us to examine how current portfolios would have fared under various historical scenarios and to evaluate the impact of big moves in selected economic and model variables. We also compare results from two dates to show how vulnerabilities can differ through time. 

What accounts for these differences? There are many reasons, generally related to portfolio construction. How is the factor defined? How many stocks are in the portfolio and how are they weighted? How often does the fund rebalance? Are exposures to other factors controlled? And so on. 

We chose two different US-based ETFs in each of four categories. The ETFs chosen have substantial NAV and are widely owned (figure 1). 

Managers typically use stress tests to highlight potential negative impacts on their portfolios. For this study, however, we are looking at negative events, even if they might have a positive impact on the portfolio. Most of our tests are linear, which means that one could reverse the sign on the stressor and get an impact of equal magnitude in the opposite direction. The focus of our results is the expected active return versus the S&P 500 as the result of the test. 

The results of this study point to three major conclusions: 1) portfolios with names that sound virtually identical can have very different reactions to stressors, suggesting they are not quite as similar as they sound; 2) factor-based portfolios (ETFs or otherwise) may have exposures to many other sources of risk, and may therefore be impacted by a big move in a factor, even if it seems to be unrelated; and 3) stress tests should not be a ‘one-and-done’ exercise. The impact of a given shock can vary substantially over time as a result of changes in holdings, factor volatilities and factor correlations. 

etfs used in the study

First test: big market downturn 

Our first study was simple: what would happen to our portfolios if the markets were to fall by 20%? As many of our portfolios are chosen by investors for their defensive nature, we would expect this test to show positive results, and they do. What interests us is the difference in magnitude of the effect on portfolios that tilt on the same factor.

While MTUM (see figure 1) would be expected to underperform the down market significantly, its momentum counterpart FDMO does not seem to be correlated with the market at all. At the same time, while high-dividend HDV would be expected to outperform by more than three percentage points, its counterpart would be only one percentage point ahead. Low volatility strategies appear to be the most defensive, but their expected level of outperformance differs, too. 

In terms of MTUM, the fund’s exposure to momentum is not the driver of the expected shortfall. That factor is actually expected to have a positive return should the market fall. Instead, MTUM’s return is expected to be dragged down the most because of its positive exposures to market sensitivity and volatility, both of which are positively correlated with the market and therefore expected to fall. FDMO has an exposure to market sensitivity that is much closer to zero and is therefore spared the impact of that factor. This difference illustrates the importance of understanding the underlying factor exposures in your portfolio of choice. 

What would happen to our portfolios if history were to repeat itself? For this set of tests, we chose some well-known historical events (see figure 2). The expected performance is based on the portfolios’ current factor exposures, and how those factors performed in the historic event. Note that specific risk is ignored (as it is in all of our tests). 

historical stress test expected active returns as of 30 april 2018

These well-known events mostly had a big impact on the market overall, although some, such as the bursting of the dotcom bubble in 2000, generally impacted just a segment of the market – but that same set of industries or exposures may be prevalent in some funds today. 

Whereas the direction of the expected active return for a given shock may not be surprising (for example, we would expect many of these funds to be defensive and therefore outperform when a big negative event occurs, or momentum portfolios may currently be heavily weighted in technology companies), what stands out is the magnitude of the expected return for funds that presumably target similar concepts. 

For example, while MTUM would be expected to lag the market by 10% should we have another big downturn in technology names, FDMO would be hurt much less, with an expected active return of a little more than half of MTUM’s. Similarly, the Quality ETF SPHQ would likely lag the market in an event similar to the Long-Term Capital Management crisis, but QUAL’s return would match that of the market. Or in the event of another market rout like the Lehman collapse, HDV (one of our high dividend yield ETFs) appears to be much more defensive than VYM. 

Shocking a Factor 

We then turned our attention to shocks in a number of the style factors that underlie our risk models. An investor in, say, momentum may be focused on the benefits of targeting that factor, but because most ETFs do not provide a pure exposure to a factor the investor may end up with other unintended, and perhaps unwanted, exposures, such as a large tilt against value. Because factor returns may be correlated, a big move in one factor can have a big impact on another. And some correlations can be large in magnitude. 

For these tests, we wanted to assume a factor shock big enough to have an impact, but not so big that it seemed implausible. Therefore, the factor shocks were chosen to be three standard deviations away from the long-term average, above for a factor whose average return was negative, and below for a factor with a positive average return (to reflect what would be assumed to be “bad” news). We used the model-predicted volatility for the factor at the end of April 2018 and assumed the return would unfold over the ensuing month. 

Positive returns for market sensitivity, size and volatility were associated with an up market, while other factors had little correlation with the market direction. 

The magnitude of the expected active return was relatively low for the momentum and quality ETFs, with the exception of the return to momentum portfolios when the momentum factor was shocked. Even there, however, the portfolios outperformed the underlying factor, owing to offsetting results from some of the other factor exposures they held. And some of the expected moves were in opposite directions for same-family ETFs (see exchange rate sensitivity shocks), or in the same direction but of quite different magnitude (the impact of shocking size or growth for the quality ETFs). 

The impact size was larger for the high dividend yield and low volatility portfolios in some cases. While expected returns were in the same direction, the impact here could also be quite different within the same type of fund (for example, market sensitivity and volatility for both types, and momentum for high yield). 

These results highlight two of the major conclusions we stated up front: a big factor move can have a large impact on a portfolio that says nothing about that factor in its name, and similar types of portfolios can expect a different effect from the same factor. 

Summary and conclusions 

Through a variety of tests that replayed history and stressed macroeconomic and factor variables, we have shown that a set of US factor-based smart beta ETFs may be vulnerable to big (and even not so big) market and economic events. 

In some cases, even funds that may be chosen for their defensive characteristics (such as low volatility or high dividend yield) may not offer as much protection against negative events as expected (although in many cases they probably would). In addition, funds that target the same factor may be so different from each other, even though their names suggest they would be quite similar, that their reactions to a given stress could differ substantially. 

We also showed that funds that tilt on one particular factor may still see a big reaction if there is a big move in a different factor, because they have not neutralised their other exposures and many factors’ returns are correlated. 

Finally, the size and direction of the expected active return related to a particular shock can change over time, with the differences in reaction a function of the current holdings, a factor’s volatility and/or changes in correlations among factors. 

Presumably these sensitivities are not confined to our small test set, suggesting that investors in smart beta (or other quantitatively driven) products may want to make sure they understand the underlying characteristics and vulnerabilities of those funds. 

A good stress-testing tool, employed periodically, can help investors understand these vulnerabilities and take appropriate action if desired. Fund managers should also be aware of these issues. Even if they do not change their methodology to address them, they will still be able to better understand the funds’ performance, as will the investors in their funds. 

Melissa R Brown is managing director, applied research, and Sebastian Ceria is CEO at Axioma