How are managers deploying natural language processing to analyse management sentiment in earnings calls?

Key points

  • Investment managers are constantly searching for new AI techniques that can create new data sources or harvest better insights from traditional data
  • ‘Natural language processing’ enables managers to identify likely outperformers from cues embedded in earnings call Q&A sessions
  • Consultants advise that investors evaluate the materiality of AI techniques to any given strategy and monitor the efficacy of AI tools to ensure widespread adoption does not dilute impact

The phrase ‘tell me what you really think’ is typically employed as a witty rejoinder to lower the temperature of a conversation when someone has been holding forth in a particularly emphatic fashion.

Now quant investors want to harness that verbal energy to tell what a speaker is really thinking about a topic – and if a CEO or CFO is talking about a company’s earnings or a new product, gleaning such insights from supposedly well-scripted discussions could help investment managers decide whether to buy a stock, hold, or sell.

Investment managers are using and exploring numerous ways to improve investment performance by using digital technology to process large data sets. One of the most widely-used is natural language processing (NLP), a range of artificial intelligence (AI) techniques used to analyse textual material. One use of NLP is to discern the relative strength of sentiment conveyed by different ways of a expressing a thought by assessing the use of words that convey certainty or hesitancy.

Sentiment analysis, originally used to have computers rapidly read earnings statements, is being focused on the Q&A section of earnings call transcripts. Why focus on the Q&A? In short, the discussion following the delivery of prepared remarks is more spontaneous, allowing NLP tools to get a better read on a speaker’s natural delivery style and detect deviations that might indicate the speaker’s underlying emotional response to the content of the discussion.

Virtually all senior executives delivering company results on a conference call with analysts and investors tend to project a confident picture of their performance. But there are notable differences between the more and the less certain, and AI tools can help investors tell what an executive really thinks.

screen shot 2019 05 31 at 14.35.11

“There is ample research that shows analysts tend to make revisions in the right direction and this can be predicted based on their prior revisions,” says Michael Fraikin, the Frankfurt-based global head of research for Invesco Quantitative Strategies. “Sentiment analysis allows us to predict future analysts’ revisions more accurately than the analysts’ themselves. There are subtle messages conveyed during the calls that analysts are not fully aware of at the time.”

To develop its sentiment strength signal, Invesco worked with linguists to develop several text-mining dictionaries that assign words a positive or negative value, along with an intensity score, for context-specific words used in the financial industry. “We applied considerable effort to refining the dictionaries,” Fraikin says (see In the quant world of AI investing, words matter). “Standard NLP dictionaries designed to evaluate written texts do not contain enough ‘emotion’ words,” he explains. There are additional issues: the word ‘question’, for example, is typically coded as a negative sentiment, but in Q&A transcripts where managers frequently say ‘thank you for your question’ the negative value of the word is greatly reduced. Earnings call analysis ignores moderators; it is the sentiment of company management that is in focus.

There are two important and distinct innovations happening in terms of new sources of data, commonly referred to as big data, and new techniques relating to artificial intelligence, says Yazann Romahi, chief investment officer, quantitative beta strategies, JP Morgan Asset Management. “The terms are often used interchangeably when they are in fact separate and it’s important to distinguish between the two,” he says. You can, for example, use traditional statistical techniques on new sources of data. Likewise, AI techniques can be used on traditional and new sources of data. 

Invesco’s sentiment signal, Fraikin says, “belongs in the camp of the momentum factors”. But although sentiment strength is correlated with other momentum factors, Invesco’s indicator functions in a very specific way, he adds. “For at least 70% of the universe, the signal provides no meaningful differentiation, which is different from the other momentum factors,” he explains. “The signal tells you something about the edges of the investment universe, about the stocks that are most attractive and least attractive in terms of sentiment.” For those stocks, the sentiment signal helps predict the likely positioning of a company relative to peers – if the signal suggests a positive revision, for example, the company is likely to do better than its peers.

Risk reduction
Artificial intelligence tools can also be used to mitigate risk, Romahi says. The JP Morgan IM quantitative beta team uses NLP to identify stocks that are subject to event risk, so those stocks can be excluded from portfolios seeking to capture the effects of specific factors such as value and momentum. The share prices of companies that are the subject of M&A rumours or an activist campaign, for example, can respond to news in an up-or-down manner that would dilute their factor-based performance. 

“We don’t want to have that sort of binary risk in a factor portfolio,” Romahi says. “We use an NLP algorithm to sift through news to look for stocks subject to binary events, as a way of reducing idiosyncratic risk.” Like most things in investing, there is a tradeoff. “The cost of being wrong is very low compared to the improvement that results from removing idiosyncratic risk from a factor portfolio.”

Ensuring how an AI signal will perform in a portfolio is a key consideration, says Andy Iseri, senior vice-president and non-US investment consultant in the global manager research group at consulting group Callan. Iseri says: “The role AI plays depends on the strategy, and many AI signals are fast moving. AI is certainly more prevalent at quant firms, but we are seeing fundamental shops conducting AI research,” he adds. 

screen shot 2019 05 31 at 14.38.22

Callan assesses the “materiality of AI compared to the value of good old fundamental factors like value, growth and earnings”. But the primary concern with AI techniques is the ability of signals to “maintain their efficacy as adoption increases”, as Iseri puts it. Quant managers have been devising techniques to detect factor crowding and to monitor the efficacy of factors since 2007, he adds. While the benefits of widely pursued factors have not yet been arbitraged away, “if everyone is processing earnings transcripts immediately, for example, will that change the behavioural patterns that we see in the market?” he asks. One thing is certain, he says: “This is a fascinating area.”

Only time will tell. But managers are constantly searching for new AI signals. Invesco is planning to implement its sentiment strength indicator as a small additional weight in the momentum compartment of multi-factor portfolios later this year, and is already working on the next stage of its research – including tracking individuals as they move from one company to another.

It may not be long before CEOs and CFOs enrol in linguistics training to be sure they are saying what they really think.