When President Harry Truman asked to be sent a one-handed economist, he aptly summarised the conundrum that Andrew Lo sets out to resolve in his book, Adaptive Markets.

andrew lo

Truman was vexed by advisers telling him “on the one hand, this might happen,” but “on the other hand, that might happen”. While the influence of economists has increased since Truman’s fulminations – apace with the complexity of the mathematics finance theorists derived to support their models – the profession continues to invite the application of Truman’s drastic solution.

Today, the problem is defined in the tug-of-war between efficient market theorists, who assume investors and markets are rational, and behaviourists, who believe markets are fraught with inefficiencies but investors are too irrational to benefit these from anomalies.

Lo proposes a new framework that integrates both theories. His adaptive markets hypothesis posits that markets can be efficient when the environment is stable, but when markets are unstable investors resort to instinctive reactions, hard-wired over the course of human evolution. The result, Lo writes his introduction, is that “the wisdom of crowds is sometimes overwhelmed by the madness of mobs”.

Adaptive Markets seeks to reconcile “the apparent contradictions between the academic perspective of rational markets and the behavioural evidence to the contrary” into a consistent theory about how markets work, and why they can fail. The result is a compelling drawing on psychology, evolutionary biology, neuroscience, artificial intelligence, and other fields. Lo describes how the brain works, how humans make decisions, and how human behaviour evolves and adapts, and connects those dots to the building and bursting of bubbles. 

Lo says the financial crisis “illustrated the relevance of the adaptive markets framework and also some of the shortcomings of the existing paradigms that we had been using.” 

He continues: “I grew up in the tradition of efficient markets and was very much attached to it. Yet from time to time, market events would call into question some of the ideas.” At the same time, behaviourist critics of efficient market theory had not offered an alternative. For Lo, it takes a theory to beat a theory, so he took a dive into behaviourist theory: “I was sympathetic to what they were doing, but at the end, felt dissatisfied because they didn’t offer an alternative.”

Lo set out to construct a framework to explain the anomalies identified by the behaviourists. “It can’t be that all of us are so irrational all the time,” he said. “There’s got to be a reason for some of the market irregularities we see.”

This led him to explore neuroscience, the scientific literature on evolutionary biology, and artificial intelligence. His conclusion? “Our fear is a precision instrument.” Humans react much faster out of fear than the conscious mind is able to perceive – and the same neural circuits shaped in response to predators during hundreds of thousands of years of evolution are often triggered when humans are threatened in other ways. But the fight or flight response “almost surely won’t help you when the stock market crashes and your 401(k) turns into a 201(k)”.

In short, fear makes humans vulnerable in the market. Psychologists and behavioural economists, Lo argues, agree that sustained emotional stress impairs our ability to make rational decisions. 

Debating whether people are rational or emotional seemed unproductive. “A better middle ground,” Lo says, “is to understand how emotion and rationality coexist and interact.”

survival of the finches

The crux of Lo’s adaptive markets hypothesis is that markets are efficient when investors have had a chance to adapt to existing business conditions, and those conditions remain stable over a long enough period of time. He admits that sounds like the fine print of an insurance policy. The effect is intentional, because business conditions often shift and the notion of how long is “long enough” for investors to adapt to a given environment hinges on many factors.

Between the 1930s and the 1990s, a 60/40 equity/fixed-income portfolio “worked remarkably well” for many investors, Lo says. But, they faced a very special environment where the United States was growing, had a highly educated population, and business opportunities were developing very rapidly. The question is whether this will continue.

The practical implications of that question are significant for investors and their beneficiaries, as well as asset managers, policymakers and trustees responsible for overseeing the allocation of capital.

Lo’s adaptive approach suggest institutions develop an environmentally-based forecast that goes beyond traditional economic data. Investors must hazard conjectures about the evolution of complex factors like market volatility and expectations of risk premia over time. “Risk premia are not constant,” Lo says. “There’s no God-given right to earn 6% on equities year in and year out.” That recognition “means investors have got to be more dynamic at shifting their weight as a function of changes in the environment”.

In fact, Lo says, “the whole idea of risk premia is an evolutionary adaptation to a changing environment.” Instead of focusing on asset classes investors now focus on factor allocations. “Even as conservative an organisation as CalPERS has started to change the way they think about asset allocation, moving to a factor-based representation of their holdings. They recognise that what used to be called fixed income ends up having a lot of equity exposure, and if you invest in junk bonds, you’ve got a lot of equity beta. The language we’re using is changing and adapting to the fact that there are commonalities among asset classes that never existed 20 or 30 years ago.”

Hedge funds are the Galapagos finches of the investment world.  “There is an ebb and flow of species in any environment, and hedge funds adapt before your very eyes. Hedge funds do well for a while, out-compete each other, and end up losing a lot of resources because they can’t continue generating the performance.”

After unhappy investors withdraw, “a lot of hedge funds die off and the ones that survive are able to mutate and evolve and develop other strategies. This is the kind of dynamic that the adaptive markets hypothesis explains. You can’t explain it any other way. It doesn’t fit efficient markets and it doesn’t fit behavioural theories.” 

Andrew Lo is Professor at the MIT Sloan School of Management and Director of the MIT Laboratory for Financial Engineering