Commodity futures investors are not ‘evil speculators', finds Martin Steward. But do they enable the farmers who are?

Bryan Agbabian and Michael Ramirez, manager and product specialist respectively on RCM's Global Agricultural Trends fund, regularly tune in to a radio news and talk show for farmers in the US Mid West. "Today they were kicking themselves for selling their wheat too early," says Agbabian. Ramirez adds: "What you pick up is that farmers aren't worrying about the things my great grandfather worried about - looking out the window and wondering if it would ever rain. They're focused on macroeconomics and world financial markets as indicators of their future profitability."

Which is a bit worrying for a pension fund with a responsible investing policy that is active in the commodity-related financial markets. Shouldn't these farmers be basing their decisions on fundamental supply-and-demand dynamics, rather than trying to market time sentiment from institutional investors and hedge funds? Are they merely recognising that the tail wags the dog - that commodity-related financial markets like futures lead the spot prices for the commodities themselves? And if that is the case, will a preponderance of long futures positions from pension funds push the price of commodities beyond what is justified by supply-and-demand dynamics?

"I have been quite involved with one US public pension plan, the trustees of which were really struggling with this issue," says Jason Lejonvarn, commodities strategist with Hermes Fund Managers, which manages $2bn in index and enhanced-index commodity strategies for institutional investors. "It comes up a lot with public plans in particular."

Three questions need to be addressed. First, where does price discovery for commodities happen - in the spot price or the futures price? Second, are those prices driven by fundamentals? And third, are there too many financial speculators at work in these markets?

The first question is a key one for investors who, buying long-only indexed exposure for strategic diversification, are largely price agnostic. "If futures lead spot, then the key question is to what destination?" says Michael Gorham, the first director of the Commodity Futures Trading Commission's (CFTC) Division of Market Oversight from 2002-2004 and now industry professor at the IIT Stuart Centre for Financial Markets. "If traders are simply taking long-only positions as part of a naive attempt to include commodities in their portfolio, no matter what the price, then unless there are a sufficient number of fundamental traders to sell to them and keep prices close to fundamental value, there is a problem."

Spot and futures prices certainly exhibit strong correlation, for obvious reasons. If futures are priced higher than spot as the time for delivery approaches (that is, in contango), traders can short the contract, buy the commodity and pocket the spread when they deliver that commodity. But if everyone spotted this opportunity and started shorting the future, its price would move down towards the spot price - which is what tends to happen.

So information certainly flows - but in which direction? A number of academic studies show spot prices being discovered in futures markets, from Garbade and Silver's 1983 paper in the Review of Economics and Statistics (which included evidence on corn and wheat), through Brorsen, Bailey and Richardson's 1984 paper on cotton in the Western Journal of Agricultural Economics, to Crain and Lee on wheat price volatility in the Journal of Finance in 1996. These were brought up to date in June 2010 by Manueal Hernandez and Maximo Torero, in a paper for the International Food Policy Research Institute (IFPRI). Using the standard Granger tests for statistical causality, the null hypothesis that returns in futures markets do not cause the returns in spot was rejected at the 1% significance level for corn, soybeans and wheat.

That seems to make sense. There are three main inputs into the price differentiation between spot and futures. First, the market's expectation for (higher or lower) spot prices in the future. Second, the cost of carry: how much does it cost to store the commodity between today and the future delivery date? And third, the ‘convenience yield': how much value is attached to having the commodity right here, right now, rather than promised at some point in the future? In theory, the futures price equals the current spot price plus the cost of carry, plus or minus the expected spot price movement. If the market expects the future spot price to exceed the current spot price plus the cost of carry, demand (and therefore spot price) for the physical commodity goes up until the current and future spot prices net of carrying costs come back into line. This contango is generally characteristic of grain markets with healthy supply, and it enables a farmer to store grain and pay his cost of carry by rolling a short futures position - in theory, he need only sell his grain once the carrying costs are no longer covered by his return from selling the futures. This certainly feels like price discovery in the futures.

But what about the convenience yield? In a supply shock - when buyers will fight to have the stuff right here, right now - the term structure can move to steep backwardation as the current spot price rises so fast that it leaves the futures behind. The farmer can no longer pay his carrying costs by shorting the futures, leaving him with a naked long spot position, and growing pressure to sell with every tick upwards in the price. Remember those radio talk show wheat farmers kicking themselves for selling early? That begins to feel like price discovery in spot.

"You will see a flat curve whenever stocks-to-use ratios are at or above historic norms. As the ratio falls below the norm, you see an exponential increase in the price of the commodity and a massive spike in the convenience yield," says Lenjarn. "A classic example is cotton. Two years ago, it wasn't very exciting and the convenience yield was non-existent because it was in contango; now we have a massive shortage of cotton, stocks-to-use have gone down to unprecedented levels and the spot price has reached nominal levels not seen since the US Civil War. So there you are certainly seeing the reverse causality - the physical market fundamentals are leading the spot price."

The statistical analysis by Hernandez and Torero raises important questions, but it seems fair to conclude that while the term structure is in contango, prices are being discovered in the futures; while it is in shallow backwardation the same is probably true - after all, it appears to be the futures market's way of signaling to farmer-arbitrageurs that they should release more grain; but when it is in steep backwardation, price discovery moves rapidly forward on the curve.

That sounds like a good thing: when food is in short supply, price rises cannot be blamed on futures markets. But keep in mind the arbitrage opportunity that farmers enjoy in a contango market - while they can cover their carrying costs by selling futures, they may hold onto their inventory. If the curve moving into backwardation is their signal to release inventory, we would not want this signal to be obscured by excessive long-only speculation in futures: farmers could still hold their inventories cost-free, even as spot prices soared. This would be a case of price discovery happening in the futures at precisely the time when, in an efficient market, it should be happening in spot - and it is a theoretical corrective to the commonsense adage that ‘the best cure for high prices is high prices'.

Now, high spot prices do seem to be the best cure for high spot prices. As McGlone notes, agricultural products have been one of the lagging sectors in the S&P GSCI index in 2011 after a storming 2010. "As a farmer myself, anecdotally I can tell you that high corn prices are certainly an incentive to plant more corn," he says, "And you can bring a lot of supply on within a few months in a way that you can't in industrial metals or petroleum." But that is quite different from high futures prices being the best cure for high spot prices - they certainly are not, if grain producers are able to sell them to hedge their storage costs.

This is crucial: futures prices respond quite rationally to supply-and-demand fundamentals - they tend to go up as stocks-to-use ratios decline. But isn't one of the social functions of a responsible futures market to stand up every now and again to the real speculators - the producers - and say: "These prices are unjustified, and we will withdraw our liquidity until you release your inventory"? This is a delicate balancing act. Ideally, there would be enough net-long financial speculation in the futures market to allow producers to sell forward and therefore plant crops in the first place, but not so much that they are given a free-lunch arbitrage even during major supply shocks.

As the many academics who have studied these numbers observe, speculation (by both financials and farmers) can only be considered ‘excessive' in relation to total hedging demand. Nearly all of those academics - from the classic papers of the 1950s and 60s by Working, through those of the 1980s by Peck and Leuthold that focused on agricultural markets - found that speculative interest was inadequate to meet hedging demand.

The latest study in this area for agricultural futures is a 2008 paper by Sanders, Irwin and Merrin for the Department of Agricultural and Consumer Economics at the University of Illinois at Urbana-Champaign. This paper applied the classic ‘Working's T' ratio of speculators versus hedgers from 1995 until 2008. It not only found that speculative interest was quite low (there was about 8% more than the minimum needed to offset short and long hedging needs in corn and soybeans, and about 15% more in wheat), but comparing its findings with previous studies, it found recent ratios to be "about equal to or smaller than the averages from the historical estimates spanning the late 1940s to the late 1970s". The authors concluded that "speculation is barely large enough to meet total hedging demands".

Bringing the data set up-to-date, we can see that while total open interest in corn, wheat and soybean futures has soared over the past 10 years, the same cannot be said for either the non-commercial percentage of long-only open interest or the ratio of non-commercial longs to commercial shorts - with the exception of corn, perhaps reflecting its special role in biofuels.

"The ratio of non-commercial longs to commercial shorts has been very stable for many years," observes Hakan Kaya, a portfolio manager in Neuberger Berman's $1.8bn Quantitative Investment Group who speciaises in commodities and asset allocation. "For corn, wheat and soybeans, commercial short interest is almost always higher than non-commercial longs. But it's very difficult to say whether it's non-commercial longs leading commercial shorts, or commercial hedging demand that's leading the non-commercial longs."

Unfortunately, that might be the $64,000 question. It seems healthy that commercial interests appear able to use all of the liquidity coming from financial investors. But to conclude that that means speculative interest is only meeting hedging demand is too simplistic.

There is no evidence that more long-only futures interest pushes spot prices up, independent of supply-and-demand fundamentals. Livestock futures markets, which have tended to have the highest concentration of long-only speculative positions, have not experienced significant price momentum, whereas some markets with no or virtually no futures market, like durum wheat, edible beans, rice and milk, certainly have. Supply-and-demand fundamentals can push spot prices up, and over the medium term those same dynamics will ensure that those prices self-correct. And right now, we should probably take comfort from the fact that wheat is in steep contango, its inventories having recovered significantly since 2008, and corn and soybeans, with their tighter supply, are in backwardation along the majority of their respective curves (although not the near-end).

But it is a distinct possibility that during supply shocks, excessive long-only futures interest can enable commercial producers to speculate on the short-term spot price by selling futures short and holding back inventory. If that is the case, every socially-concerned long-only investor must ask if short-term price spikes and volatility are tolerable so long as long-term price movement reflects fundamentals.