It is difficult to overstate the importance of sovereign credit ratings; investors rely on them, either because they want to or they have to, or both. It is estimated that a tad more than $50trn (€36.5trn) of global savings invested in sovereign bonds may be guided directly or indirectly by the views of the three largest credit rating agencies (CRAs) – Moody’s, S&P and Fitch – which account for roughly 95% of the market.

It is, therefore, crucial that the behaviour of these agencies is continually scrutinised. Together, the so-called Big Three have a special status, written into law in both the US and Europe; regulations require some institutional investors to hold only investment-grade debt. Consequently, rating changes across the investment-grade boundary can have a significant effect on yields as they change the number of potential buyers.

CRAs make rating decisions, which rely – according to them – on a combination of measurable (or objective) and judgement (or subjective) factors. In the past there was much  opacity surrounding their rating methodologies, and staff were reportedly sworn to secrecy. Recently they have made some small steps to become more transparent. Consequently, they have started publishing some information on the fundamental factors that they employ to rate sovereigns, together with a (still opaque) description of the subjective steps they take to ‘enhance’ the signal they receive from these fundamental components – necessary because, they argue, some factors are impossible to quantify. (Typically, CRAs will gather the information from the fundamental signals and then they will proceed with a number of meetings to decide upon the final rating based on the judgement of their analysts.)

In a UniCredit paper I wrote recently with Daniel Vernazza and Erik Nielsen, we set out initially to measure the implicit weights that CRAs assign to a set of fundamental variables (the objective component), alongside the extent of the subjective component that highlights the degree to which their committees overrule the signal from the hard data. 

Our methodology is based on modern panel data estimation techniques, with ratings information for more than 100 countries over more than 10 years. More specifically, we regress the sovereign ratings (after converting them from alphanumeric ratings to numeric ones) on a number of fundamental factors to reveal the implicit weight assigned to each one of them by the agencies. Our choice of fundamental factors was guided by the information provided by CRAs themselves, as well as basic economic intuition. 

The list of variables we ended up using is presented in figure 1, which also shows the means of these measurable fundamentals by Moody’s rating category. The aggregate ‘objective’ component of the sovereign rating is then simply the weighted sum of the fundamentals.

Averages by rating category

Where we believe our analysis adds the most value is in measuring the ‘subjective’ component, which is computed as the difference between the actual rating and that implied by fundamentals. Put simply, what cannot be explained by fundamental factors can be attributed to the subjective decisions of the CRAs. In other words, it is the rating committees’ override (or adjustment) of the objective signal they receive.

Overall, our results suggest that typically 75-82% of the rating decisions are explained by fundamentals, leaving the remaining 18-25% to the subjective component (depending on the exact specification of the econometric model). 

Given the importance of sovereign credit ratings in the decision making of capital allocation by investors, we feel that this finding is very significant. It illustrates that, frequently, fundamental signals are overruled and the final rating is much influenced by the judgemental factor. 

Now, intuitively, there is nothing strange or inappropriate in adding judgement to the objective signals: it is conceivable (though not immediately obvious) that CRAs’ rating committees have considerable insights and expertise that enables them to rank sovereigns more accurately than would a simple fundamental model. This led us to our second investigation.

Ultimately, the usefulness of credit ratings depends on their ability to predict actual default events. What we established previously is that this rating is the outcome of an objective and a subjective component. In our view, the important question is to what extent each component helps in forecasting the probabilities of default. In addressing this question, we used data on the 26 default events in our sample since 1997. Our results – based on formal estimation techniques – were profound.

We found the objective component is highly statistically significant in predicting sovereign defaults in up to five-year horizons. However, the subjective component (the rating committees’ overruling of the macro signals) adds no value to the predictability of default until less than one year before the actual event. The latter is illustrated in figure 2, which shows that there is little difference between the subjective components assigned to those that default and those that do not over a two-year horizon. This is staggering, considering that credit rating agencies claim their assessment horizon is around two-to-three years ahead.

Subjective component

These results suggest a frightening conclusion: although the subjective component is found to be nothing else but noise on the aggregate, at the individual country level it matters and can have a distorting effect. This means that history is littered with sovereigns having been overrated and underrated by the subjective component overruling the objective component from the macro fundamentals. Since such misvalued ratings guide trillions of dollars of assets, the distortionary (subjective) tweaking of the objective signals typically has s significant impact on capital flows and asset prices. 

The EMU crisis and current misalignments

Of particular interest is what role the CRAs’ subjectivity played during the 2009-11 euro-zone debt crisis, when the euro-zone periphery was sharply downgraded. Figure 3 shows the subjective component of ratings for Italy, Spain, Portugal, Ireland and Greece from 1989 to 2013.

Strong dislike

For these countries, the subjective component was, on average, slightly positive in the first decade of the euro. But then between 2010 and 2011 the subjective component turned sharply negative, on average subtracting from the objective signal three notches by end-2011, and then six notches by end-2012. This deeply negative subjective component is much bigger in absolute value than almost anything else seen in our sample. (An important exception where a large negative component now appears to have been justified is for Greece. That country is a special case because it lied about its accounts. Excluding Greece, on average the periphery was still rated five notches below its objective rating at end-2012).

The rating committees’ overruling of the objective macro signals contributed to distortions to capital flows on a magnitude that became systemically dangerous. To illustrate, following the downgrades of the euro-zone periphery, banks reporting to the BIS reduced their exposure to the euro-zone periphery by a whopping $1.4trn (€1trn or 43%) between the end of 2009 and Q3 2013. Apart from clearly contributing to the depth of the recession in the periphery during these years, and hence unprecedented social hardship, the downgrades and withdrawal of capital appear to have become a virtually self-fulfilling spiral that was only broken by the ECB’s introduction of Outright Monetary Transactions.

Eventually ratings should catch up with those suggested by fundamentals as the rating committees realise their errors. Indeed, they have already started backpedalling by upgrading some of the euro-zone periphery countries – although by very modest steps, and they have a long way to go, as we estimate that peripheral countries are now rated, on average, four to five notches below what their fundamentals would suggest. This is likely to take at least another couple of years – pride is an issue – but it will eventually lead to further normalisation of the massive capital allocation distortions that were produced because of the CRAs’ subjective mistakes.

In light of our findings that suggest a meaningful and distortionary component in CRAs’ sovereign ratings, we suggest that these agencies be stripped of their regulatory powers. We suggest that an international body be mandated with the development of a set of macro indicators and that these indicators replace the role in regulatory and other policy matters presently played by the CRAs.

Failing that (and it would be a failure) the ratings agencies should be forced to substantially increase transparency, including publishing a separate breakdown of the objective and subjective components of ratings, the minutes of the rating committees, and the voting records.

Vasileios Gkionakis is global head of FX strategy at UniCredit Research. This is a shorter version of ‘The Damaging Bias of Sovereign Ratings’, by Gkionakis, Erik Nielsen and Daniel Vernazza, published in April 2014 as part of UniCredit’s Global Themes Serie