Reply #30: Why S.&P.’s Ratings Are Substandard and Porous By NATE SILVER

[Demeter]

http://fivethirtyeight.blogs.nytimes.com/2011/08/08/why-s-p-s-ratings-are-substandard-and-porous/?utm_source=Budget+Campaign&utm_campaign=6a785474db-Balance_Sheet5_11_2011&utm_medium=email

Five years ago, if you were an investor looking for guidance on which country’s debt was the safest to invest in, Standard & Poor’s ratings wouldn’t have done much to help you navigate the headwinds of the financial crisis. Investors now think that Ireland has more than a 40 percent chance of a default or debt restructuring at some point in the next five years. The country is penalized with double-digit interest rates when it wants to borrow money. But in 2006, Standard & Poor’s had Ireland’s debt rated with its top-of-the-line, AAA rating. It didn’t downgrade Ireland until March 30, 2009, long after its financial problems had become obvious, and the price to buy insurance on its debt had increased tenfold from a year earlier...Spain, which markets now posit has about a three-in-ten chance of default or restructuring, also had a AAA rating, which it maintained until January 2009. Today it still has a AA rating, one notch higher than Japan’s...Iceland, the tiny country with the oversized banking sector that came perilously close to national bankruptcy, was in 2006 rated AA+, the same rating the United States now has...Greece, which now appears more likely than not to endure at least a technical default, had debt rated A, lower than most European countries but a reasonably good grade by world standards. It too was not downgraded until January 2009, and its bonds were still rated as investment-grade until March 2010.

Although Standard & Poor’s assigns ratings based on a series of letter grades, they can easily be translated into a numerical scale — sort of like the way that letter grades in high school are translated into a grade-point average:



This allows to test the reliability of the ratings in various ways, as well as to reverse-engineer them and see how the sausage is made. What factors is S.&P. looking at when it rates sovereign debt? A country’s debt-to-G.D.P. ratio? Its inflation rate? The size of its annual deficits?...S.&P. does look at each of these factors. But it also places very heavy emphasis on subjective views about a country’s political environment. In fact, these political factors are at least as important as economic variables in determining their ratings. For instance, the S.&P. ratings have an extremely strong relationship with a measure of political risk known as the Corruption Perceptions Index, which is published annually by Transparency International. These ratings have been the subject of much criticism because they are highly subjective, relying on a composite of surveys conducted among “experts” at international organizations who may have spent little time in most of the countries and who may instead base their judgments on cultural stereotypes.



I don’t know whether or not S.&P. looks at these ratings. But the fact that the two sets of ratings are so closely related is troublesome. It suggests that S.&P. is making a lot of judgment calls about countries they have no particular knowledge about. Keep in mind that even when it comes to the United States, S.&P. made a $2 trillion error that reflects their lack of understanding of the way that bills are scored by the Congressional Budget Office. Are we to expect that they add value based on their perceptions of the political climate in Kazakhstan, or Cyprus, or Uganda? Other factors that S.&P. looks at, which can be determined through regression analysis, include a country’s G.D.P., its inflation rate, its recent deficits and its long-term debt. But the subjective Corruption Perceptions Index is more closely related to the S.&P. ratings than any of these economic fundamentals. In addition, in 2006, S.&P. tended to rate European countries higher than others, even after controlling for all these other factors — something which has been especially problematic since, with some exceptions like Venezuela and Lebanon, countries in the euro zone now dominate the list of those most likely to default...None of this would be a problem if S.&P.’s ratings had performed well. But there is little evidence that they do. The next chart presents a comparison of S.&P. ratings as of June 30, 2006, to the risk of default five years later (on June 30, 2011) as measured by the prices of credit default swaps, financial instruments that pay an investor if there is a default on a bond obligation.



S.&P.’s bond ratings from five years ago would have told you almost nothing about the risk of a default today. They had no insight about the threats in European markets, nor about which countries in Europe were relatively more likely to default. (Norway, which remains among the most solvent countries in the world, had a AAA rating in 2006, but so did Ireland and Spain.) By comparison, simply looking at a country’s ratio of net debt to G.D.P. would have been a better predictor of default. It wouldn’t have done well by any means: it only explains about 12 percent of default risk. Still, this simple statistical indicator does better than the S.&P. ratings. (Nor is it the case that some combination of debt-to-G.D.P. ratios and S.&P. ratings does better than either one taken alone. Once you’d accounted for a country’s debt-to-G.D.P. ratio, the S.&P. ratings would not have improved your projections of default risk by a statistically significant margin.) Certainly, one might contemplate more sophisticated models than this (for instance, accounting for a country’s inflation rate in addition to its debt seems to be helpful). But when considerably more advanced studies have been published by academic economists like Carmen M. Reinhart, they have come to similar conclusions. Ms. Reinhart found that, although S.&P. rating changes have some value in predicting defaults, they are significantly outperformed by objective, statistical indicators....The fact that billions of dollars in wealth are tied up in the judgments of a company with such a poor record is all the proof you should require that the global financial system is in need of reform.