The ECB Systemic Risk Indicator

September 24, 2017

The European Central Bank has an indicator of systemic risk called, CISS. So what sort of signal does it send and what is it to be used for?

The
CISS is not supposed to be a predictor of systemic crisis. Apparently not even an indicator of systemic risk, instead it is to measure the state of the financial system, presumably in real time. So what is it to be used for? The ESRB recommends that it be used for determining whether the countercyclical capital "buffer should be maintained, reduced or fully released."

Is it a good indicator of the state of the system? Take a look at the annual average in the following figure.

On the day after the Brexit vote in June 2016, it told us that systemic risk was 0.3198, on a scale from 0 to 1. That was certainly much higher than the 0.1844 the week before, not to mention the 0.0577 at the start of 2016 when we seem to have been particularly safe. Fortunately, Brexit systemic risk is not as bad as it was in December 2008 when it hit 0.8391, worse than after Lehman’s failed in September 2008 when it was “only” 0.7091.

These figures look exact. The four significant digits tell us so: 0.3199 is much more accurate than mere 0.3 and suggest the ECB has a fantastically accurate measurement technology. But is precision realistic? As Warren Buffett once said:

“We don’t like things you have to carry out to three decimal places. If someone weighed somewhere between 300-350 pounds, I wouldn’t need precision — I would know they were fat.”

The 0.3199 is not more accurately measured the simply 0.3 would be, and the extra precision only serves to mislead.

Despite its apparent precision, there is a strange omission from the dashboard since it does not tell us anywhere what the numbers really mean. Presumably the numbers are ordinal so 0.3199 is worse than 0.0704 and better than 0.8301, but by how much? Does the dashboard allow us, or anyone, to do more than say "systemic risk seems to be higher (or lower) this month than last month", before we return to whatever we were doing before?

In other words, it omits any analysis of statistical significance. When I took my first course in statistics at university, the professor hammered into us the importance of significance testing. Create confidence intervals.

If the standard deviation of the numbers on the systemic risk dashboard is 0.1 the change in systemic risk from 0.1844 to 0.3198 on the day of the Brexit vote is not statistically significant. If the standard deviation is a 0.3, the Brexit systemic risk is not statistically different from the December 2008 number of 0.8391. If it it 0.5, the historically highest and lowest numbers are not statistically different from each other.

If we require our undergraduate students to produce confidence bounds, is it too much to ask the ECB to the same? How should we, or anyone, make use of the dashboard otherwise?

I could forgive all of that if the dashboard gave a warning, an indication that the risk was building up and a crisis might be on the horizon. It does no such thing. The next figure shows the indicator right until the crisis started in 2007/8:

In June 2007 it tells of the systemic risk is 0.0818. That is the month when the 2007/8 crisis started, when the quant funds got the difficulty and Northern Rock became unable to sell their mortgages. Indeed, the lowest observations of systemic risk are observed in the five years before the crisis started. It sent the signal that everything was fine and it was okay to take risk at exactly the same time when all the bad decisions that led to the crisis were being taken.

Meanwhile, the highest number is recorded in December 2008, after the worst was over. On average, 2009 is the riskiest year in the sample. Imagine you observe the risk of a dam bursting. When is the risk highest? Before it bursts or after? Of course before, because the dam can't burst twice. It is the same with the highest risk recorded in December 2008, it is simply wrong. The reason for this anomalous outcome is mechanical, how the CISS is measured. Even if it is obvious to all that the dam already burst.

If I then look at the latest numbers, in August 2017 it is 0.0518, which is close to its historical low. The world must be really safe.

The Systemic Stress Composite Indicator is systematically wrong in all states of the world. Too low before a crisis. Too high after a crisis.

If it actually will be used to determine whether the countercyclical capital "buffer should be maintained, reduced or fully released." the financial authorities would therefore be directly procyclical, allowing capital ratios to fall at the very moment they should be increasing them.

I think, the ECB could save itself a bit of money by taking out a subscription to the Financial Times, or perhaps the ECB's home newspaper, Handelsblatt. Because both the press and the system indicator react to market prices, at roughly the same time, the newspaper front pages will tell us a crisis is underway at the same time as the CISS.

---

---