The risk you measure is probably not the risk you care about

March 1, 2025

The risk we measure is usually not the risk we care about. Why and does it matter?

When measuring financial risk, are we capturing what is actually important?

So what matters? The biggest investments most of us make are real estate and pensions. Then, the most relevant risk is the value of those investments years and probably decades into the future.

It isn't that different for financial firms, such as banks and insurance companies. For them, the most important risk is the existential — bankruptcy — followed by the chance of large losses over the next years and decades.

The problem is that such a risk is rather difficult to quantify. We have scarce data on such events, and the few observations we have tend to be inconsistent, irrelevant and misleading.

The starting place for how to risk manage what matters is the objective.

So, what is the objective of most investment strategies and risk management? The following picture frames the problem. It shows the distribution of aggregate outcomes for the problem we are considering, from the worst to the best.

The following picture shows what we want to achieve. We want to reduce the likelihood of bad outcomes by thinning the lower tail and fattening the upper tail to increase the chance of good outcomes.

We then face a problem. We have plenty of data in the middle of the distribution but practically no data in the tails. In other words, we know plenty about a problem we don't really care about, the middle of the distribution, but very little about what matters to us: long-term gains and extreme losses.

If we are fortunate enough to have high-quality asset managers, we end up widening the area over which we exercise control. Becoming better able to reduce the risk of bad but not disastrous outcomes, and similarly, achieve moderate gains.

What tends to happen in practice is for practitioners using the middle of the distribution to inform us about what happens in tails.

The obvious way to accomplish that is to use Value-at-Risk, Expected Shortfall, CDS spreads, and their ilk. They certainly will allow us to model the middle of the distribution reasonably accurately.

And when we have a mathematical distribution, we can plug in any possible combination of probabilities and outcomes you want. Now we can say, "Ah, now I can measure risk over 10 years or 50 years or a decade or a millennia". The technical name for this is probability shifting.

We get a number, but is that number accurate? No. It is not accurate because data do not inform the estimation. I called this risk model hallucination last year.

You might as well fire up your Excel and type in rand () to get your risk estimate.

I have never seen anyone make a good argument for why these market risk measures provide a reliable quantification of the most important risk. Plenty say it is true. I related this to scientific socialism a few weeks ago.

And that is why the risk we care about is usually not the risk we measure.

One example of The Illusion of Control.

---

---