Why do we rely so much on models when we know they can't be trusted?

February 26, 2016

There a lot of evidence that models are less than perfectly reliable. Why then do we rely so much on models in decision-making, and especially financial regulations? Because there are three types of people: Believers in true model, skeptics who accept model risk and nihilistic rejectionists.

The received wisdom has that a major contributor to the crisis from 2007 was the excessive reliance on models that weren't exactly good. A good early take on it is What happened to the quants in August 2007?. A large number of various pieces from government authorities make similar points. Some of my take on it is Complexity kills, why risk is hard to measure and model risk and a lot of other people have made similar points.

So, why do regulations rely so much on models, much more now than in 2007?

I think there are two reasons for this.

First, there are three types of people in the world.

1. Those who treat risk forecast as an approximate truth, and use them to allocate portfolios and set risk limits and regulate without too many questions asked. This is certainly the view of the financial regulators and many a risk controller and internal auditor. These are the people who must believe in the one true model;

2. The healthy skeptics, often risk managers, who use a statistical risk measure as an indication of the underlying risk, one indication amongst several. The risk manager might run several different models in parallel and augment that with non-model information;

3. The nihilist, who rejects statistical risk forecasting altogether.

To me, the only sensible approach is the second.

Why then do the authorities more or less fall into the first category? One reason is discussed here.

---

---