stress test

The NYT tells us that bank regulators are examining the financial condition of 18 of the largest banks. Treasury Secretary Geithner says the regulators are going to do stress testing on the banks’ assets, telling us that the analogy is to the medical term. Dana Milbanks thinks this is funny, like because we’re all feeling so stressed out, so we need a stress test. It turns out that stress testing is a standard part of risk evaluation in financial institutions, and has a completely different and technical meaning.

Banks are in the business of taking in money in deposits and lending it out again, so when we think about the bank’s balance sheet, we would expect to see cash and loans. But it’s not that simple. One of the largest categories of assets of the balance sheet of a perfectly good community bank is securities bought and held for sale. This asset is about 14.2% of the total assets of the bank, and it isn’t just sitting there waiting for the market to go up. When we look further into the financial statements, we see the level of activity in the portfolio: during the 9 months ended September 30, 2008, about half of the portfolio turned over.

That was a small community bank. The Citibank trading account is $ 457.5bn at the same date and includes a huge portfolio of bonds, stocks, and derivatives, including credit default swaps. By the way, Citi trades that portfolio aggressively. The traders are the people demanding those crazy bonuses: they argue that they create the revenue and they ought to get a share no matter how badly the rest of the bank does.

Every bank wants to make sure it is taking special care with this portfolio, because the securities markets are risky. One of the main tools for managing the risk is called value at risk, usually shortened to VaR. Joe Nocera wrote an entertaining description of the theory here. One of the problems with VaR is that it relies on a form of the bell-shaped curve for its evaluation of risk. Nocera explains that VaR evaluates risk for 99% of the time, but it has no way to see what will happen the other 1% of the time:

The fact that you are not likely to lose more than a certain amount 99 percent of the time tells you absolutely nothing about what could happen the other 1 percent of the time. You could lose $51 million instead of $50 million — no big deal. That happens two or three times a year, and no one blinks an eye. You could also lose billions and go out of business. VaR has no way of measuring which it will be.

In part, this is true because VaR assumes that things are “normal”. One aspect of normal is the specific time frame for historical market data, and that tells us what is considered normal. Nate Silver gives us a simple example in Esquire. If we use a time frame of the last 60 years, we estimate the likelihood of a crash at 3.17%. If we use the last 20 years, we get .04%.

Stress testing is supposed to help solve the problem of what happens the other 1% of the time. It changes some of the risk factors in the VaR calculation to match some other hypothetical, like the crash of the Asia markets, or the sudden drop in the market in 1987, or the big oil spike of the late 70s or something like that. The literature describes several of such tests. I like this one from the European Union, starting at page 15.

Stress testing is expensive, and it suffers from two interesting defects. First, portfolios are evaluated using mathematical models, including the not-really-erotic Gaussian Copula. These models, in turn, depend on other equations which describe the risk in a specific investment. When a stress test is done, computer people fiddle with those equations, changing one or more parameters. That will work fine if the original equations make sense. But, if not, the new equations are junk, like the originals, or worse.

Second, it turns out that applying the tests gives an answer, but not a probability that the tested scenario will occur. The humans reading the results have to decide how to evaluate the risk of occurrence. This kind of evaluation is hard for humans, as Nate Silver described in the linked article.

Maybe we all need a cardiac stress test, after all.