Friday, March 27, 2009

The Problem With Models

I’ve meant for a long time to post on the problems with using financial models, but there’s just too much to say, and too much that has already been said more clearly than I can say it. Here are some links on this topic:

Data series too short – for example, extreme economic conditions are not captured in the model. See Underestimating the tails, at  Revolutions.

Overreliance on past patterns – assuming the future will be like the past. See Maths and markets at FT.com.

Bad assumptions – for example, housing prices won’t fall. See Don’t Blame the Quants, Felix, at Falkenblog.

Network externalities – for example, failure of your counterparty’s counterparty was not considered in your model. See Andrew Haldane’s “Why Banks Failed the Stress Test” paper starting at page 9.

Failure to adjust models for evolving conditions – see John Kay, “Financial models are no excuse for resting your brain”.

Failure to properly account for low probability events – see Naked Capitalism, More on Global Alpha, Quant Woes, and Joe Nocera, “Risk Mismanagement”.