American Elephants

Determining risk by using computer models is more complicated than you might think. by The Elephant's Child

Last Friday the U.S. National Snow and Ice Data Center admitted that their sea ice data was off by as much as a half-million square kilometers due to “sensor drift” of the old satellites that they were using.

A new satellite is available, and it is far more precise, but the Snow and Ice Data Center does not use it because its data is inconsistent with their historical data.  The idea of an “ice-free pole” is so much the consensus that they do not want ordinary facts to get in their way.

The extent of Arctic sea ice is an important measure of how rising or cooling temperatures are affecting Earth’s climate.  The satellite sensors caused scientists to underestimate the extent of sea ice by 50,000 kilometers (193,000 square miles) or about the size of California.  But this small error does not change their findings that Arctic ice is retreating.

An article from Anthony Watts blog by Machine Design editor Leland Teschler points out that financial institutions have employed legions of PhD mathematicians and statisticians to model the risks that the firms were assuming under a variety of scenarios.

Huybert Groenendaal’s PhD is in modeling the spread of diseases.  He points out that:

In risk modeling you use a lot of statistics because you want to learn from the past.  That is good if the past is like the future, but in that sense you couuld be getting a false sense of security.

That sense of security plays directly into what happened with banks and financial instruments based on mortgages.  It gets back to the use of historical data.  One critical assumption people had to make was that the past could predict the future.  I believe in the case of mortgage products, there was too much faith in the idea that past trends would hold.

In our experience people have excessive confidence in their historical data.  That problem isn’t unique to the financial area.  You must be cynical and open to the idea that this time, the world would change.  When we work with people on models, we warn them that models are just tools.  You have to think about the assumptions you make.  Models can help you make better decisions, but you must remain skeptical.

Climate change is similar to financial markets in that you can’t run experiments with it as you might when you are formulating theories in physics.  That means your skepticism should go up.

Lots to ponder here.  Read the whole thing.

%d bloggers like this: