Use the Tornado Diagram to Focus on What’s Important

Use the Tornado Diagram to Focus on What’s Important

Thursday, 02 September 2010

In the Intelligent Financial Modeling group on LinkedIn, Geoffrey Kearney of Praedx asked, “If you could list the single greatest shortcoming of the forecasting process, what would it be?” Cue an opportunity to talk about tornado diagrams and how they answer the important what-if questions.

The single biggest mistake that a lot of people make is to insist on a single number for a forecast, a point estimate. Sam Savage calls it the Flaw of Averages. You must think in ranges of uncertainty for all the variables you put into your forecasting models and develop a full picture of risk and reward for the decision at hand.

Tornado Diagram

Typically, I’d start by assessing every input variable to the forecasting model with a range of three values: the 10th, 50th (median) and 90th percentiles. That is, there is a 10% chance that the true value will turn out lower than the “low”, 10% chance higher than the “high”, and the “base case” is the 50/50 point in the middle.

For binary variables (eg, success/failure of something) assess the probability directly and pick for the base case whichever scenario makes for the better discussion. For many drugs in development, for instance, failure is more likely than success, but it makes little sense to have failure as the “base case”. There is no revenue or future value for the model to show!

Sensitivity analysis is a matter of systematically varying each variable away from its base case to its low and high, one variable at a time, and seeing what happens to value. This is best viewed with a tornado diagram, where each bar, if we have been consistent with the 10-50-90 assessment, represents 80% of probability. The long bars at the top are therefore the variables we should focus on.

For developing a full probability distribution around value, in the past I have used decision tree software such as SuperTree to look at all the combinations of low/base/high. I have recently, though, become a fan of XLSim for Monte Carlo simulation. The DIST standard for encapsulating random scenarios allows for a thorough mining of the outputs to derive value-of-information and other calculations not easy to do with old-style MC software.

Comments (0)Add Comment

Write comment

busy

Search, View and Navigation

Additional Information