Prediction is difficult

There is a really thoughtful post in the Economist. The gist:

In a nutshell: I've become far less confident about our ability to accurately describe possible outcomes more than a decade out. Correspondingly, I've become increasingly sceptical of the value of analyses of decisions now that attempt to assess the costs and benefits of action over horizons any longer than a decade.

I think that this was a very good complement to yesterday's discussion of inference from observational medical research.  Models are hard.  The more complicated the model is, the more likely something is to go wrong.  Future predictions suffer from these sorts of complications -- we honestly do not know what the circumstances will be like in the future or how many unlikely events will actually happen.  Over the short run, predictions can bank on it being unlikely that a lot of "low event rate but high impact" events will happen.  We can also neglect the slow (but incremental variables) that are currently unnoticed but which will make a huge difference in the future.

In the same sense, looking at low event rate outcomes in incomplete data (most of pharmacovigilence), leads to a lot of innate uncertainty.  In both cases, I think it makes a lot of sense to be humble about what our models can tell us and to focus on policy that accepts that there is a lot of innate uncertainty in some forms of prediction.

Hat-tip: Marginal Revolutions