And yet behind every complex set of rules is a paper showing that it outperforms simple rules, under conditions of its author’s choosing. That is, the person proposing the complex model picks the scenarios for comparison. Unfortunately, the world throws at us scenarios not of our choosing. Simpler methods may perform better when model assumptions are violated. And model assumptions are always violated, at least to some extent.One of the hardest things with simulation studies is that we get to develop our own set of assumptions. So we actually know how to correctly model the phenomenon of interest.
The problem is that we usually do not know how much error is introduced when the complex (and often non-linear) model fails. On the other hand, it is amazing how far one can get with a clear set of rules of the thumb.
I wonder if it would be better if we had a different person test the model than the one who proposed it?
This comment has been removed by a blog administrator.
ReplyDelete