Friday, February 19, 2010

Multiple Testing

Interesting. False positives in popular fields appear to be much more strongly driven by the number of groups testing these hypotheses rather than by fiddling with data. A very comforting result, insofar as it is true.

More troublesome, is that it is unclear what we can do about it. Being better about publishing negative results helps but is never going to be a perfect solution; especially when reviewers may be more skeptical about results that do not match their intuition.

2 comments:

  1. And by intuition you mean pre-conceived conclusions. If you'd collected positive data under identical circumstances you sail through peer-review.

    ReplyDelete