Tuesday, April 13, 2010

The ongoing tyranny of statistical significance testing in biomedical research

This is the title of an article that appears in the Europen Journal of Epidemiology. I think that it is a good contribution to the recent discussions on p-values. The authors seem to be focusing on the difference between clincial significance and statistical significance (and pointing out the many cases where the two may diverge).

Usually this is happens for strong effects with small samples sizes (real associations are hard to establish) and weak effects with very large sample sizes (where unimportant differences can be demonstrated).

I was surprised that the authors did not consider the possibility of observational meta-analysis -- which seems to me to be one of the possible ways to handle the issue of small sample sizes in any specific experiment. But I think the message is in concordance with the larger mesage that these tests are not a substitute for personal judgement. In the context of designed experiments they may be fine (as the experiment can be created to fit this criterion) but the uncritical use of p-values will also be an issue in the interpretation of observational research.

No comments:

Post a Comment