Despite the myriad rules and procedures of science, some research findings are pure flukes. Perhaps you're testing a new drug, and by chance alone, a large number of people spontaneously get better. The better your study is conducted, the lower the chance that your result was a fluke - but still, there is always a certain probability that it was.
Statistical significance testing gives you an idea of what this probability is.
This is not only an incorrect definition of the p-value but it also appears to be ignoring the possibility of bias and/or confounding. Even in a randomized drug trial (and drug trials are explicitly being used as an example), it is possible to induce selection bias due to non-random loss to follow-up in any non-trivial study. After all, many drugs are such that the participants can guess their exposure status (all analgesics have this unfortunate property) and this can lead to a differential study completion rate among some sub-groups. For some outcomes (all-cause mortality), complete ascertainment can be done using an intention to treat approach to analysis. But that typically induces a uniform bias towards the null.
I am always uncomfortable with how these strong and unverifiable assumptions are glossed over in popular accounts of pharmacoepidemiology.
No comments:
Post a Comment