*Among undergrad college students
[Following up on Joseph's post]
I'm know this sounds like a joke but it's really not. It's one of those ideas you learn as a sophomore then probably forget even though it's a potentially major issue that crops up frequently.
Whenever you see a claim that age or exercise or diet or whatever isn't a substantial/significant driver of something, there are all sorts of distributional assumptions lurking under the surface. The more homogenous the data set of a study is with respect to a certain variable, the less likely you are to find evidence of that variable causing anything. This is a big concern because an alarming amount of research is based on groups far less diverse than the general population. Remember the old joke about experimental psychology being a discipline built on the study of lab rats and college freshmen.
On top of that, even if a causal relationship has a trivial impact on the general population today, that impact can grow in the future if the population shifts and distributions change. The reverse can happen as well, though that's usually easier to see coming since you start out with a known relationship.
Like I said, this is all stat 101 stuff, long internalized by most of you reading this, but it's also one of those obvious/not obvious points that is almost never spelled out explicitly, and that's a mistake on our part.
No comments:
Post a Comment