Suppose newspaper reporters brainstormed some headlines for an article on this experiment. These are their suggested headlines:I've read this over multiple times and have run it past an old professor of mine who teaches experimental design and we keep coming to the same conclusion. In a lesson on drawing inferences from experimental data, the authors seem to believe that 'causal' means 'deterministic."
A. “New Treatment Helps Pericarditis Patients”
B. “Colchicine Tends to Improve Treatment for Pericarditis”
C. “Pericarditis Patients May Get Help”
7. Which of the headlines above would be best to use for the article? Explain why.
Headline A would be the best because this is a well-designed experiment. Therefore, a cause and effect relationship has been established. Headlines B and C talk about a tendency relationship, not a cause and effect relationship.
That would be bad.
I have more concerns in my original post, and those represent only a fraction of the problems I found in the section. I recommend you take a look for yourself, and when you do, remind yourself that this is widely considered the gold standard of the new wave of instructional materials.
Wow--I clicked through and this example is really bad, in some ways no worse than standard intro stat, but that's the point, perhaps. I'll write something on this.
ReplyDeleteOne perspective that I haven't seen yet in this debate (and the Feynman quote is a good example of what I'm criticizing) is whether these are fixable errors (which they are), and whether they will be fixed (which we can't know yet).
ReplyDeleteFor some reason in technology we're prepared to accept that initial products are often very flawed and only get usable/interesting after a few revisions.
In the case of curricula and educational innovations, the conclusion seems to be: these are wrong and so the people who made them are stupid and so we need to throw them out and start over.
Looking at the arc of educational reform in the last 50 years, I can't say that this approach has worked well.
So I'd like to encourage in these posts to be slightly more constructive and consider: What would it take to fix this?
Kevin,
DeleteI am trying to work in more constructive posts (check this space for an upcoming MOOCs/MOO?s thread).
That said, it is important to realize that we are not talking about betas here. EngageNY and CC in general are the product of a top-down, rapidly implemented system with no effective feedback mechanisms. This is why mistakes like "negative times a negative is a negative" can go uncorrected for so long.
In other words, we need to get to the acknowledge-the-problems phase before we can get to the fix-the-problems phase.
(also posted at the teaching blog)
Kevin:
ReplyDeleteThat's an interesting point that you raise. Let me say this: as a statistician and journalist, of course I'm disturbed when I see these serious errors. But, beyond that, I'm bothered by these examples because of what seems like their official imprimateur. I'd prefer to have no official standards, or to have very bland and noncontroversial standards, than to have standards that are actively, aggressively wrong.