tag:blogger.com,1999:blog-6976144462093297473.post2278908544975396542..comments2024-03-26T19:10:00.791-04:00Comments on West Coast Stat Views (on Observational Epidemiology and more): "Ignore the parts about crystal meth and pancakes"Josephhttp://www.blogger.com/profile/10760453165301871031noreply@blogger.comBlogger1125tag:blogger.com,1999:blog-6976144462093297473.post-12139409697849452602010-10-01T18:16:35.869-04:002010-10-01T18:16:35.869-04:00I also read article after article about research s...I also read article after article about research supposedly proving the effectiveness of one educational innovation or another, thinking all the while, “But did they control for the self-selection effect? Did they distinguish causation from correlation?”<br /><br />They almost never do. A case in point is the Clifford Adelman study “Answers in the Toolbox,” with which I had a run-in a few years back when I was teaching high school math in Arizona and my district was pressured to institute certain kinds of AP classes because that study had given rise to this federal grant program: <br /> <br />http://www2.ed.gov/programs/apincent/index.html<br /><br />which had given rise to this Arizona grant program: <br /><br />https://www.azed.gov/accountability/apip/. <br /><br />I just looked up the Adelman study today and found this description of it at Kathleen Plato´s site <br /><br />http://www.newhorizons.org/spneeds/inclusion/teaching/plato.htm :<br /><br /><b>AP is a Predictor of College Success </b><br /><br />"In 1999, a U.S. Department of Education study by Clifford Adelman titled "Answers in the Tool Box" examined the factors which could be used to predict college success – the attainment of the Bachelor's degree. Adelman was not examining factors that supported college admission; his research was focused on college completion. The study concluded, "No matter how one divides the universe of students. . . .a high school curriculum of high academic intensity and quality is the factor that contributes to a student's likelihood of completing a college degree." Courses such as AP and IB outranked, grade point, class rank, SAT scores as contributing factor. Furthermore, rigorous high school courses were shown to mitigate the effects of socioeconomic status. Just one AP course exposes a student to college-level work. Even if the examination is not taken or passed, the challenge of the course and the emphasis on critical thinking, study skills and increased content knowledge prepares a student for college work."<br /><br />Note that last part especially – the implication that it has somehow been proved that even students who do not take or pass the AP exam are more likely to complete college just by virtue of having enrolled in an AP class.<br /><br />The Adelman study did not prove that; there was nothing in it that in any way broke out students who were enrolled in AP but didn´t take or pass the test. (It´s been a while since I read it, but I don´t remember that it even quantified how many such students there were.) There was nothing in it that controlled for the two sorts of selection I would be willing to bet were in play among the AP-enrolled students: (1) They were motivated and willing to work hard and (2) In all likelihood there were prerequisites for enrolling in the AP classes in the first place, which may have well included minimum scores in previous classes. If some disaster on day 2 of the AP class had prevented it from going forward, those students enrolled <i>still</i> would have completed college. <br /><br />MY COMMENT IS OVERLONG SO I AM CONTINUING IT AS A COMMENT TO YOUR OCTOBER 1 POST.Elizabethhttps://www.blogger.com/profile/13614491408921240125noreply@blogger.com