tag:blogger.com,1999:blog-6976144462093297473.post3401287086067340915..comments2024-03-26T19:10:00.791-04:00Comments on West Coast Stat Views (on Observational Epidemiology and more): Homework and treatment levelsJosephhttp://www.blogger.com/profile/10760453165301871031noreply@blogger.comBlogger3125tag:blogger.com,1999:blog-6976144462093297473.post-41730824637569958782011-01-31T06:47:54.072-05:002011-01-31T06:47:54.072-05:00I think the worry about imperfect randomization is...I think the worry about imperfect randomization is even worse; only a tiny amount of cherry picking of "star students" could have a massive influence on the difference in the mean between the two groups.Josephhttps://www.blogger.com/profile/10760453165301871031noreply@blogger.comtag:blogger.com,1999:blog-6976144462093297473.post-46212482338374998252011-01-31T04:56:48.525-05:002011-01-31T04:56:48.525-05:00Jim,
"But isn’t the difference in homework a...Jim,<br /><br />"But isn’t the difference in homework assignment just one of many aspects of the “treatment” offered by the two alternative schools that comprise the difference under study?"<br /><br />Yes and no. The issue here is that certain treatments (or treatment levels) are only viable for certain relatively homogeneous sub-populations. This raises questions not only about generalizability but about the use of lottery-based analyses in these cases. If we assume that each school gives optimal homework for its student body (a huge 'if' but we're being hypothetical here), then the standard analysis would put this down as a difference between schools not students, even though switching samples would also switch the two schools' performance.<br /><br />To invoke a frequently-made point, in order for a lottery-based analysis to be really reliable, you need to keep the lottery losers in their own classes and not mix them in with the general population.<br /><br />Thanks for the comment,<br />MarkMarkhttps://www.blogger.com/profile/14705408455380402571noreply@blogger.comtag:blogger.com,1999:blog-6976144462093297473.post-52425873215376471132011-01-30T12:03:11.715-05:002011-01-30T12:03:11.715-05:00Thanks for the post.
You say that,
”Since the reje...Thanks for the post.<br />You say that,<br />”Since the rejected students will receive a treatment level that was optimized for the general population while the accepted students will receive a treatment level optimized to their particular subgroup, we expect the charter school students to do better, leading the lottery-based analysis to incorrectly conclude that there is no selection effect.”<br />But isn’t the difference in homework assignment just one of many aspects of the “treatment” offered by the two alternative schools that comprise the difference under study. <br />Your point does, of course (and as you’ve argued in other posts), go to the question of whether such a “homework-heavy” model has positive effects that could be generalized to any population beyond the scholarship applicants.<br /><br />Best,<br />Jim ManziUnknownhttps://www.blogger.com/profile/15955294294200773317noreply@blogger.com