"No, I said 'self-select for attitude.'"

One of the many weird aspects of the charter school debate has been the non-exchanges over selection effects. Critics suggest that students who choose to go through the application process will tend to be more motivated, more committed to school, more interested in college, and more open to the idea of a heavier work load and will tend to have more stable, supportive families.

Here's an example:

Some observers, such as the authors of The Charter School Dust-Up, say that KIPP's admission process self-screens for students who are both motivated and compliant, from similarly motivated and compliant—and supportive—families. Parents must commit to a required level of involvement, which rules out badly dysfunctional families.

This would seem to be fairly straightforward. Almost every school could do a good job if it were populated solely by students who worked hard, followed instructions and had the full support of their families. A strong selection effect here could explain away most or all of the positive results observed in KIPP schools. This hypothesis even raises the possibility that KIPP schools are actually doing worse than public schools would under comparable circumstances.

(You'll notice that no mention is made of race, poverty or test scores.)

Now here's how SRI responds:

KIPP schools’ higher-than-expected test score results draw both attention and claims that they “cherry-pick” high-achieving students from poor neighborhoods. This is the first report to closely scrutinize the praise and criticisms associated with KIPP, as well as key challenges facing Bay Area KIPP schools today.

In the three KIPP schools where they were able to draw comparisons, SRI researchers found that students with lower prior achievement on the CST were more likely to choose KIPP than higher-performing students from the same neighborhood, suggesting that, at least at these schools, cherry-picking does not occur.

And here's the much-touted Mathematica report:

Our nonexperimental methods account for the pre-KIPP (baseline) characteristics of students who subsequently enter KIPP schools: not only demographic characteristics such as race/ethnicity, gender, poverty status, and special education status, but also their achievement test results for two years prior to entering KIPP. We examine the achievement levels of the students for up to four years after entering KIPP schools. In our preferred models, we compare these trajectories to the achievement trajectories of a matched set of local public school students who have similar achievement test results and demographic characteristics in the baseline period (typically third and fourth grades) but who do not enroll in KIPP.