A Partial N-space of Eduction Reform -- another preblogged footnote

If we are going to have an intelligent conversation about education (which at this point would be a refreshing change of pace), we have to start by thinking about the n-space. There are multiple dimensions that have to be considered here. As long as the debate fails to acknowledge them or approaches them in a sloppy way, the analyses will continue to be fatally flawed.

We could look at this on the level of classes or individual students, but in this case it probably makes the most sense to think of each school as representing a point in this multidimensional space. We assume these points are more or less fixed with respect to some of these dimensions (grade level, population density [rural/suburban/urban], demographics, region, etc.) but we like to believe that we can change the position of these points with respect to other dimensions (retention, discipline, standardized test scores, etc.).

Why is it so important to think in terms of this multidimensional space? Because there are few meaningful statements that are valid across these various axes. When Doug Staiger and Jonah Rockoff (here via Ray Fisman) made radical suggestions about teacher hiring policies, they based them on a study of arguably the two least representative school districts in the country. Even if the rest of the study were sound (rather than being a train wreck, but more on that later), the findings would be worthless for most of the country.

Worse yet, when you make a substantial change in educational policy, there is a wide range of relationships between the effects you see along different dimensions, including possible inverse relationships between retention and other measures of school performance (the fastest and most reliable way for a school to improve its performance is to get rid of the students it can't handle).

Even with the most careful of reasoning, the most clearly stated questions and the most closely examined assumptions, this kind of complex, multidimensional system can react to new conditions in dramatic, counterintuitive ways. If you approach it with the kind of sloppy thinking that has dominated the education debate, you are asking fate to do some very bad things.


[thanks to Wikipedia for the hypercube]