More on the looting phase of the education reform movement

Having watched the rise and fall (?) of the education reform movement for years now, I often find my attention split the story and the metastory. The story is what's being done in education, how it's working and what should be done differently. The metastory is about the way the reform narrative grew and entrenched itself (particularly with parts of the left) and about the way movement reformers (again, particularly on the left) failed to deal with certain longstanding concerns.

Putting aside the question of political agendas (which really shouldn't be considered when assessing arguments -- bad motives don't preclude good reasoning), the concerns fell into three broad categories:

1. The underlying pedagogical and economic assumptions were flawed;

2. There were intractable data issues that were being overlooked;

3. The system was vulnerable to (and had huge incentives for) gaming at a level that could undo any of the benefits that might come from the proposed reforms.

It is the third concern that prompted me to speculate about affinity effects (a point disputed here). Though evidence supporting all three has been accumulating for years, discussions of the first two got very technical very quickly. Support for the third, though, was obvious on a common sense level.

When a group repeatedly fails to notice something as it becomes increasingly obvious, you can legitimately start looking for some external influence, and the problem with gaming has become increasingly obvious. We started with clear vulnerabilities, then saw the rise of charismatic figures who asked us to trust them with tremendous amounts of money and power based on extravagant but questionable claims. Almost immediately after this, reports starting flowing in of suspicious test results, extensive student dumping and other signs of aggressive data manipulation.

How were so many otherwise alert and skeptical movement reformers caught off guard by problems anyone could have seen coming? I believe part of the answer lies in the culture and narratives of the reform movement. Cultural affinity meant that reformers (who generally had good intentions themselves) tended to project similar values on those who displayed the correct cultural signifiers while hero/villain narratives made it difficult to accept the idea that some of those cast in heroic role might be corrupt while some of those cast in the villainous parts might be right.

Though overt cheating has been getting most of the press lately, larger, more systemic data manipulation is potentially the bigger concern. Mike the Mad Biologist has been on this beat and sends us to this depressing data point.[emphasis added]

One of the first things a visitor sees when stepping into Kipp Academy is a graph that shows how Kipp is outperforming Metro schools in every subject.

However, Kipp Academy is also one of the leaders in another stat that is not something to crow about.

When it comes to the net loss of students this year, charter schools are the top eight losers of students.

In fact, the only schools that have net losses of 10 to 33 percent are charter schools.

"We look at that attrition. We keep an eye on it, and we actually think about how we can bring that back in line with where we've been historically," said Kipp Principal Randy Dowell.

Dowell said Kipp's 18 percent attrition is unacceptable.

MNPS feels it's unacceptable as well, because not only are they getting kids from charter schools, but they are also getting troubled kids and then getting them right before testing time.

"That's also a frustration for the zoned-school principals. They are getting clearly challenging kids back in their schools just prior to accountability testing," said MNPS Chief Operating Officer Fred Carr.

Nineteen of the last 20 children to leave Kipp Academy had multiple out-of-school suspensions. Eleven of the 19 are classified as special needs, and all of them took their TCAPs at Metro zoned schools, so their scores won't count against Kipp.

"We won't know how they perform until we receive results and we see. We would be happy to take their results, frankly. The goal is getting kids ready for college. The goal is not having shiny results for me or for anyone on the team," Dowell said.

There's a surreal quality to Dowell's responses here. The kind of attrition we're talking about in this case is almost entirely at the administrator's discretion. What's more, it generally takes a great deal of time and paperwork to make it happen.

As mentioned before, these schools already have selection bias and social norming working in their favor. If, on top of that, a principal like Dowell selectively gets rid of almost one out of five students based on behavior and performance, it is difficult for a school not to look good.