In the center, the National Review. On the right, the New Republic

Jim Manzi has an excellent column discussing proposed teacher evaluation metrics from a business perspective, a column that raises some of the same questions that teacher's unions have brought up. There's nothing particularly surprising about that -- Manzi is an intelligent man with a well known independent streak. He's not going to disagree with a position just because he's a conservative.

Jonathan Chait dismisses Manzi's points with some sweeping generalities, completely ignores his point about fairness to the evaluated and ends up being significantly less sympathetic to the concerns of labor than Manzi. Sadly this not surprising either. Chait is one of the most brilliant pundits we have but on the topic of education he combines intense feelings with an apparent lack of knowledge of the important research in the field. This has caused him to embrace certain popular narratives even when they lead him to conclusions that contradict his long standing values.

But as unsurprising as the parts may be, when you put them together the strangeness of the current education debate just sweeps over you. Formerly right-wing positions like privatizing large numbers of schools or denying unions the right to protect workers from unfair termination are now dogma for much of the left. It has reached the point where when a writer for the National Review suggests, as part a larger analysis, that teachers can have legitimate concerns about the reliability of the metrics used to evaluate them, the voice of the New Republic dismisses the possibility without even feeling the need to make an argument.

Even without the political role reversal, Chait's response is strange and oddly disengaged. Judge for yourself.

[I'm presenting these out of order for reasons that will obvious]

1. You need some system for deciding how to compensate teachers. Merit pay may not be perfect, but tenure plus single-track longevity-based pay is really, really imperfect. Manzi doesn't say that better systems for measuring teachers are futile, but he's a little too fatalistic about their potential to improve upon a very badly designed status quo.

Argument by modifier with not one but two 'really's and a 'very' to sell the point. What he doesn't give is any kind of supporting evidence whatsoever. With millions of teachers and a small but thriving industry of think tanks digging up damning anecdotes, you can always find something negative to say, but Chait doesn't even bother coming up a bad argument.

There's an odd, listless quality to the entire post. Chait is normally an energetic and relentless debater. Here he just goes through the motions. He doesn't even bother to proof his prose (I'm pretty sure he either meant to say "the search...is futile"). He also makes a huge jump from the specific techniques Manzi is focusing on to "better systems." I'm pretty sure that Manzi believes better systems can improve the status quo; he just questions how big a role value-add metrics will play in those systems.

As for the case for longevity vs. value-added, I'll let Donald Rubin take it from here:

We do not think that their analyses are estimating causal quantities, except under extreme and unrealistic assumptions.

This is not to say that there isn't a case to be made for merit pay. I don't have any problem with rewarding teachers who do exceptional work, but the methods being discussed here are simply not the way to do it.

Chait's third point runs along similar lines:

3. In general, he's fitting this issue into his "progressives are too optimistic about the potential to rationalize policy" frame. I think that frame is useful -- indeed, of all the conservative perspectives on public policy, it's probably the one liberals should take most seriously. But when you combine the fact that the status quo system is demonstrably terrible, that nobody is trying to devise a formula to control the entire teacher evaluation process, and that nobody is promising the "silver bullet" he assures us doesn't exist, his argument has a bit of a straw man quality.

More argument by adverb and a strange double straw man (straw-straw man? straw straw man man?) continued from the soon-to-be-discussed point 2. The first 'nobody' is doubtful; Chait seems to jump from the fact that no state currently bases evaluations primarily on value-added metrics to the conclusion that no one is even looking into the possibility. The second 'nobody' is just plain wrong; many reform movement followers have so much faith in the silver bullet status of value-added metrics that they have seriously proposed firing more than half of our teachers based on that one number.

But the weirdest part came in point 2.

2. Manzi's description...

evaluating teacher performance by measuring the average change in standardized test scores for the students in a given teacher’s class from the beginning of the year to the end of the year, rather than simply measuring their scores. The rationale is that this is an effective way to adjust for different teachers being confronted with students of differing abilities and environments.

..implies that quantitative measures are being used as the entire system to evaluate teachers. In fact, no state uses such measures for any more than half of the evaluation. The other half involves subjective human evaluations.

Argument by ellipses. Take a look at the whole paragraph:

Recently, Megan McArdle and Dana Goldstein had a very interesting Bloggingheads discussion that was mostly about teacher evaluations. They referenced some widely discussed attempts to evaluate teacher performance using what is called “value-added.” This is a very hot topic in education right now. Roughly speaking, it refers to evaluating teacher performance by measuring the average change in standardized test scores for the students in a given teacher’s class from the beginning of the year to the end of the year, rather than simply measuring their scores. The rationale is that this is an effective way to adjust for different teachers being confronted with students of differing abilities and environments.

Manzi explicitly says "widely discussed attempts." Now, for the sake of comparison, check out the New York Times' similar wording:

A growing number of school districts have adopted a system called value-added modeling to answer that question, provoking battles from Washington to Los Angeles — with some saying it is an effective method for increasing teacher accountability, and others arguing that it can give an inaccurate picture of teachers’ work.

The system calculates the value teachers add to their students’ achievement, based on changes in test scores from year to year and how the students perform compared with others in their grade.

Manzi was perfectly clear with his wording and used language consistent with the New York Times' coverage. It was only by excerpting his paragraph mid-sentence that Chait was able to get even the suggestion of a distortion.

I have somewhat mixed feelings Manzi's business-based approach. There are certain aspects of education that are, if not unique, then at least highly unusual and you have to be careful when drawing analogies (obviously the subject for another, much longer post). That said, all of his points about the way evaluations work are valid and useful.

This is not a bad place to start the debate.

[You can read Jim Manzi's somewhat bewildered reaction to Chait's column here.]