Following Mark's link on education

As a follow-up to Mark I wanted to specically call out some of the pieces of Felix Salmon's piece on test scores and education. 

Instead, reformers are rushing to use this data as a quantitative performance-review tool, something which can get you a raise or which can even get you fired. And by so doing, they’re turning it from something potentially extremely useful, into a bone of contention between teachers and managers, and a metric to be gamed and maximized.
 

When all decisions on based on a single score, you incent behavior which maximizes the score and minimize additional focus.  Felix makes an interesting point that if you used this data to provide coaching and feedback then it could actually be really useful.  Teachers would still want students to do well on the test (it is much, much nicer to talk to your principal about how generally well your students are doing than to get coaching on how to try and shore up a weak point). 

I also think that this point was really sharp:

School reformers in general, it seems to me, tend to be obsessed with the idea of Good Teachers and Bad Teachers, as though the quality of the education a kid gets in any given classroom is somehow both predictable and innate to the teacher. And yes, at the extremes, there are a few great inspirational teachers who we all remember decades later, and a few dreadful ones who had no idea what they were talking about and who had no control of their classes. But frankly, you don’t need student surveys to identify those outliers. And the fact is that schools are much more than just the sum of their parts: that’s one of the reasons that reformers love to talk about excellent principals who can turn schools around.

He is very cleverly and accurately pointing out a form of equivocation that is being used here.  There are extreme examples, but they were never the problem in terms of identification.  There are some odd employment rules in some places that made acting on this knowledge awkward, but very few people saw these as being good policy.  The real use of these tests to to try and break apart the middle of the distribution.  But, by definition, the gain in the middle of the distribution is much less than the difference between exceptional and abysmal.  You are not taking Jaime Escalante versus an incompetent as your contrast.  You are taking pretty good versus very good as your contrast, and thus setting things up for a life event to move people back and forth in the distribution.  Your child gets ill, you are more tired and work so you lsoe your job because you slip below the median.  No wonder teachers are suspicious of such metrics. 

Data is good but one of the lessons of the MBA approach to management is that not everything can be broken down into numbers on a spreadsheet to be maximized.  I fear we'll figure that out, sooner or later.