Unfortunately, like so many appealing theories, it didn't do that well in the messiness of the real word. First researchers concluded that the data was too volatile and confounded to identify poor teachers, and now a major study by Vanderbilt and Rand has failed to show anything more than trivial results from incentive pay. Faced with these unpalatable facts, reform supporters have stayed true to their conviction that education (or at least education reform) should be run like a business and have done what so many project managers before them have done: they've moved the goalposts.
Eric A. Hanushek, from the Hoover Institution, assures us that he knew it all along:
"The biggest role of incentives has to do with selection of who enters and who stays in teaching - i.e., how incentives change the teaching corps through entrance and exits," Hanushek said. "I have always thought that the effort effects were small relative to the potential for getting different teachers. Their study has nothing to say about this more important issue."
Jonathan Chait echoed the Hoover line (bet you never thought you'd hear that one):
Of course, the point of performance pay isn't to wring better results out of the same teaching pool. It's to change the composition of the teaching pool. Teachers tend to come from the lower ranks of college graduates. That's natural, because the profession pays poorly compared with other jobs requiring college degree and does not offer financial rewards for success. The idea of merit pay is that you lure into the profession people who want to be treated like professionals -- they run the risk of being fired if they're incompetent, but they can also earn recognition and higher pay for exceptional performance.
It's true that there important secondary selection benefits from a well-designed incentive system, but how credible is the claim that all the reformers were interested in from the beginning were the selection effects, that the Vanderbilt results were unimportant, even, according to Hanushek, expected?
Why did these reform supporters push ahead with high profile research that they believed would prove nothing and would make the movement look bad? This was not a cheap study. In addition to conventional funding, a private donor, presumably a reform movement supporter, put up 1.3 million dollars of his own money to test the hypothesis that incentive pay for teachers would improve student test performance. If "the point of performance pay [wasn't] to wring better results out of the same teaching pool," why waste over a million dollars to see how well performance pay did just that?
For a fraction of that money, you could have funded research that would have directly addressed the question of teacher self-selection by conducting a quick and cheap survey-based study that would look at the correlation between attitudes toward incentive pay and factors like GPA.
And even if we accept the I-meant-to-do-that response, the Vanderbilt study still presents advocates of incentive pay with a huge problem. The assumption behind their theory is that competent, hard-working people will go where competence and hard work are rewarded. Unfortunately, the study indicates that either the incentive metrics are largely out of teachers' control or teachers were generally doing what they could to maximize student performance before bonuses were on the table.
Keep in mind, we're talking about individual bonuses, not the kind you get for an organization meeting some goal. Do we have any evidence, even anecdotal, to show that more desirable employees are attracted to compensation plans with large individual incentive components even when the employees have been shown to have little if any influence over the value of those incentives?
If we were talking about a good, well-designed compensation scheme you could make a strong case for positive selection effects, but we're not even close. We are talking about incentive pay based on hopelessly confounded and volatile data. We are talking about incentive pay based on easily manipulated metrics. We are talking about incentive pay that does not incent.
You really need to read that last one out loud to get the full effect:
We are talking about incentive pay that does not incent.
Just so we're clear, the reform advocates are saying that we take money from things like salary and training and divert it to bonuses that are based on poor-quality data and have not been shown to provide incentive value. We should do this because this poorly-designed compensation scheme will attract a better class of applicant.
And on top of all that, they're asking us to believe this was their plan from the very beginning.
