How did we miss this?

It's terrifying when Brad DeLong is more on top of interesting stories on education than we are!  But this is from James Kwak: 

I have no problem with the premise: better teachers improve student outcomes, which is worth a lot of money. But do you see what’s going on? To get better teachers, the authors say, requires ”attracting more qualified people” and then “identifying and retaining” the most effective ones.

That just doesn’t follow. And anyone who’s worked in an actual company should realize that. Yes, it’s always better to have better workers. One way to get better workers is to hire more effective people and to fire less effective people. But the other way—which, in most industries, is by far more important—is to make your current workforce more effective. You do that in part by figuring out what attributes or processes make people more effective, and in part by training people and implementing processes in ways that improve productivity.

The idea that the only way to improve teacher effectiveness (remember, they said “requires”) is to increase quality at the front end and link retention to quality on the back end is the kind of illogical, impractical inference you draw if you have a certain type of attitude toward workers: the attitude that there’s only one abstract attribute that matters (quality) and that it’s intrinsic and unchanging. What’s surprising is that this is a non-obvious kind of fallacy: again, anyone who has run a business realizes that what matters more is what you do with the workforce you have.

I think that the whole idea of organizations like Teach for America falls into the trap above.  Focusing entirely on the recruitment and retention piece and not at all on the teacher development piece.  In some ways this seems to be a malaise of our age -- the want to have plug and play human resources instead of developing the work force with patience and practice. 

The conclusion is interesting as well:

But I also suspect that there’s a feeling, maybe not among these authors, but among the billionaires who like investing in education, of “if only more people like us became teachers”—that there are highly productive people and less productive people, and all we need is to adjust the incentives so more of the former go into teaching. I don’t think the world is that simple.

 That would also be a big issue with the whole inter-state competition piece.  If you think ability is an intrinsic quality (and a scalar) that is invariant between positions then this type of approach is very sensible.  Alternatively, if you want job insecurity to be high, using fear to motivate employees is actually surprisingly effective.

That would put reform into an awkward place, wouldn't it?

TFA Context -- one interesting table

On the heels of Gary Rubinstein's recent post on his first year in Teach for America (reviewed here) and this piece by Jennifer Berkshire on the generous contributions the organization gets from numerous corporate donors like FedEx and Subaru, and given TFA's way of popping up in various ed reform discussions, this seems like a good time to review some basic and uncontroversial facts about this very controversial subject.

Let's start with Wikipedia's introduction:

Teach For America (TFA) is an American non-profit organization whose mission is to "eliminate educational inequity by enlisting high-achieving recent college graduates and professionals to teach" for at least two years in low-income communities throughout the United States.

Of course, TFA does other things, but this is very much the persona-mission of the organization, so it's useful to evaluate the performance of TFA as a teacher recruiting organization. There is a great deal of disagreement over how to interpret the data on TFA members' effectiveness and retention, but some other aspects are much more clear-cut.

TFA is a small, expensive program.

More precisely, this was a very small and expensive program; it's become a merely small but very expensive one.

Year	# of Applicants	# of Incoming Corps Members	# of Regions	Operating Budget
2003	15,708	1,646	20	$29.8M
2004	13,378	1,626	22	$34.0M
2005	17,348	2,181	22	$38.4M
2006	18,968	2,464	25	$55.6M
2007	18,172	2,895	26	$77.9M
2008	24,718	3,614	29	$122.3M
2009	35,178	4,065	35	$153.4M
2010	46,359	4,493	40	$176.0M
2011	47,911	5,066	43	$229M
2012	48,442	5,800[21]	46	$244M
2013	57,000	6,000[22]	48

The cost per recruit has increased dramatically

In order for the TFA model to be scalable, the cost per recruit needs to drop sharply at some point. No sign of that yet.

There is reason to be concerned about selection effects

Actually, double selection effects. First there's self-selection. TFA is known for a daunting and highly selective application process that tends to attract high-achievers with solid resumes. Second, there's the process itself:

In 2010, 46,366 candidates applied and 5,827 were initially admitted, making the acceptance rate 12.6%. However, that number does not include those who earned eventual acceptance into the program from the waitlist of 932 candidates. If all on the waitlist were given acceptance, the acceptance rate would be 14.6%. Since some but not all were accepted from the waitlist, the exact 2010 acceptance rate is unknown, but it ranges from 12.6-14.6%. The acceptance rate for 2011 corps members was less than 11%

The result of all this is a tremendously unrepresentative sample of the pool of potential teachers. As a result, it's difficult to apply anything we learn from TFA data to general policy questions in education.

Equal enforcement of contracts?

Dean Baker is strident:

If the connection with AIG isn't immediately apparent, then you have to look a bit deeper. Folks may recall that AIG paid out $170m in bonuses to its employees in March 2009 with its top executives receiving bonuses in the hundreds of thousands of dollars.

These were people who not only shared responsibility for driving the company into bankruptcy; they also had been at the center of the financial web that propelled the housing bubble into ever more dangerous territory. In other words, the bonus beneficiaries were among the leading villains in the economic disaster that is still inflicting pain across the country.

The prospect of executives of a bailed out company drawing huge bonuses at a time when the economy was shedding 600,000 jobs a month provoked outrage across the country. President Obama spoke on the issue and said that unfortunately no one in his administration was smart enough to find a way that could keep the bonuses from being paid. The problem according to Larry Summers, then the head of President Obama's National Economic Council, was that the bonuses were contractual obligations and they had to be honored.

This provides a striking contrast to what might happen to current and former city employees in Chicago and may happen to current and former employers of the state of Illinois and Detroit. In these cases, it seems that the contracts workers had with their employers may not be honored. Employees who worked decades for these governments, with part of their pay taking the form of pensions in retirement, are now being told that these governments will not follow through on their end of the contract.

The differing treatment of contracts in these situations is striking for several reasons. First, the AIG executives stood to gain much more money with their bonuses on a per person basis. In contrast to the six-figure bonuses going to top executives, pensions for Detroit's workers average just $18,500 a year. Pensions for Chicago's workers average over $33,000 a year, but almost none of these workers will get Social Security, so this will be their whole retirement income.

I am not as upset at the difference in size of the individual compensation, the difference in liability for global economic problems, or the asymmetry in harm -- these points are decent but they are secondary to the key argument.  The real issue is whether the need to honor contracts is being applied equally.  And, it is true that bailing out pensioners has a serious element of moral hazard.  But it would be intriguing to see the argument for bank bailouts NOT having at least some element of moral hazard.   

The real question is what is the status of contracts as being fundamental; the ability to amend a long term agreement when the terms work out in somebody else's favor would really change the dynamics of our ability to run a modern economy.  Remember, it is a small walk from this to deciding not to pay for materials that have been purchased because your business plan did not work . . .

Non-linear relation of the day

There is a modern ethic that more and harder work is always an unbounded good.  But this perspective piece argues that too much pressure can actually reduce outcomes:

When knowledge workers are pressed too hard, the intimate connection between workers and their work is compromised and many things go wrong. Stress and anxiety harm mental health and, hence, performance. Competition rises and team cohesion—a major source of productivity gains—declines. Animosity may develop among staff, or between staff (who feel exploited) and management. Overworked workers take on extra tasks and pay a task-switching productivity penalty that DeMarco estimates at 15%, minimum. And it isn't just average productivity that declines; it is also peak productivity, those rare moments of transcendence when important breakthroughs are made.

Simply assuming a linear relation between effort and output seems like a dangerous assumption, and one that leads to bad policy.  It's also worth noting that we'd prefer to be on the other side of the productivity/pressure curve if we wanted to accept this lower productivity -- happy, unstressed workers tend to be easily retained by an employer.  But unhappy/stressed workers and low productivity seems like a bad trade. 

h/t: Mike

Good reporting alert

From the Seattle Times:

But what’s remarkable is the degree Boeing also wants the public to pay its basic costs. Including, it said, “acquiring site, constructing facility, building infrastructure and procuring equipment/tooling.”

That’s right — we are to buy the land, the factories and the machines that go inside, or at least a share of them. And give it all to Boeing, or let them use it rent-free.

“Company preference is toward a location that will share in the cost of all capital expenditures,” Boeing wrote.

Now that’s socialism. It’s the corporate variety, and it isn’t all bad, what with the good-paying jobs and boost to the local economy. But having the public buy the means of production — socializing the capitalists’ risks (though definitely not their profits) — is a far cry from free enterprise. Almost as far, in a different direction, as what the socialist was going on about.

I think that this point is right on.  It is one thing to ask for tax relief or talk about infrastructure, but when the company is asking the state to build the means of production then it really is socialism.  Whether these profits go to a corrupt apparatchik or to the management of a publicly traded corporation is irrelevant.  What is important is that nobody can realistically compete with the state.  So this type of behavior gives a subsidy to some folks, and prevent entry into the market by competitors with fewer political connections.  

Food for thought, really.  

An extraordinary post by Gary Rubinstein on his first year of teaching

If you're just joining us, Gary Rubinstein is a distinguished math teacher and education writer. He's also, as you'll see, both an alumni and a sharp critic of Teach for America. In this post, he takes a remarkably hard and honest look back at his first year of teaching.

This kind of walk down memory  is not something I'd like to do. My first year appears to have gone a little smoother than his did (at least none of the students hit me), but it was still highly stressful and I still have to live with the knowledge that a lot of kids learned less than they should have because their teacher was just beginning to figure out what he was doing.

The whole piece rings exceptionally true and I recommend reading it in its entirety  but there are a few secondary points I'd like to highlight because they fit in with some of the ongoing threads on our blog.

1. Under the circumstances,  a $5,000 recruitment fee seems high. It's been a few years since I was in the field, but back then applicants were expected to shoulder all of their own expenses (the best I ever got out of the deal was the very occasional  free lunch) and it's a lot easier for schools to find qualified applicants than it was twenty years ago.  It should also noted that some of the schools that pay significant fees to schools in areas that have a large surplus of qualified and experienced teachers, perhaps the most egregious example being Huntville, a major aeronautics research center (hence the nickname 'Rocket City') that gets dozens of qualified applicants for every teaching position. (For a bit of context, the organization had a 2012 operating budget of $244M and put out 5,800 new corps members.);

2. You have to have a thick skin to teach junior high and high school. Kids can be cruel, but more to the point, they are going through an incredibly intense, confusing and stressful period and, as a characteristic of their age, they have a natural proclivity for misplaced emotions. Combine that with an instinctive need to test boundaries and you are left with a room filled with people who will make your life miserable if you let them. With time, you learn how respond without letting them feel they've pushed your buttons and, ironically, in the long run they'll be happier having failed. When you're just starting out, though, you might as well be prepared.

3. I realize I've been spending a lot of time on the culture of the ed reform movement lately, but I don't think you can fully understand what's been going on in the education debate without understanding that culture. TFA is a big part of that picture. Reading accounts of TFA alumni, I'm always struck by what an intense bonding experience the training and first two years would have been. Even though Rubinstein may not have been the best fit with TFA, he still says half the people he invited to his wedding were fellow core members.
Add to that official rhetoric that creates distance between TFA members and established traditional teachers, and you can see why compromise and self-policing might be difficult for the movement they largely define and why the movement is so quick to converge on certain sometimes unusual positions like pedagogical theories that seem to owe more to Derrida than to Piaget.

4. Putting aside those occasional unusual positions, whenever I read accounts of the training material, there's generally an odd disconnect. Most of what I hear is relatively familiar, pretty much in line with what you'd hear in a typical education class, but it's usually described as something special, perhaps even unique to TFA (apparently trainees sometimes also pick up on these discrepancies -- check out what this writer had to say about these “uncommon techniques”). Apparently, assertive discipline* is another one of those overlaps between TFA and lots of traditional educators. I used the technique for a year when I first started teaching twenty-plus years ago (then dropped it like a fresh horseshoe). It was a hot trend and its proponents assured all the new teachers that it was almost completely effective if done according to the instructions. In practice, most new teachers seemed to get results closer to Rubinstein's.

Here are some excerpts:

Nowadays, districts pay a $5,000 recruitment fee to TFA for the opportunity to hire the new TFA trainees.  I don’t know if back then TFA got any money for me.  I hope they didn’t.  At my school that year there were about 150 teachers.  Three of us were new, me, Jon, and Mitzi, all TFA.  I don’t know who the school would have hired if not for us, so it is hard to say whether or not we were a positive or negative influence on the school compared to what it would be without us.  Though that year did nearly break me, and wasn’t so kind to Mitzi or Jon either, I’ve always felt that, in the scheme of things, we didn’t do ‘damage’ to our students.  One reason for this was that we taught middle school so I was just one out of seven teachers my students had each day.  The other six teachers knew what they were doing so in some ways my class became an opportunity for students to try to learn self-control since the teacher wasn’t doing a very good job at creating a controlled learning environment.
...
But having lived through the TFA experience and having many TFA alumni who are close friends of mine — I think half of the friends I invited to my wedding were people I met through TFA — I do know that there is a big difference between ‘the organization’ which is a greedy and power hungry one that will lie about their statistics when it benefits themselves to do so and ‘the corps,’ the mostly twenty-two year olds who just graduated college and who try to use whatever skills they have along with their very limited training to get through that first year and try to still make some kind of difference.  And while I am sure that TFA, the organization, at least in its current incarnation, harms children, teachers, and communities, overall, I do think that most of the individual teachers don’t.
...
Mr. Popcorn Pinnochio Afro Rubinstein is one of the meaner nicknames I’ve been called as a teacher.  (The Pinnochio is because of the size of my nose, and not, I think, because they thought I was a liar).  Better nicknames have been Mr. Frankenstein and even Mr. Robitussin.  As can be seen in the picture, till the end I had faith in the TFA endorsed ‘Assertive Discipline’ method of writing names on the board and putting checks next to them.  Still, you either have to hate a guy a lot to make a picture like this, or maybe like them a little.

* I later learned that assertive discipline was recent variant of the far older Rumplesnitz Method of crisis management.

Weekend blogging -- auteur edition

I've always wondered why, despite Orson Welles reputation, his films were so difficult to find. Here's one of the few exceptions.

Comment on last post

Blogger ate another comment so I am going to put it out as a post.

Mark wonders why a movement with smart people and admirable goals can have issues with corruption.  I think that the key piece to understand here is that, in a real organization, the goals will be a mix of real goals and marketing. 

You only have to look at the rhetoric of he top 1% of earners being job creators (remember trickle down economics) to see how this can happen.  After all, the goal of making everyone wealthier and more prosperous is noble.  But it is a "heads I win, tails you lose" scenario.  If trickle down happens the elite get more money.  If it doesn't happen the elite get more money.  It's worth noting that the elite do pretty well regardless of what the truth is.

Similarly, if charter schools improve outcomes for all children than the people who run charter schools will get rich and help kids.  If they fail, these same people will simply get rich (as we won't be able to see failure fast enough to prevent vast profits.  Now I am positive that 100% of these people want the first scenario.  Everybody would like to be wealthy and to be a moral hero at the same time. 

It gets tricky when these goals conflict.  These things range from the subtle (that one study is concerning but maybe it was a fluke) to the less so (this stuff is great marketing and provides cover for all sorts of actions).  It isn't helped by our media preference for news about problems and not news about "another senior class graduates from a public school with a solid education provided by hard working union members" -- which is anodyne and unremarkable news.

So I think that option #5 (it is all marketing) should show up.  It's a variation of #3, except it does not necessarily require the people saying this stuff to necessarily believe it. 

"Carerruption" -- the biggest threat to the education reform movement

The following is the beginning of a blistering piece of satire over at EduShyster:

How can something so wrong feel so righteous? 

For far too long, three little words have been holding back the adult interests that long to put students first. What are they you ask? The teachers union conflict of interest laws. Thankfully these outdated and antiquated restrictions are now out of sight and out of mind where they belong. And while haters like you are already mumbling about ethics, may I take this opportunity to remind you that it isn’t corruption if you do it for the right reasons? I believe the word is *carerruption*

From there the blogger, Jennifer Berkshire, proceeds to list a few select examples of questionable education reform initiatives (totaling in the billions) pushed or brokered by officials who personally stood to profit from the enterprises. It's a great read but it leaves what might be the most important question unasked: why is a movement dedicated to such admirable goals and run by such smart people so vulnerable to blatant scams and rampant corruption?

I'll be filling out more details, but these are the bare bones of my attempt at an explanation.

1. Group dynamics -- given the scope, size and influence of the movement, we are talking about a surprisingly tight-knit, highly interconnected group. Add to that a relatively high level of homogeneity in terms of background, education, class, teaching experience and pedagogical philosophy. Under these conditions we would expect a tendency toward group-think, excessive social norming, powerful group identity, and us vs. them attitudes. We would also expect affinity cons.

2. Culture -- those us/them tendencies are greatly heightened by a dogma that implicitly and sometimes explicitly blames the failures of schools on "some combination of apathy or incompetence" on the part of non-movement educators. You can find many more examples on Gary Rubinstein's blog.

3. Spin feedback -- the reform movement has been extraordinarily aggressive in its well-financed lobbying and PR efforts. For a while, it was quite successful at selling the twin narrative of impending disaster and shining hope. We've since seen growing popular skepticism about this narrative but very little of that skepticism seems to have made it into the movement itself. Unfortunately, if perhaps inevitably, the rhetoric intended to convince the rest of the world is most resonant within the group.

4. Lack of immediate external checks -- the press has tended to be sympathetic and has generally held off from criticizing until there was overwhelming evidence that something was wrong (Michelle Rhee, the LA IPad fiasco).

The result of all of this is a movement that has no natural defenses against internal abuses, and, given the amount of money we're talking about, abuse is pretty much inevitable.

Update: If you're coming in via link, make sure to check out Joseph's reply.

"But the Pension Fund Was Just Sitting There"

For some reason I started thinking about the title of this old Doonesbury book. Funny how the mind works.

On a completely unrelated note, David Sirota has a fascinating article on Detroit's financial troubles. As our favorite auteurist used to say, check it out.

"A grain of discovery"

[Dictated into a smart phone. I tried to catch all of the homonyms, but, hell, it's late and I'm fading.]

Having opened up what will probably be a long thread on Common Core, we really need to bring George Pólya into the conversation.

Pólya was a first rate mathematician and an enthusiastic and dedicated teacher. so dedicated that he spent somewhere in the neighborhood of 70 years in front of a classroom and he spent a large part of those years thinking long and hard about how we learn mathematics.

I'll go into more detail later about his conclusions. In addition to being insightful and interesting in their own right, they provide a useful alternative when discussing both the rigorously axiomatic approach of 'new math' and the reductionist 'deliberate practice' being pushed by advocates of common core.

For now though, I think it's important to start with the underlying assumptions of Pólya's approach to mathematics: Humans are by nature curious and drawn to interesting problems; therefore, if taught correctly, math should be both stimulating and enjoyable (as are the many popular games and puzzles based on mathematics).

In other words, if my kids hate math, I'm not teaching it right.

This is how Pólya opens the preface to How to Solve It:

A great discovery solves a great problem but there is a grain of discovery in the solution of any problem. Your problem may be modest; but if it challenges your curiosity and brings into play your inventive faculties, and if you solve it by your own means, you may experience the tension and enjoy the triumph of discovery. Such experiences at a susceptible age may create a taste for mental work and leave their imprint on mind and character for a lifetime.

Thus, a teacher of mathematics has a great opportunity. lf he fills his allotted time with drilling his students in routine operations he kills their interest, hampers their intellectual development, and misuses his opportunity. But if he challenges the curiosity of his students by setting them problems proportionate to their knowledge, and helps them to solve their problems with stimulating questions, he may give them a taste for, and some means of, independent thinking. 

Here's the entire preface, preprefaced with more of my thoughts on the subject.

Diane Ravitch provides some historical perspective on international math rankings

A perverse problem with education reform is that, on a high enough level, almost everyone agrees it's a good thing. Unfortunately, it is easy to confuse the question of "is it a good idea to improve education?" with some far murkier questions such as: 

Why is it urgent now to change education radically?

What constitutes ideal educational outcomes?

What are the most effective steps for improving education?

These are difficult questions which interact with each other in highly complex ways. Discussing them is hard enough under the best conditions but it gets much worse when advocates for certain proposals dismiss criticism by saying the people on the other side simply don't care about children (a practice that far predates the current education reform debate).

As for the question of urgency, members of the education reform movement have almost always pointed to either our decline in international tests such as PISA or our race and class based achievement gaps. The second of these is very real and very worrisome, but it often has a rather indirect relationship with reform proposals like introducing common core or greatly reducing teacher tenure.

As for the first, education historian and movement reformer turned gadfly, Diane Ravitch offers the following context.

International testing began in the mid-1960s with a test of mathematics. The First International Mathematics Study tested 13-year-olds and high-school seniors in 12 nations. American 13-year-olds scored significantly lower than students in nine other countries and ahead of students in only one. On a test given only to students currently enrolled in a math class, the U.S. students scored last, behind those in the 11 other nations. On a test given to seniors not currently enrolled in a math class, the U.S. students again scored last.

The First International Science Study was given in the late 1960s and early 1970s to 10-year-olds, 14-year-olds, and seniors. The 10-year-olds did well, scoring behind only the Japanese; the 14-year-olds were about average. Among students in the senior year of high school, Americans scored last of eleven school systems.

In the Second International Mathematics Study (1981-82), students in 15 systems were tested. The students were 13-year-olds and seniors. The younger group of U.S. students placed at or near the median on most tests. The American seniors placed at or near the bottom on almost every test. The “average Japanese students achieved higher than the top 5% of the U.S. students in college preparatory mathematics” and “the algebra achievement of our most able students (the top 1%) was lower than that of the top 1% of any other country.” (The quote is from Curtis C. McKnight and others, The Underachieving Curriculum: Assessing U.S. Mathematics from an International Perspective, pp. 17, 26-27). I summarized the international assessments from the mid-1960s to the early 1990s in a book called National Standards in American Education: A Citizen’s Guide (Brookings, 1995).

The point worth noting here is that U.S. students have never been top performers on the international tests. We are doing about the same now on PISA as we have done for the past half century.

Does it matter?

In my recent book, Reign of Error, I quote extensively from a brilliant article by Keith Baker, called “Are International Tests Worth Anything?,” which was published by Phi Delta Kappan in October 2007. Baker, who worked for many years as a researcher at the U.S. Department of Education, had the ingenious idea to investigate what happened to the 12 nations that took the First International Mathematics test in 1964. He looked at the per capita gross domestic product of those nations and found that “the higher a nation’s test score 40 years ago, the worse its economic performance on this measure of national wealth–the opposite of what the Chicken Littles raising the alarm over the poor test scores of U.S. children claimed would happen.” He found no relationship between a nation’s economic productivity and its test scores. Nor did the test scores bear any relationship to quality of life or democratic institutions. And when it came to creativity, the U.S. “clobbered the world,” with more patents per million people than any other nation.

Baker wrote that a certain level of educational achievement may be “a platform for launching national success, but once that platform is reached, other factors become more important than further gains in test scores. Indeed, once the platform is reached, it may be bad policy to pursue further gains in test scores because focusing on the scores diverts attention, effort, and resources away from other factors that are more important determinants of national success.” What has mattered most for the economic, cultural, and technological success of the U.S., he says, is a certain “spirit,” which he defines as “ambition, inquisitiveness, independence, and perhaps most important, the absence of a fixation on testing and test scores.”

Baker’s conclusion was that “standings in the league tables of international tests are worthless.”

Fairness and regulation

Dean Baker has some good points:

For the last three decades the government has pursued a wide range of policies that have had the effect of redistributing income upward. For example our trade policy, by deliberately placing manufacturing workers in direct competition with low paid workers in the developing world, has lowered the wages of large segments of the work force. By contrast, we have left in place the restrictions that protect doctors and other highly paid professionals from foreign competition, ensuring that their pay stays high.

and

The question at issue is not the amount of redistribution, the question is the direction of the redistribution. The Post seems to want readers to imagine that the upward redistribution of the last three decades was just a fact of nature, as opposed to being an outcome of government policy. That is a major distortion of reality.

This really is the heart of the argument about inequality today.  The notion that the wages earned by high status professions (medicine, finance) is a result of pure market forces and nothing to do with the regulatory environment is weak.  In the same sense, the huge returns to intellectual property have a lot to do how the laws protecting this property are implemented.  And it is not an accident that 401(k) style retirement plans are popular among companies that get to collect management fees.  This isn't to say all IP law is bad (it isn't) or all wages earned by finance gurus is unearned (also untrue). 

But it is a worthwhile starting point to realize that the web of laws and customs that we do have does benefit some groups at the expense of others.  So it is a very weak claim to argue fairness of economic outcomes in the face of differential regulatory treatment. 

Happy PISA Day, everybody!

Richard Rothstein and Martin Carnoy of the Economic Policy Institute have some strong words about how PISA data is released.

It is usual practice for research organizations (and in some cases, the government) to provide advance copies of their reports to objective journalists. That way, journalists have an opportunity to review the data and can write about them in a more informed fashion. Sometimes, journalists are permitted to share this embargoed information with diverse experts who can help the journalists understand possibly alternative interpretations.

In this case, however, the OECD and ED have instead given their PISA report to selected advocacy groups that can be counted on, for the most part, to echo official interpretations and participate as a chorus in the official release.* These are groups whose interpretation of the data has typically been aligned with that of the OECD and ED—that American schools are in decline and that international test scores portend an economic disaster for the United States, unless the school reform programs favored by the administration are followed.

The Department’s co-optation of these organizations in its official release is not an attempt to inform but rather to manipulate public opinion. Those with different interpretations of international test scores will see the reports only after the headlines have become history.

Such manipulation in the release of official government data would never be tolerated in fields where official data are taken seriously. Can you imagine the Census Bureau providing its poverty data in advance only to advocacy groups that supported the administration, and then releasing its report to the public at an event at which these advocacy groups were given slots on a program to praise the administration’s anti-poverty efforts? What if the Bureau of Labor Statistics gave its monthly unemployment report in advance to Democrats, but not to Republicans, and then invited Democratic congressional leaders to participate in the official release?

I know I've hammered this point before, but the education reform movement has been playing a very aggressive long game when it comes to lobbying and PR. Add to that a tradition of advocate research and a culture that tends to eschew firewalls and turn a blind eye to conflicts of interest, then lubricate the gears with a flood of government contracts and private grants. The result is a movement prone to all manner of problems and abuses.

The problem isn't that most of the people in the movement are uncaring and insincere; it's that they aren't. The typical movement reformer cares deeply about kids and genuinely feels that our education system is in a state of crisis. Given that mindset, it's easy to understand the decision to feed the data to groups that will give it the 'correct' interpretation in those first few days when the narrative starts to set.

But to understand the behavior is one thing. To condone it is another. Official data needs to be presented so that all sides start on an equal footing, even if that means your side may lose.

* The Alliance for Excellent Education, Achieve, ACT, America Achieves, the Asia Society, the Business Roundtable, the Council of Chief State School Officers, the College Board, the National Board for Professional Teaching Standards, and the National Center on Education and the Economy.

"Up until about a month ago, we all knew what merit pay meant."

Dana Goldstein has a good write-up on a recent education study:

In 10 cities, including Los Angeles, Miami, and Houston, researchers at Mathematica identified open positions in high-poverty schools with low test scores, where kids performed at just around the 30th percentile in both reading and math. To fill some of those positions, they selected from a special group of transfer teachers, all of whom had top 20 percent track records of improving student achievement at lower poverty schools within the districts, and had applied to earn $20,000 to switch jobs. The rest of the open positions were filled through the usual processes, in which principals select candidates from a regular applicant pool.

If a transfer teacher stayed in her new, tougher placement for two years, she’d earn the $20,000 in five installments, regardless of how well her new students performed. In public education, $20,000 is a whopping sum, far more generous than the typical merit pay bonus of a few hundred or a few thousand dollars.

In the process, a remarkable thing happened. The transfer teachers significantly outperformed control-group teachers in the elementary grades, raising student achievement by 4 to 10 percentile points—a big improvement in the world of education policy, where infinitesimal increases are often celebrated.

 but I have one big problem with the way she presents the findings.

That’s why the results of a new study, the Talent Transfer Initiative, financed by the federal government, are so important. Surprisingly, this experiment found merit pay can work.

Up until about a month ago, we all knew what merit pay meant. Under these systems of compensation, your pay would vary based on certain performance metrics from the previous period.

The initial rationale for applying this type of compensation to education was based on the assumption that teachers would work harder and do a better job if their performance (defined by their students' test scores) was tied to their pay . This played in very closely with the sub-narrative that problems in American education were largely driven by lazy, tenured teachers.

Unfortunately, when this idea was was tried in various pilot programs, it failed to show any substantial effect. It almost appeared as if teachers were already, for the most part, trying to do their best even when their pay didn't vary with performance.

This failure of concept led to a wave of revisionism from movement reformers such as Jonathan Chait. They quickly came up with a new claim. Under the revised history, rationale for merit pay had never been about incentives; push had always been about selection and retention of the best people.

There were always significant problems with this new rationale. Unless you were to assume that being a good teacher was strongly correlated with being bad at mathematics, the size of the bonuses would have to be very large to compensate for the deferment and variability of compensation. Even more troubling, the metrics proposed to evaluate teachers had been shown to be wildly unstable, thus teachers would have very little idea of what their annual take-home pay would be from year-to-year.

The flaws in that second rationale were so obvious, it almost suggested that the proponents were simply looking for a face-saving claim to make before backing away from the issue.

That face-saving quality is even more prominent in the coverage of the recent Chicago experiment in hiring bonuses for teachers. As generally presented by Goldstein and others, this seems like a partial win for both sides. Movement reformers can point to the findings and say that merit pay works while counter-reform advocates can point out that the teachers who got these great results were highly experienced and certified.

The problem with that story is that the first part of the claim is based on a complete 180-degree redefinition of merit pay. The bonus had nothing to do with metrics of success; instead it was solely contingent on those teachers serving out a specified term in the position. In other words, it used the exact same rationale that movement reformers have always objected to when applied to tenure and pay raises for seniority. (You can see Diane Ravitch making some of the same points here.)

I understand the benefits of keeping a debate civil. We should constantly make it clear that the vast majority of people on both sides of this debate share the common goal of improving education. We cannot, however, let the desire for civility become an excuse for dishonesty, particularly not when we are reporting on research that affects open policy questions.

This study suggest taking something like the Canadian approach to managing and compensating teachers. In other words, roughly the opposite of what those advocating merit pay have called for. You can question the quality of this study. You can call it impractical and question whether the results can be scaled up to a useful level. What you cannot do is claim that the data supports your position because you have changed your definitions mid-argument.