Why we forgive him the puns

Today's column in the New York Times is another reminder of why Paul Krugman is so essential. It directly contradicts some of the most cherished conventional wisdom about the relationship between education and economic opportunity, but he doesn't say anything that I haven't been hearing from researchers and academicians.

It is a truth universally acknowledged that education is the key to economic success. Everyone knows that the jobs of the future will require ever higher levels of skill. That’s why, in an appearance Friday with former Florida Gov. Jeb Bush, President Obama declared that “If we want more good news on the jobs front then we’ve got to make more investments in education.”

But what everyone knows is wrong.

...

The fact is that since 1990 or so the U.S. job market has been characterized not by a general rise in the demand for skill, but by “hollowing out”: both high-wage and low-wage employment have grown rapidly, but medium-wage jobs — the kinds of jobs we count on to support a strong middle class — have lagged behind. And the hole in the middle has been getting wider: many of the high-wage occupations that grew rapidly in the 1990s have seen much slower growth recently, even as growth in low-wage employment has accelerated.

Why is this happening? The belief that education is becoming ever more important rests on the plausible-sounding notion that advances in technology increase job opportunities for those who work with information — loosely speaking, that computers help those who work with their minds, while hurting those who work with their hands.

Some years ago, however, the economists David Autor, Frank Levy and Richard Murnane argued that this was the wrong way to think about it. Computers, they pointed out, excel at routine tasks, “cognitive and manual tasks that can be accomplished by following explicit rules.” Therefore, any routine task — a category that includes many white-collar, nonmanual jobs — is in the firing line. Conversely, jobs that can’t be carried out by following explicit rules — a category that includes many kinds of manual labor, from truck drivers to janitors — will tend to grow even in the face of technological progress.

And here’s the thing: Most of the manual labor still being done in our economy seems to be of the kind that’s hard to automate. Notably, with production workers in manufacturing down to about 6 percent of U.S. employment, there aren’t many assembly-line jobs left to lose. Meanwhile, quite a lot of white-collar work currently carried out by well-educated, relatively well-paid workers may soon be computerized. Roombas are cute, but robot janitors are a long way off; computerized legal research and computer-aided medical diagnosis are already here.

And then there’s globalization. Once, only manufacturing workers needed to worry about competition from overseas, but the combination of computers and telecommunications has made it possible to provide many services at long range. And research by my Princeton colleagues Alan Blinder and Alan Krueger suggests that high-wage jobs performed by highly educated workers are, if anything, more “offshorable” than jobs done by low-paid, less-educated workers. If they’re right, growing international trade in services will further hollow out the U.S. job market.

As a statistician, I might quibble with the "If they're right."

So what does all this say about policy?

Yes, we need to fix American education. In particular, the inequalities Americans face at the starting line — bright children from poor families are less likely to finish college than much less able children of the affluent — aren’t just an outrage; they represent a huge waste of the nation’s human potential.

This is another point worth dwelling on for a moment. The educational reform movement likes to draw its poster children from poor urban and rural schools. Having taught in both Watts and the Mississippi Delta, I'm usually glad to see attention focused on these areas, but it's clear in this case that the plight of these kids is being used to market general changes in education that have little if any special relevance to the schools that need the help.

But there are things education can’t do. In particular, the notion that putting more kids through college can restore the middle-class society we used to have is wishful thinking. It’s no longer true that having a college degree guarantees that you’ll get a good job, and it’s becoming less true with each passing decade.

So if we want a society of broadly shared prosperity, education isn’t the answer — we’ll have to go about building that society directly. We need to restore the bargaining power that labor has lost over the last 30 years, so that ordinary workers as well as superstars have the power to bargain for good wages. We need to guarantee the essentials, above all health care, to every citizen.

What we can’t do is get where we need to go just by giving workers college degrees, which may be no more than tickets to jobs that don’t exist or don’t pay middle-class wages.

Update: Lawrence Mishel makes some important related points here.

Does running schools like a business actually argue for keeping LIFO? -- part I, the nature of layoffs

Last In First Out (LIFO) has become one of the causes célèbres of the educational reform movement, a handy, easy to enunciate little acronym that beautifully captures what's wrong with our current system.

But if we really wanted to run schools like a business, just how wrong would LIFO be? This may seem like a strange question. After all, the reform movement is often portrayed as the domain of show-me-the-data empiricists and clear-headed economists. Surely anything they suggest is going to be based on sound business principals.

Here's how one of those economists puts it:

An economist myself, let me try to explain. Economists tend to think like well-meaning business people. They focus more on bottom-line results than processes and pedagogy, care more about preparing students for the workplace than the ballot box or art museum, and worry more about U.S. economic competitiveness. Economists also focus on the role financial incentives play in organizations, more so than the other myriad factors affecting human behavior. From this perspective, if we can get rid of ineffective teachers and provide financial incentives for the remainder to improve, then students will have higher test scores, yielding more productive workers and a more competitive U.S. economy.

The trouble is, having been a well-meaning business person myself (more recently than I was a teacher), and having built hiring and retention models for a couple of big companies (let's just say you've heard of them and leave it at that), most of the reformers don't seem to approaching this the way a business would, at least not a well-run business.

LIFO is a compromise, but not always an unhappy one. Unions would prefer no lay-offs at all. Companies would prefer no restrictions at all on who they let go. As a solution to this conflict, LIFO isn't perfect but it does offer some positives for both parties.

Though employers complain loudly about LIFO, they often end up resorting to something close to a de facto form of the practice when left to their own devices. Established employees have more experience, training and institutional knowledge. They tend to be more stable and reliable, more invested in the job and in the community, much less likely to make a dash for the door when the economy turns around and you find yourself understaffed. Removing them can disruptive and can lower morale.

(It's important not to confuse headcount reduction, which we're talking about here, with cleaning out the deadwood, where how long a person has been with the company should not be a factor. Even there, though, companies will often go to alarming lengths to avoid dismissing an established employee, even one who has cost the company massive amounts of money -- buy me a beer sometime and I'll tell you the examples that don't make it to the blog.)

If you don't have a transparent and reliable performance metric like sales, the consequences of deviating from LIFO can be even worse. Your best qualified and most ambitious employees tend to jump ship (Dilbert's 'brightsizing' writ large). Between the people who are definitely leaving and the uncertainty over who will be let go, a kind of mass learned helplessness can take take hold. As a manager this is not something you want to deal with.

For employees, the advantages are more obvious. LIFO offers a deferred compensation package where the compensation is security. For most of us, the demand for security increases as we age. We start families, sign mortgages, start getting serious about saving for retirement. Viewed in this light, LIFO starts to look like a case of markets doing a remarkably good job of allocation. Employees get security when they value it most and in exchange are willing to work for lower wages. (looks kind of like a win for Adam Smith)

There is, of course, a more immediate need for LIFO in many cases. It is the most effective means of preventing employers from using lay-offs as cover for improper dismissal. The classic example here is punishing employees for their role in a union. This was the issue that so offended Jonathan Chait when the victims were journalists.

Chait's position on teachers was notably different, which is interesting because not only is the potential for the abuse that Chait was implying also a concern for teachers; it is accompanied by other, even greater concerns caused by the nature of teaching and the org charts of schools.

Teachers have three primary roles: instruction; counselling; and evaluation. Because we have a complex, multidimensional, badly defined target variable and some of the nastiest confounded data you'll ever see, every method suggested for measure teacher effectiveness has been either overly-complicated, opaque, unreliable or some combination of the above and all seem to have the potential for gaming by unscrupulous administrators.

An experienced and highly respected superintendent I knew used to tell new teachers, "never trust a superintendent," and while all but one or two of the administrators I have dealt with have been dedicated professionals, I understand where he was coming from. The nastiest politics I have ever seen have been at school board meetings. That's the world administrators have to survive in. If they're successful they can be looking at a plum job in a large (but not huge) district making 170K. If they're inept they can find themselves stuck as assistant principals making little more than they made as teachers.

Put another way, it is entirely possible for a few angry parents and a couple of political missteps to cost a principal fifty thousand or more (possibly quite a bit more). As mentioned before, evaluation is part of a teacher's job and a tough grader will generate more than a few angry parents. This is one of the reasons we have tenure.

Though it's rare, I have seen principals pressure teachers to change the grades of students with influential parents. It is also rare but not unheard of for principals to shift difficult students and unpleasant classes away from allies on the faculty.

Now imagine a system where the principal can arrange, with relatively little effort and no fear of reprisals, for certain teachers to be safe from lay-offs and other teachers to be first in line. I'm sure most principals wouldn't use this unchecked power to put tough graders, independent voices, and strong union supporters in the expendable pile. There would, however, be abuses and the administrators who commit these abuses will have a tremendous advantage in the competitive landscape.

Worse yet, every decision that administrators make will be met with suspicion. Every time a teacher gets a larger than average class with more than the average number of kids with behavior problems, colleagues will wonder if the assignment had something to do with flunking students who blow off their term papers, or pushing for a tough stance in contract negotiation, or just voicing too many concerns in faculty meetings.

Businesses go to great lengths to avoid the atmosphere of distrust and labor/management tension I've just described. For all their talk of looking to business for ideas, advocates of the reform movement don't seem to give these issues much thought.

Next, the different kinds of lay-offs and their implications

More big burdens, more small shoulders

From Jonathan Cohn:

Please indulge me while I share some local news: Rick Snyder, newly elected governor in my home state of Michigan, announced this week that he will call for massive cuts in state spending on education.* (*Note: My [Cohn's] wife is a professor at a public university that would lose some funding under Snyder’s plan. I doubt she'll feel much impact from these cuts. But, as you can guess from this item, I think my kids will.) Very roughly, it will result in a reduction of about $470 per student.

I know enough about public education, and public education bureaucracies, to believe that school districts could find ways to reduce spending without hurting the quality of education. And, yes, it would probably mean teachers and staff making more concessions on salaries or, more likely, benefits.

But could they find $470 per student that way? I don't think so. On the contrary, I expect that schools--including the ones that my sons attend--would end up with fewer teachers, fewer courses, and fewer extracurricular offerings if the legislature approves Snyder's plan. And my kids would be among the lucky ones. It would be much worse in places like Detroit, where an ongoing funding crisis is about to swell some classes to 60 students. (No, that’s not a misprint.)

The biggest burdens go on the smallest shoulders

In today's New York Times, Paul Krugman reminds us just how hypocritical all the talk we hear about "putting children first" really is:

And in low-tax, low-spending Texas, the kids are not all right. The high school graduation rate, at just 61.3 percent, puts Texas 43rd out of 50 in state rankings. Nationally, the state ranks fifth in child poverty; it leads in the percentage of children without health insurance. And only 78 percent of Texas children are in excellent or very good health, significantly below the national average.

But wait — how can graduation rates be so low when Texas had that education miracle back when former President Bush was governor? Well, a couple of years into his presidency the truth about that miracle came out: Texas school administrators achieved low reported dropout rates the old-fashioned way — they, ahem, got the numbers wrong.

It’s not a pretty picture; compassion aside, you have to wonder — and many business people in Texas do — how the state can prosper in the long run with a future work force blighted by childhood poverty, poor health and lack of education.

But things are about to get much worse.

A few months ago another Texas miracle went the way of that education miracle of the 1990s. For months, Gov. Rick Perry had boasted that his “tough conservative decisions” had kept the budget in surplus while allowing the state to weather the recession unscathed. But after Mr. Perry’s re-election, reality intruded — funny how that happens — and the state is now scrambling to close a huge budget gap. (By the way, given the current efforts to blame public-sector unions for state fiscal problems, it’s worth noting that the mess in Texas was achieved with an overwhelmingly nonunion work force.)

So how will that gap be closed? Given the already dire condition of Texas children, you might have expected the state’s leaders to focus the pain elsewhere. In particular, you might have expected high-income Texans, who pay much less in state and local taxes than the national average, to be asked to bear at least some of the burden.

But you’d be wrong. Tax increases have been ruled out of consideration; the gap will be closed solely through spending cuts. Medicaid, a program that is crucial to many of the state’s children, will take the biggest hit, with the Legislature proposing a funding cut of no less than 29 percent, including a reduction in the state’s already low payments to providers — raising fears that doctors will start refusing to see Medicaid patients. And education will also face steep cuts, with school administrators talking about as many as 100,000 layoffs.

Interesting paper on peer effects

I'm not sure this is the ideal methodology to approach this problem (I'd like to see this combined with something that captures the actual interactions in the classroom), but, based on what I've seen, this is a definite improvement.

From VoxEU (via Thoma)

Our recent research uncovers peer effects in education as distinct from the contextual and other correlated influences. Our econometric strategy uses the topological structure of social networks as well as network fixed effects to identify each of these effects separately.

Our analysis is made possible by the use of a unique database on friendship networks from the National Longitudinal Survey of Adolescent Health (AddHealth). The AddHealth database has been designed to study the impact of the social environment (i.e. friends, family, neighbourhood and school) on adolescents' behaviour in the US by collecting data on students in grades 7-12 from a nationally representative sample of roughly 130 private and public schools in years 1994-95 (wave I). A subset of adolescents selected from the rosters of the sampled schools, was then interviewed again in 1995-96 (wave II), in 2001-2 (wave III), and again in 2007-2008 (wave IV). For our purposes, the most interesting aspect of the AddHealth data is the friendship information, which is based upon actual friend nominations. It is collected at wave I, i.e. when individuals were at school. Indeed, pupils were asked to identify their best friends from a school roster (up to five males and five females). As a result, we can reconstruct the whole geometric structure of the friendship networks.

In Calvó-Armengol et al. (2008), we exploit such information to test a peer-effect model which relates analytically equilibrium behaviour to network location. This analysis shows that the structure of friendships ties is an important, and so far unnoticed, determinant of a pupil performance at school. In Patacchini et al. (2011), we follow this line of research by exploiting the other AddHealth waves in order to investigate whether such effect is carried over time. Indeed, the longitudinal structure of the survey provides information on both respondents and friends during the adulthood. In particular, the questionnaire of wave IV contains detailed information on the highest education qualification achieved.

We analyse the impact of the friends' educational attainment on an individual's educational attainment where they are identified as friends during school and in to adulthood. We find that peer effects in education are not only strong but also persistent over time. We find that the most relevant peers are the friends people make in grade 10-12, from when they are around 15 years old. This suggests that individuals are more likely to work towards and apply to college if this choice is popular among their peers, especially in the last years at school. This could represent the effect of contagion and collective socialisation and mean that any education policy targeting specific individuals will have multiplier effects.

If you don't post it someone else will

I was planning to do a post on these comments by Edward Glaeser, but I decided to put it off because:

a) Joseph told me he was reading the book and could give me a better assessment of how the ideas held up;

b) I'd recently criticized Glaeser at some length;

c) I had about a dozen other posts I wanted to do.

One of the comments that annoyed me was Glaeser's comparison of public schools and restaurants, now Dominik Lukeš has spent two thousand plus words taking the analogy apart stone by stone and salting the ground. If you're in the mood for a good dismantling and salting you should definitely follow the link.

Having made fun of Paul Krugman's puns, there's no way I can use the title "Darby's Rhee Lapse"

From the New Republic's Seyward Darby:

The mantra goes, “You either love or hate Michelle Rhee.” In the education world, there is no figure as polarizing as the former chancellor of Washington, D.C.’s public schools, who famously warred with the city’s teachers’ union and left abruptly when her boss, Mayor Adrian Fenty, lost reelection last year. Since then, she has started an organization called StudentsFirst to push for education reform nationwide. She announced the group in a Newsweek cover story, and it raised more than $700,000 in its first week. Andrew Rotherham, an education policy expert, told me, “Do people say, ‘I [am] kind of uncertain about Michelle Rhee’? No way.”

Count me, then, as one of the uncertain few. To be sure, I am generally a fan of Rhee. The world of liberal education policy consists, more or less, of two factions: reformers, who support performance pay, charter schools, and weakening seniority-based job protections for teachers; and opponents of these ideas, who are often allied with teachers’ unions. Like most reformers, I greatly admired Rhee’s tenure in D.C., in which she closed failing schools, fired underperforming teachers, and helped raise student achievement.

But, in reading about Rhee’s recent moves, I’ve felt a nagging sense of disappointment. She is now advising several conservative governors who line up with reformers on certain issues but whose commitment to public education is questionable. Meanwhile, she hasn’t offered robust answers to some of the thorniest matters facing education policymakers. Last week, I put these challenges to Rhee directly. And I came out of our conversation much as I went in: with decidedly mixed feelings about her vision for the education-reform movement.

I have long had decidedly mixed feeling about Seyward Darby (as you can see for yourself with a quick keyword search). Her reporting and analysis of the education reform movement has been, to put it simply, bad (better than Chait's but still bad). She was overly eager to accept the movement's preferred narrative, credulous about its claims, negligent about digging into the research that called these claims into question, and dismissive of those on the other side.

That last trait is still on display even now with the weasel-worded "more or less, of two factions... opponents of these ideas, who are often allied with teachers’ unions." The implication here is that the opponents are shilling for the unions. I don't know the alliances of everyone out there but I can tell you that I'm not allied with the unions (I never even bothered to join one when I was a teacher), nor is Joseph, nor is David Warsh, nor, to my knowledge are most of the people in my corner of the blogosphere. In my experience, it would have been more accurate to say "opponents who question the evidence presented by the reformers."

But there was no question in my mind that Darby is an intelligent, competent and basically honest journalist and that eventually the internal contradictions would start to get to her. One of the ways that people deal with cognitive dissonance is by convincing themselves that things have changed. Rather than question their original assessment and reaction, they convince themselves that they were right then but they are taking the opposite position now because things are different.

Michelle Rhee hasn't changed. She is constant as the northern star. Every point on her career trajectory is collinear. Those who didn't see her current incarnation coming either weren't paying attention or weren't being honest with themselves.

The rest of Darby's article is behind a paywall and I haven't had a chance to look at it. Perhaps something in the piece will invalidate something I've said here. If so let me know and I'll gladly make the appropriate retractions.

The Good Principal Principle

This looks promising. While it is difficult to build a model for good management, this still strikes me as a simpler problem than modelling effective teaching. For one thing, K through 12 teaching includes a large management component. More importantly, the data on teachers runs into some nasty nesting issues. Things are much cleaner when you go up a level.

Yet each school year thousands of principals beat the odds and do excel, women and men who love their leadership positions, relish the challenges and take pride in running schools that perform well year after year. Who are these people? And what are they doing that so many others aren’t?

“We know that principals matter for a school’s success, but we don’t know much about why and how they matter,” says Jason Grissom, an assistant professor of public affairs in MU’s Truman School of Public Affairs. Grissom and Susanna Loeb, a professor of education at Stanford University, are working to provide answers, thanks in part to a $1 million grant from the Institute of Education Sciences, the self-described “research arm” of the U.S. Department of Education.

“Our goal at the end of this study is to be able to offer some tangible recommendations for making principals more effective in terms of improving student outcomes,” Grissom says. “We’re excited about the proposal because it’s pretty ambitious. The kind of study we’ve proposed is essentially the first of its kind.”

Administration

I was reading Mark's post today on principals and I thought it was interesting to see that Dean Dad has a post from the other side of the fence today as well. Some of the points in the Dead Dad post are extremely insightful. Consider the following question:

I’ll answer the question with another question. Good, strong, solid, peer-reviewed scientific data has made it abundantly clear that poor eating habits lead to obesity and all manner of negative health outcomes. There’s no serious dispute that obesity is a major public health issue in the US. And yet people still overeat. Despite reams of publicity and even Presidential support for good eating and exercise habits, obesity continues to increase. Why?

In other words, reform is hard to do even when you know where you want to go. In cases where the evidence is weak or where budgets are falling, the problem gets even worse. And, of course, right now we are experiencing a fall in most education budgets in the United States.

However, it was interesting how Dead Dad was unable to resist worrying about tenure as a barrier to reform:

There’s also a fundamental issue of control. Faculties as a group are intensely protective of their absolute control of the classroom. Many hold on to the premodern notion of teaching as a craft, to be practiced and judged solely by members the guild. As with the sabermetric revolution in baseball, old habits die hard, even when the evidence against them is clear and compelling. There’s a real fear among many faculty that moving from “because I say so” to “what the numbers say” will reduce their authority, and in a certain sense, that’s true. In my estimation, this is at the root of much of the resentment against outcomes assessment.

Even where there’s a will, sometimes there just isn’t the time. It’s one thing to reinvent your teaching when you have one class or even two; it’s quite another with five. And when so many of your professors divide their time among different employers, even getting folks into the same room for workshops is a logistical challenge.

Of course, accountability matters. Longtime readers know my position on the tenure system, so I won’t beat that horse again, but it’s an uphill battle to sell disruptive change when people have the option of saying ‘no’ without consequence. The enemy isn’t really direct opposition; it’s foot-dragging.

I think that this line of thinking may also be part of why there is such a huge push for reform of teacher job security. Administrators are under enormous pressure to reform the education system and teachers may be very resistant.

Of course, one element that may be left out is that the teachers may be resistant to change for good reasons. When you have been in an organization long enough, you realize that a lot of reform can be about trying "old ideas all over again". These reforms can be both time consuming and ineffective. They may even lower outcomes due to the friction of implementation.

Let us consider a business analogy. One way that corporations tried to handle bad outcomes is a series of "re-orgs". These changes in structure have two good properties. One, the people in charge seems to be doing something to address issues by making changes. Two, a series of re-organizations can make it very hard to track a long term pattern of bad management as units break apart too often for performance to be easily tracked.

The ability of tenured people to resist cosmetic reforms is, obviously very frustrating to administrators who have little ability to influence the organization but seemingly unlimited accountability. However, endless re-organizations did not, in the end, help corporations like General Motors. Instead, they may well have accelerated the decline by focusing on making changes that were more cosmetic than effective. So do we really need to import the worst practices of modern corporations into the educational system?

[This post is also relevant -- Mark]

Do good teachers make difficult employees?

A few years ago I did a stint as an instructor at a large state school teaching, among other things, business calculus. The sections for that course tended to be good-sized, usually running from fifty to one fifty. At one time, I probably would have found the experience a bit intimidating but I was just coming off a couple of years as a TA for a professor who routinely taught sections of more than three hundred so I considered myself lucky to be able to make out individual faces.

With few exceptions, experienced teachers are comfortable addressing large groups and with very few exceptions, effective teachers are comfortable demanding the full attention of those group. Along with knowledge of the subject, strong communication skills, and commitment to the students, a "when I talk, you listen" attitude is a defining trait of an effective instructor.

That doesn't automatically translate to a room full of kids sitting quietly while the teacher drones on. Often the result is just the opposite. Teachers are more likely to have looser classes with more student participation if they feel in control. As a rule of thumb, you should never be more than ninety seconds away from having every student seated and reading quietly. For really good teachers, even the most adventurous lesson plans fall into that ninety second radius.

Put another way, it comes down to authority. A teacher's job is to teach, counsel and objectively evaluate his or her students. A sense of authority is an essential trait for all these tasks but it's an incredibly annoying one to find in a direct report.

Good principals (and I've met some excellent ones) are masters at the difficult art of managing managers. They can exercise their authority in a way that actually enhances the authority of those under them.

Even with the best administrators, however, there is always an element of tension and it only gets worse with less competent principals and superintendents. This is something to keep in mind when you hear about plans to improve education by giving principals more authority to get rid of bad teachers. Sometimes bad doesn't mean incompetent; it means inconvenient. (I don't have the book in front of me, but Diane Ravitch' Death and Life has some notable examples.)

The Principal Effect -- repost

[I'm working on a post about this New York Times story about not holding principals accountable for failing schools so I thought I'd use this to get the discussion started.]

When it comes to education reform, you can't just refer to the elephant in the room. It's pretty much elephants everywhere you look. There is hardly an aspect of the discussion where reformers don't have to ignore some obvious concern or objection.

The elephant of the moment is the effect that principals and other administrators have on the quality of schools. Anyone who has taught K through 12 can attest to the tremendous difference between teaching in a well-run and a badly-run school. Even the most experienced teacher will find it easier to manage classes, cover material, and keep students focused. All of those things help keep test scores up, as does the lower rate of burn out. For new teachers, the difference is even more dramatic.

On top of administrator quality, there is also the question of compatibility. In addition to facing all the normal managerial issues. teacher and and principal have to have compatible educational philosophies.

As we've mentioned more than once on this site, educational data is a thicket of confounding and aliasing issues. That thicket is particularly dense when you start looking at teachers and principals and, given the concerns we have about the research measuring the impact of teachers on test scores, I very much doubt we will ever know where the teacher effect stops and the principal effect starts.

In the center, the National Review. On the right, the New Republic

Jim Manzi has an excellent column discussing proposed teacher evaluation metrics from a business perspective, a column that raises some of the same questions that teacher's unions have brought up. There's nothing particularly surprising about that -- Manzi is an intelligent man with a well known independent streak. He's not going to disagree with a position just because he's a conservative.

Jonathan Chait dismisses Manzi's points with some sweeping generalities, completely ignores his point about fairness to the evaluated and ends up being significantly less sympathetic to the concerns of labor than Manzi. Sadly this not surprising either. Chait is one of the most brilliant pundits we have but on the topic of education he combines intense feelings with an apparent lack of knowledge of the important research in the field. This has caused him to embrace certain popular narratives even when they lead him to conclusions that contradict his long standing values.

But as unsurprising as the parts may be, when you put them together the strangeness of the current education debate just sweeps over you. Formerly right-wing positions like privatizing large numbers of schools or denying unions the right to protect workers from unfair termination are now dogma for much of the left. It has reached the point where when a writer for the National Review suggests, as part a larger analysis, that teachers can have legitimate concerns about the reliability of the metrics used to evaluate them, the voice of the New Republic dismisses the possibility without even feeling the need to make an argument.

Even without the political role reversal, Chait's response is strange and oddly disengaged. Judge for yourself.

[I'm presenting these out of order for reasons that will obvious]

1. You need some system for deciding how to compensate teachers. Merit pay may not be perfect, but tenure plus single-track longevity-based pay is really, really imperfect. Manzi doesn't say that better systems for measuring teachers are futile, but he's a little too fatalistic about their potential to improve upon a very badly designed status quo.

Argument by modifier with not one but two 'really's and a 'very' to sell the point. What he doesn't give is any kind of supporting evidence whatsoever. With millions of teachers and a small but thriving industry of think tanks digging up damning anecdotes, you can always find something negative to say, but Chait doesn't even bother coming up a bad argument.

There's an odd, listless quality to the entire post. Chait is normally an energetic and relentless debater. Here he just goes through the motions. He doesn't even bother to proof his prose (I'm pretty sure he either meant to say "the search...is futile"). He also makes a huge jump from the specific techniques Manzi is focusing on to "better systems." I'm pretty sure that Manzi believes better systems can improve the status quo; he just questions how big a role value-add metrics will play in those systems.

As for the case for longevity vs. value-added, I'll let Donald Rubin take it from here:

We do not think that their analyses are estimating causal quantities, except under extreme and unrealistic assumptions.

This is not to say that there isn't a case to be made for merit pay. I don't have any problem with rewarding teachers who do exceptional work, but the methods being discussed here are simply not the way to do it.

Chait's third point runs along similar lines:

3. In general, he's fitting this issue into his "progressives are too optimistic about the potential to rationalize policy" frame. I think that frame is useful -- indeed, of all the conservative perspectives on public policy, it's probably the one liberals should take most seriously. But when you combine the fact that the status quo system is demonstrably terrible, that nobody is trying to devise a formula to control the entire teacher evaluation process, and that nobody is promising the "silver bullet" he assures us doesn't exist, his argument has a bit of a straw man quality.

More argument by adverb and a strange double straw man (straw-straw man? straw straw man man?) continued from the soon-to-be-discussed point 2. The first 'nobody' is doubtful; Chait seems to jump from the fact that no state currently bases evaluations primarily on value-added metrics to the conclusion that no one is even looking into the possibility. The second 'nobody' is just plain wrong; many reform movement followers have so much faith in the silver bullet status of value-added metrics that they have seriously proposed firing more than half of our teachers based on that one number.

But the weirdest part came in point 2.

2. Manzi's description...

evaluating teacher performance by measuring the average change in standardized test scores for the students in a given teacher’s class from the beginning of the year to the end of the year, rather than simply measuring their scores. The rationale is that this is an effective way to adjust for different teachers being confronted with students of differing abilities and environments.

..implies that quantitative measures are being used as the entire system to evaluate teachers. In fact, no state uses such measures for any more than half of the evaluation. The other half involves subjective human evaluations.

Argument by ellipses. Take a look at the whole paragraph:

Recently, Megan McArdle and Dana Goldstein had a very interesting Bloggingheads discussion that was mostly about teacher evaluations. They referenced some widely discussed attempts to evaluate teacher performance using what is called “value-added.” This is a very hot topic in education right now. Roughly speaking, it refers to evaluating teacher performance by measuring the average change in standardized test scores for the students in a given teacher’s class from the beginning of the year to the end of the year, rather than simply measuring their scores. The rationale is that this is an effective way to adjust for different teachers being confronted with students of differing abilities and environments.

Manzi explicitly says "widely discussed attempts." Now, for the sake of comparison, check out the New York Times' similar wording:

A growing number of school districts have adopted a system called value-added modeling to answer that question, provoking battles from Washington to Los Angeles — with some saying it is an effective method for increasing teacher accountability, and others arguing that it can give an inaccurate picture of teachers’ work.

The system calculates the value teachers add to their students’ achievement, based on changes in test scores from year to year and how the students perform compared with others in their grade.

Manzi was perfectly clear with his wording and used language consistent with the New York Times' coverage. It was only by excerpting his paragraph mid-sentence that Chait was able to get even the suggestion of a distortion.

I have somewhat mixed feelings Manzi's business-based approach. There are certain aspects of education that are, if not unique, then at least highly unusual and you have to be careful when drawing analogies (obviously the subject for another, much longer post). That said, all of his points about the way evaluations work are valid and useful.

This is not a bad place to start the debate.

[You can read Jim Manzi's somewhat bewildered reaction to Chait's column here.]

Jim Manzi has some smart things to say about teacher evaluations

From the National Review (via Chait, but more on that later)

This seems like a broadly sensible idea as far as it goes, but consider that the real formula for calculating such a score in a typical teacher value-added evaluation system is not “Average math + reading score at end of year – average math reading score at beginning of year,” but rather a very involved regression equation. What this reflects is real complexity, which has a number of sources. First, at the most basic level, teaching is an inherently complex activity. Second, differences between students are not unvarying across time and subject matter. How do we know that Johnny, who was 20 percent better at learning math than Betty in 3rd grade is not relatively more or less advantaged in learning reading in fourth grade? Third, an individual person-year of classroom education is executed as part of a collective enterprise with shared contributions. Teacher X had special needs assistant 1 work with her class, and teacher Y had special needs assistant 2 working with his class — how do we disentangle the effects of the teacher versus the special ed assistant? Fourth, teaching has effects that continue beyond that school year. For example, how do we know if teacher X got a great gain in scores for students in third grade by using techniques that made them less prepared for fourth grade, or vice versa for teacher Y? The argument behind complicated evaluation scoring systems is that they untangle this complexity sufficiently to measure teacher performance with imperfect but tolerable accuracy.

Any successful company that I have ever seen employs some kind of a serious system for evaluating and rewarding / punishing employee performance. But if we think of teaching in these terms — as a job like many others, rather than some sui generis activity — then I think that the hopes put forward for such a system by its advocates are somewhat overblown.

There are some job categories that have a set of characteristics that lend themselves to these kinds of quantitative “value added” evaluations. Typically, they have hundreds or thousands of employees in a common job classification operating in separated local environments without moment-to-moment supervision; the differences in these environments make simple output comparisons unfair; the job is reasonably complex; and, often the performance of any one person will have some indirect, but material, influence on the performance of others over time. Think of trying to manage an industrial sales force of 2,000 salespeople, or the store managers for a chain of 1,000 retail outlets. There is a natural tendency in such situations for analytical headquarters types to say “Look, we need some way to measure performance in each store / territory / office, so let’s build a model that adjusts for inherent differences, and then do evaluations on these adjusted scores.”

I’ve seen a number of such analytically-driven evaluation efforts up close. They usually fail. By far the most common result that I have seen is that operational managers muscle through use of this tool in the first year of evaluations, and then give up on it by year two in the face of open revolt by the evaluated employees. This revolt is based partially on veiled self-interest (no matter what they say in response to surveys, most people resist being held objectively accountable for results), but is also partially based on the inability of the system designers to meet the legitimate challenges raised by the employees.

I found the point about techniques that hurt futures performance particularly good. When I was teaching, how well a class would go was greatly influenced by how well previous teachers had done their jobs. Did the students understand the foundations? Did they have a good attitude to the material? Good work habits and study strategies?

Teachers want reliable evaluations not just because they want to be rewarded for good work but also because they want to see incompetent teachers identified so that those teachers can be encouraged to do better, given training to improve their performance or, should the first two fail, fired. What they object to is having their fates rest on a glorified roll of the dice.

Catching up with Dana Goldstein

I'm going to try to comment on each of these individually later. Feel free to beat me to the punch.

How Politically Astute is Michelle Rhee?

The Revival of the Private School Voucher Movement

Wisconsin Teachers' Union Prepares for Battle with GOP Gov

"If you don't do well in school, your descendants could grow up to be Morlocks"

OK, maybe it's not that bad, but Berkeley professor Claude Fischer still paints a grim picture. (via Thoma, of course.)

Degree inequality

It is now generally understood that economic inequality has expanded greatly since about 1970. (Well, there are exceptions. For a couple of decades, some commentators denied that economic inequality was growing, claiming that it was all a statistical illusion. A few holdouts against reality may remain.) Now the debate has shifted to what – if anything at all – should be done about inequality.

Most of that discussion has been about income inequality. Between 1979 and 2007, the one-fifth of American households with the highest income experienced a roughly 100% increase in their annual, inflation-adjusted, after-tax income (280% [!] for the highest one percent of households); the middle one-fifth got about 25% more income; and the poorest one-fifth got about 15% more (see pdf). For wealth – property, stocks, and the like – the gap is enormously greater and has also widened over the last few decades.

Less discussed is the widening college degree gap. Yet its implications go considerably beyond money, to widening differences in life experiences and ways of life. (I draw in particular on the work of my colleague, Michael Hout, notably here [pdf], and on two books we wrote together, here and here.)

Fischer follows this with a number of troubling statistics. I found this part particularly striking:

Even is more happening along the education gap: Increasingly, college graduates marry college graduates and live among college graduates. Increasingly, Americans group by education and their ways of life diverge by education.

Although the trends are complex (see here), Americans today are likelier to marry people of the same educational level as themselves than was true decades ago. Some of this development results from educated men increasingly marrying educated women; for example, the lawyer who married his secretary is now a lawyer who marries another lawyer. And some of this change is due to poorly-educated men becoming ineligible as spouses; drop-outs can no longer support families on brawn alone.

Then there is residential separation: A study by Thurston Domina (pdf) shows that college graduates are concentrating in some metropolitan areas (San Francisco and Raleigh-Durham, for example) and seem to be avoiding others (Indianapolis and Las Vegas, for example) and also that neighborhood segregation by college education grew substantially between 1970 and 2000. It grew faster than segregation by income, even as segregation by race declined. Another study documents how the highly-educated are concentrating in the downtowns of the most booming cities. And a recent story reported that these degree-holders are starting to raise their children in center cities — even in Manhattan. Thus, enclaves of the highly-educated are growing in chic, gentrified, non-smoking neighborhoods, while the less educated move to the scraggly, sprawling suburbs of stagnating cities.

What is less clear, although certainly plausible, is that this widening separation carries along with its economic and social divisions, a widening gap in values and ways of life: two different Americas, divided educational attainment.

I would find the use of just desserts as a justification for policy more palatable if we weren't seeing an alarming decline in social mobility.

Another Education Post

Dana Goldstein:

The fact is that where such programs exist, they are oversubscribed--parents don't seem to mind sending their kids out-of-district (sometimes just a 5 minute drive away) into a town where they do not enjoy political representation when the result is a better education.

I think this direction is non-controversial. However, this scenario assumes one of two things:

1) There are more students in the district which is accepting students. As the United States funds a lot of education out of local property taxes, this may create less resources per student in the destination district (and ways to try and balance this out are quite tricky)

2) Some students need to be sent the other way. If the perception is that going the other way leads to a lower quality education (even if untrue in fact) then that could be an issue.

In the second case, I suspect that the lack of representation will be a much bigger issue (as it makes it hard for parents to engage the new district to try and remedy issues that are seen as concerns).

Raising tuition

I have quite a bit to say about this post from Richard Posner but no time to say it at the moment. I'll try to get back to this but in the meantime feel free to start without me.

In any event, there is no case at all from an overall social standpoint for subsidizing students who would pay full college tuition, without the inducement of a subsidy; the subsidy does not induce students to obtain a college education who otherwise would not because they could not afford to; it is a windfall to their families. Private colleges recognize this. They charge very high tuition (though not high enough to cover all their costs—but they have other sources of funds, such as alumni donations), but grant scholarships or loans to students whose families can’t afford the tuition. Charging low tuition to everyone, as public colleges do for residents of the state in which the college or university is located), does not make economic sense; it merely as I said provides windfalls to families willing and able to pay the full tuition. As Becker points out, this results in regressive redistribution of income, because families that can pay full tuition are wealthier than the average taxpayer, who pays for the costs of public colleges that tuition doesn’t cover.

Mini-colloquium: Homework

One of the cool things about about Andrew Gelman's blog is the quality of discussions you'll see in his comment section. The observations are almost always well-informed and the tone strikes a nice balance between spirited and civil.

Prof. Gelman's recent post on homework (which responded to this post we did earlier on OE) prompted one of these excellent discussions. Joseph has already built a post around one particularly interesting comment, but all of them are worth reading.

Evaluating the evaluations

Busy morning so I don't have time to do more than provide some links and the abstract for this paper on the effectiveness of college teachers.

In primary and secondary education, measures of teacher quality are often based on contemporaneous student performance on standard-ized achievement tests. In the postsecondary environment, scores on student evaluations of professors are typically used to measure teaching quality. We possess unique data that allow us to measure relative student performance in mandatory follow-on classes. We compare metrics that capture these three different notions of instructional quality and present evidence that professors who excel at promoting contemporaneous student achievement teach in ways that improve their student evaluations but harm the follow-on achievement of their students in more advanced classes.

Here's the ungated version via Tyler Cowen. May not be quite the same as the published one.

Here's Andrew Gelman's reaction.

Credit where credit is due

Seyward Darby has a thoughtful and balanced piece on education in today's New Republic. Here's the conclusion:

But, even when better evaluation methods exist, is it really a good idea to publish, en masse, the ratings of every public school teacher? I’m not convinced. Yes, the information should be available to those in the public who want it—namely parents. But schools or school districts, not newspapers, should share it with parents in a constructive manner, so that they are able to ask questions and understand fully what the information means. Teachers’ unions and districts should also use it to remove underperforming instructors from their jobs, and to ensure that no school has a high concentration of ineffective teachers, such that its student are getting the short end of the stick. And teachers should use it either to ask for additional training resources—or to gain recognition of the good, hard work that they’ve done.

I’m all for transparency. But a wide-open view of incomplete information isn’t what we need to improve education. What’s more, broadly publicizing even the most thorough of information isn’t always productive; complexities and nuances are often best conveyed in smaller settings, with the stakeholders who matter most.

The media shouldn’t focus on shaming individual teachers, because there are bigger fish to fry. Indeed, across the country, they should focus on shaming the entrenched bodies, structures, and policies that allow poor teaching to continue unchecked, fail to reward good teaching, and don’t provide enough support for teachers who want to improve their skills.

I don't agree with everything Darby says here and there are omissions that bother me (relying on Kyle Spencer's reporting probably didn't help). On the whole, though, this is an entirely reasonable piece of analysis. I complained earlier that there wasn't really an education debate in this country. Perhaps we're about to start one.