If you want to understand what's wrong with American education...

You need to follow stories like this.

The Annotated "Evaluating New York Teachers, Perhaps the Numbers Do Lie"

As promised, here are some comments (in brackets) on Michael Winerip's NYT article on the city's teacher evaluation process.

Last year, when Ms. Isaacson was on maternity leave, she came in one full day a week for the entire school year for no pay and taught a peer leadership class.

...

[One thing that Winerip fails to emphasize (though I suspect he is aware of it) is how common stories like this are. Education journalists often portray ordinary excellence as something exceptional. This is partly due to journalistic laziness -- it's easier to describe something as exceptional than to find something that actually is exceptional -- and partly due to the appeal of standard narratives, in this case the Madonna/whore portrayal of teachers (I would used a non-gender specific analogy but I couldn't come up with one that fit as well.)]

The Lab School has selective admissions, and Ms. Isaacson’s students have excelled. Her first year teaching, 65 of 66 scored proficient on the state language arts test, meaning they got 3’s or 4’s; only one scored below grade level with a 2. More than two dozen students from her first two years teaching have gone on to Stuyvesant High School or Bronx High School of Science, the city’s most competitive high schools.

...

[Everything in this article inclines me to believe that Ms. Isaacson is a good teacher but we need to note that this is a fairly easy gig compared to other urban schools, particularly for someone with her background. Students at places like the Lab School tend to be more respectful and attentive toward academically successful people like Ms. Isaacson. In many schools, this can actually make students initially distrustful.]

You would think the Department of Education would want to replicate Ms. Isaacson — who has degrees from the University of Pennsylvania and Columbia — and sprinkle Ms. Isaacsons all over town. Instead, the department’s accountability experts have developed a complex formula to calculate how much academic progress a teacher’s students make in a year — the teacher’s value-added score — and that formula indicates that Ms. Isaacson is one of the city’s worst teachers.

According to the formula, Ms. Isaacson ranks in the 7th percentile among her teaching peers — meaning 93 per cent are better.

[One of the fallacies that follow from this Madonna/whore narrative is the idea that, since you have such a clearly bi-modal distribution, any metric that's correlated with teaching quality should be able to winnow the good from the bad. In reality you have a normal distribution with noisy data and a metric that doesn't correlate that well. The result, unsurprisingly, is a large number of teachers apparently misclassified. What is surprising is that more people didn't foresee this fairly obvious outcome.]

This may seem disconnected from reality, but it has real ramifications. Because of her 7th percentile, Ms. Isaacson was told in February that it was virtually certain that she would not be getting tenure this year. “My principal said that given the opportunity, she would advocate for me,” Ms. Isaacson said. “But she said don’t get your hopes up, with a 7th percentile, there wasn’t much she could do.”

That’s not the only problem Ms. Isaacson’s 7th percentile has caused. If the mayor and governor have their way, and layoffs are no longer based on seniority but instead are based on the city’s formulas that scientifically identify good teachers, Ms. Isaacson is pretty sure she’d be cooked.

[Well, as long as it's scientific.]

She may leave anyway. She is 33 and had a successful career in advertising and finance before taking the teaching job, at half the pay.

...
[This isn't unusual. I doubled my salary when I went from teaching to a corporate job. Plus I worked fewer hours and they gave us free candy, coffee and the occasional golfing trip.]
...

The calculation for Ms. Isaacson’s 3.69 predicted score is even more daunting. It is based on 32 variables — including whether a student was “retained in grade before pretest year” and whether a student is “new to city in pretest or post-test year.”

Those 32 variables are plugged into a statistical model that looks like one of those equations that in “Good Will Hunting” only Matt Damon was capable of solving.

The process appears transparent, but it is clear as mud, even for smart lay people like teachers, principals and — I hesitate to say this — journalists.

[There are two things about this that trouble me: the first is that Winerip doesn't seem to understand fairly simple linear regression; the second is that he doesn't seem to realize that the formula given here is actually far too simple to do the job.]


Ms. Isaacson may have two Ivy League degrees, but she is lost. “I find this impossible to understand,” she said.

In plain English, Ms. Isaacson’s best guess about what the department is trying to tell her is: Even though 65 of her 66 students scored proficient on the state test, more of her 3s should have been 4s.

But that is only a guess.

[At the risk of being harsh, grading on a curve should not be that difficult a concept.]

Moreover, as the city indicates on the data reports, there is a large margin of error. So Ms. Isaacson’s 7th percentile could actually be as low as zero or as high as the 52nd percentile — a score that could have earned her tenure.

[Once again, many people saw this coming. Joel Klein and company chose to push forward with the plan, even in the face of results like these. Klein has built a career largely on calls for greater accountability and has done very well for himself in no small part because he hasn't been held accountable for his own record.]

I've left quite a bit out so you should definitely read the whole thing. It's an interesting story but if anything here surprises you, you haven't been paying attention.

"Jefferson is spinning in his grave"

There's nothing remarkable about this quote until you see why the speaker thought Jefferson would be offended.

The lessons of Motown

Michael Winerip is a much better than average education reporter. He doesn't have a great grasp of the numbers or of the implications of the policies, limitations which have kept him from getting a jump on the story the way, say, This American Life did with the financial crisis, but he has kept up with it while most of his colleagues are still reporting discredited narratives from interested parties.

This article on Detroit is a good example. He doesn't connect some important dots but he does a good job reporting what he sees. (you'll find my comments in brackets):

In 2009, Detroit public schools had the lowest scores ever recorded in the 21-year history of the national math proficiency test.

The district had a budget deficit of $200 million.

About 8,000 students were leaving Detroit schools each year.

Political leaders had to do something, so they rounded up the usual whipping boys:

Wasteful bureaucrats. In 2009, the governor appointed an emergency financial manager, Robert Bobb, a former president of the Washington school board, to run the Detroit district. Mr. Bobb is known nationally for his work in school finance, and recruiting him took a big salary, $425,000 a year. He has spent millions more on financial consultants to clean up the fiscal mess left by previous superintendents.

[A large number of people are acquiring a great deal of money and power through the reform movement. This doesn't mean that these people don't have good intentions or that they are not worth the money they're being paid but it does mean that this group, which includes high profile figures like Joel Klein and Michele Rhee, has a vested interest in these policies. It also means that when Michele Rhee brags about being a counterbalance to the special interests, she's not being entirely honest.]

Greedy unions. Though Detroit teachers make considerably less than nearby suburban teachers (a $73,700 maximum versus $97,700 in Troy), Mr. Bobb pressed for concessions. He got teachers to defer $5,000 a year in pay and contribute more for their health insurance. Last week, the Republican-controlled Legislature approved a bill to give emergency managers power to void public workers’ contracts. If signed by the governor, Mr. Bobb could terminate the Detroit teachers’ union contract.

Traditional public schools full of incompetent veteran teachers. Michigan was one of the first states to embrace charter schools, 15 years ago. Currently there are as many Detroit children in charters — 71,000 — as in district schools. Now there is talk of converting the entire Detroit district (which is 95 percent African-American) to charters. Supporters say this could generate significant savings, since charters are typically nonunion and can hire young teachers, pay them less and give them no pensions.

[Before we go on, this would seem to be an almost perfect test of the large-scale charter school model (as compared to the more limited role I've advocated). Charter schools have been put forward as the solution for this very kind of troubled urban district.]

So now, two years later, how are the so-called reforms coming along?

Not great.

Since Mr. Bobb arrived, the $200 million deficit has risen to $327 million. While he has made substantial cuts to save money — including $16 million by firing hundreds of administrators [Of course, he's spent millions making those cuts] — any gains have been overshadowed by the exodus of the 8,000 students a year. For each student who departs, $7,300 in state money gets subtracted from the Detroit budget — an annual loss of $58.4 million.

[Economic conditions and demographic shifts still trump any educational reform proposed so far. People need to remember this.]

Nor have charters been the answer. Charter school students score about the same on state tests as Detroit district students, even though charters have fewer special education students (8 percent versus 17 percent in the district) and fewer poor children (65 percent get subsidized lunches versus 82 percent at district schools). It’s hard to know whether children are better off under these “reforms” or they’re just being moved around more.

[As mentioned before, there are a number of possible biasing effects (peer, placebo, Hawthorne, selective attrition, etc.) that may be inflating the charter's scores. In other words, they are not outperforming the public schools and they may be doing much worse.]

Steve Wasko, public relations director for Mr. Bobb and the Detroit schools, did not respond to a dozen voice mails and e-mails seeking comment. Those who know Mr. Wasko say he cares about Detroit and is sick of the national media portraying the city as hopeless.

[You have a public relations director who can't work a talk with the New York Times into his schedule. This alone raises questions about the Bobb administration. It also suggests some other options for cost cutting (who do you think makes more, a starting teacher or a public relations director?)]

...

Last spring, Mr. Bobb had planned to close 50 schools with dwindling enrollment. But his list was reduced to 30 after several public meetings at which parents and staff members pleaded their school’s case before the all-powerful Mr. Bobb.

In June, Mr. Bobb held a news conference at Carstens Elementary — one of the schools spared — to announce the 30 closings.

One reason Carstens survived was an article in The Detroit Free Press last March headlined “Carstens Elementary on DPS closing list is a beacon of hope.”

The school, surrounded by vacant lots and abandoned houses, serves some of the city’s poorest children. Thieves who broke into the school last year escaped by disappearing into what the police call “the woods” — the blocks and blocks of vacant houses.

Yet Carstens students perform well on state tests, repeatedly meeting the federal standard for adequate yearly progress.

[As seen before, good teachers and schools often end up bearing the brunt of our current crop of reforms.]

“We try to fill in the holes in our children’s lives,” said Rebecca Kelly-Gavrilovich, a Carstens teacher with 25 years’ experience. Students get free breakfast, lunch and — if they attend the after-school program — dinner.

To have more money for instruction, teachers sit with students at lunch, saving the school from having to hire lunchroom aides. Teachers hold jacket and shoe drives for children who have no winter coats and come to school in slippers. At Thanksgiving every child goes home with a frozen turkey donated by a local businessman. Twice a year a bus carrying a portable dentist’s office arrives, and a clinic is set up at the school so children can get their teeth checked.

Despite all this, teachers worry that Carstens’s appearance on Mr. Bobb’s closing list — even though it was brief — means the end is near. Anticipating the worst, several parents have taken their children out of Carstens, enrolling them elsewhere, including at charters and suburban schools.

Carstens’s enrollment is half of what it was a few years ago. Every hallway has empty classrooms, giving the school a desolate feeling.

Mr. Bobb has set off a vicious cycle undermining even good schools. The more schools he closes to save money, the more parents grow discouraged and pull their children out. The fewer the children, the less the state aid, so Mr. Bobb closes more schools.

[This is a pattern we're seen before. Check Ravitch for specifics.]

Carstens has also been harmed by poor personnel decisions made by the district. Last year, 1,200 teachers took the retirement buyout, and Mr. Bobb laid off 2,000 others in the spring. Then in the fall, he realized he needed to hire the 2,000 back, and chaos ensued.

[Also something we've been warning about.]

At Carstens, a kindergarten class of 30 had no teacher until October; teachers at the school took turns supervising the class. “How do you think parents feel when there’s a different teacher every day?” said Mike Fesik, the current teacher.

It’s hard to understand why any teacher who could leave Detroit stays, but they do. Kim Kyff, with 22 years’ experience, is one of the lead teachers at Palmer Park, the elementary and middle school that opened last fall. In 2007 she was the Michigan teacher of the year. She has had offers from suburban schools, but stays because she believes that in Detroit, she has a better shot at being a beacon of hope.

[I should say more but it's late and I'm already depressed as hell. It's times like these when I have trouble not believing that, not only do we not care about children, we actually go out of our way to screw over those who do care. (yes, that's a lot of 'not's but given the hour what do you expect?)]

Felix Salmon catches me off guard

When I see a headline like this:

Don’t donate money to Japan

I expect it to come from a Rush Limbaugh or some other professional xenophobe. Seeing it on Felix Solmon's site caught me off guard until I realized he was talking about earmarking money for Japan when you give:

We went through this after the Haiti earthquake, and all of the arguments which applied there apply to Japan as well. Earmarking funds is a really good way of hobbling relief organizations and ensuring that they have to leave large piles of money unspent in one place while facing urgent needs in other places. And as Matthew Bishop and Michael Green said last year, we are all better at responding to human suffering caused by dramatic, telegenic emergencies than to the much greater loss of life from ongoing hunger, disease and conflict. That often results in a mess of uncoordinated NGOs parachuting in to emergency areas with lots of good intentions, where a strategic official sector response would be much more effective. Meanwhile, the smaller and less visible emergencies where NGOs can do the most good are left unfunded.
...

That said, it’s entirely possible that organizations like the Red Cross or Save the Children will find themselves with important and useful roles to play in Japan. It’s also certain that they have important and useful roles to play elsewhere. So do give money to them — and give generously! And give money to other NGOs, too, like Doctors Without Borders (MSF), which don’t jump on natural disasters and use them as opportunistic marketing devices. Just make sure it’s unrestricted. The official MSF position is exactly right:

The ability of MSF teams to provide rapid and targeted medical care to those most in need in more than 60 countries around the world – whether in the media spotlight or not – depends on the generous general contributions of our donors worldwide. For this reason, MSF does not issue appeals for support for specific emergencies and this is why we do not include an area to specify a donation purpose on our on-line donation form. MSF would not have been able to act so swiftly in response to the emergency in Haiti, as an example, if not for the ongoing general support from our donors. So we always ask our supporters to consider making an unrestricted contribution.

I’ve just donated $400 in unrestricted funds to MSF. Some of it might go to Japan; all of it will go to areas where it’s sorely needed. I’d urge you to do the same, rather than try to target money at whichever disaster might be in the news today.

At the risk of overselling, when it comes to questions involving money, whether investing it or giving it, Felix Solmon's blog should probably be the first place you click.

Science Fiction Thoughts

In an interesting post, Dominic Lukes asks if science fiction authors dream of dictatorships. It is an interesting question. Certainly one of the examples that he gives, Robert Heinlein (author of Starship Troopers) is a noted Libertarian. So there is certainly something to this.

But I think the key issue is that science fiction authors are, at least partially, about social criticism. Look at the more recent series of Battlestar Galactica, for example. Or the novels of authors like Mercedes Lackey or Elizabeth A. Lynn that dealt with gender in a very sophisticated way.

So I think we can find all sorts of extrapolations and thought experiments in science fiction. This is, in my opinion, the real strength of the genre and its most important function.

[and yes, I am one of those people who sees Fantasy as utterly distinct from Science Fiction; and Star Wars was a Fantasy]

Nobody talks about the Caulfield problem

This is one of those fundamental, difficult-to-resolve questions with important implications that almost no one in the education debate has any interest in discussing. How do we handle students learning things that aren't "on the list"?

There are valid arguments to be made on both sides. Some of the happiest educational outcomes often start with something far from the standard curriculum. (Goosebumps leads to King who leads to Lovecraft who leads to Machen who leads to Joseph Campbell. Puzzle books lead to Gardner who leads to Smullyan who leads to Gödel and Tarski.)

But as valuable as these excursion may be, there is still a case to be made for a body of essential knowledge, things that everyone should know. Not only are things that make it into the curriculum considered more important; the very fact that they are common is itself valuable. A diverse, democratic society functions better when its people have a shared frame of reference.

Those in the reform movement have come down heavily on the side of valuing only what falls within the curriculum. Schools are actually penalized when students go off-list since time spent reading Gödel, Escher, Bach or the Hero with a Thousand Faces is being taken away from learning those things that are being measured. That is a perfectly defensible position but I get the impression that it is largely an unintentional one, that proponents of a system built around standardized testing often failed to think through the implications of their policies.

The paradox of New Haven

Mark has been discussing Edward Glaeser and his comments on how universities can create urban prosperity. Now, I am a big fan of universities and think that they serve an important role in global economic development. However, I am dubious that they make any particular community prosperous. Consider New Haven, CT -- the home of Yale University (recently ranked the #11 university in the world).

According to wikipedia, the poverty rate in New Haven is 24%, which compares unfavorably with the rest of the United States where it is 14%. The poverty rate in New Haven, despite the presence of Yale, is nearly twice that of the United States as a whole.

Now, one might note that many of the poor residents of New Haven are likely to be students. This is true. But these students still use municipal services and thus require the local tax base to support them (in addition to the long term residents). They do not (after they graduate and make additional income) send money back to New Haven so, in a sense, New Haven is actually subsiding the urban communities that Yale graduates move to.

So, it is actually possible that a large university in a small community could be a drag on the economy due to the lower per capita tax base. Plus, you have a large segment of the population with only a short term interest in the community which may make long term planning more difficult. And New Haven, CT is not the only university town that I can think of with high levels of poverty.

Furthermore, if a strong local university (like the University of Washington) is a solution to urban poverty (as it was presented in the Detroit versus Seattle comparison of Edward Glaeser) then it is unclear why a stronger economy has not grown up around Yale which is a strong school by any measure.

This example highlights, I think, that predicting what factors create a prosperous community is a difficult question and not one that has easy answers.

Update: The discussion continues here and here.

Nate Silver takes apart another favorite political myth

I get out of the habit of reading 538 when we don't have an election looming and I miss out on things like this:

There’s Nothing Special About Ohio

Weekend Gaming -- The Game of Gale

As promised, here's an introduction to David Gale's elegant Game of Gale (and, no, Gale didn't come up with the name; Martin Gardner did)"

... Since then it has been a board game (Bridg-it), a programming exercise and a great pencil-and-paper game (or at least, a great colored-pens-and-paper game). GoG is a close relative of Hex and its rules are comparably simple. Start out with a grid of dots like the one below.

Each player gets a color then takes turns connecting his or her dots. The object for the blue player is to draw a continuous path from the top row to the bottom. The objective for the red player is to connect the leftmost column and the right. The only rule is that a red line can't cross a blue and vice versa.

There's a sample game at the site. Here's the decisive turn:



Red gets to go next but it doesn't really matter at this point. Blue has two winning moves for the next turn. Red can only block one of them.

I don't know what to make of this...

From the Monkey Cage (via DeLong):

Does anyone out there know Michael Winerip?

If so, please tell him that he needs to read this discussion on Andrew Gelman's blog. There are things he that he really ought to know.

Folsom Prison Blues

I previously alluded to some threats to California's spectacular university system. The most ominous one is a product, believe it or not, of an alliance between two Republican governors and a public sector union.

NPR did a brilliant job laying out the whole tragic story:

In January 1968, Johnny Cash set up his band on a makeshift stage in the cafeteria at Folsom State Prison in California.

"Hello, I'm Johnny Cash," he said in his deep baritone to thunderous applause. Song after song, the inmates thumped their fists and cheered from the same steel benches now bolted to the floor.

The morning that Cash played may have been the high-water mark for Folsom — and for the California Department of Corrections and Rehabilitation.

The men in the cafeteria lived alone in their own prison cells. Almost every one of them was in school or learning a professional trade. The cost of housing them barely registered on the state budget. And when these men walked out of Folsom free, the majority of them never returned to prison.

It was a record no other state could match.

Things have changed. California's prisons are all in a state of crisis. And nowhere is this more visible than at Folsom today.

Folsom was built to hold 1,800 inmates. It now houses 4,427.

It's once-vaunted education and work programs have been cut to just a few classes, with waiting lists more than 1,000 inmates long.

Officers are on furlough. Its medical facility is under federal receivership. And like every other prison in the state, 75 percent of the inmates who are released from Folsom today will be back behind bars within three years.

California's prison system costs $10 billion a year. Its crumbling, overcrowded facilities are home to the highest recidivism rate in the country. And the state that was once was the national model in corrections has become the model every state is now trying to avoid.

...

Experts agree that the problem started when Californians voted for a series of get-tough-on-crime laws in the 1980s. The state's prison population exploded immediately. It jumped from 20,000 inmates, where it had held steady throughout the 1960s, 1970s and early 1980s. Today there are 167,000 inmates in the system.

Jeanne Woodford was warden of San Quentin during the prison population boom.

"The violence just went out of control," she remembers. "And then the programs started going away. I was there during an 18-month lockdown. It was just unbelievably horrific."

California wasn't the only state to toughen laws in the throes of the 1980s crack wars. But Californians took it to a new level.

Voters increased parole sanctions and gave prison time to nonviolent drug offenders. They eliminated indeterminate sentencing, removing any leeway to let inmates out early for good behavior. Then came the "Three Strikes You're Out" law in 1994. Offenders who had committed even a minor third felony — like shoplifting — got life sentences.

Voters at the time were inundated with television ads, pamphlets and press conferences from Gov. Pete Wilson. "Three strikes is the most important victory yet in the fight to take back our streets," Wilson told crowds.

But behind these efforts to get voters to approve these laws was one major player: the correctional officers union.

In three decades, the California Correctional Peace Officers Association has become one of the most powerful political forces in California. The union has contributed millions of dollars to support "three strikes" and other laws that lengthen sentences and increase parole sanctions. It donated $1 million to Wilson after he backed the three strikes law.

We currently spend as much on prisons as we do on higher education. One of these two has to give.

One way of handling worrisome demographic shifts in the electorate

On the bright side, this does have a certain directness going for it (from Yahoo):

Boosted by major electoral gains in state legislatures nationwide in the 2010 campaign, Republican lawmakers in 32 states are pushing measures that would require citizens to show a state identification or proof of citizenship to vote. Meanwhile, in New Hampshire, GOP lawmakers are proposing new limits on college students who vote in the state, potentially eliminating a key base of electoral support for Democrats in the state ahead of the upcoming presidential election.

As the Washington Post's Peter Wallsten writes, the measures have set off a partisan battle over voting rights across the country, with Democrats accusing Republicans of trying to suppress voters, including young people and minorities, who would cast their ballots for President Obama and other Democratic candidates next year.

In New Hampshire, Republicans are pushing to end rules that allow same-day voter registration in the state, which has often provided key swing votes for candidates from all parties in the state. State GOP lawmakers are also proposing new limits on students, including a bill that would allow them to vote in college towns only if they or their parents had established permanent residency in the state.

Some GOP lawmakers in New Hampshire have billed the measures as an attempt to crack down on voter fraud in the state--but recent remarks from the newly elected GOP state House speaker have suggested otherwise.

In a recent speech to a tea party group in the state, House Speaker William O'Brien described college voters as "foolish." "Voting as a liberal. That's what kids do," he said, in remarks that were videotaped by a state Democratic Party staffer and posted on YouTube. Students, he said, lack "life experience" and "just vote their feelings."

GOP lawmakers in the state have distanced themselves from O'Brien's remarks.

Though not his policies.

A useful table and a few observations

Wikipedia has an interesting article on academic rankings. We could (and probably should) spend a lot of time going over this, but here are a few points to get things started:

1. This list seems to suggest that the USA has the best university system by a good margin. This doesn't mean we shouldn't try to improve it, but this does make a case for being careful about making radical changes. When you're number one, unintended consequences can be a bitch;

2. As Joseph noted, California contains one of the two major clusters of major universities. Most Californians would like to keep it that way. I'm not so sure about most of Sacramento.

3. It's a good idea to go through this list periodically while reading Edward Glaeser (since it's obvious his editors didn't).