This is so funny it made me cry

From Dr. Girlfriend.

Is this not every bio-statistician’s worst nightmare?

They just don't care

I am sure that most people in the news media watch every episode of the Daily Show and the Colbert Report. Then they get up the next morning, go to work, and do their God-damnedest to drag American journalism to its lowest point since Benjamin Franklin Bache accused George Washington of collaborating with the British.

They know what they're doing. They just don't care.

The Daily Show With Jon Stewart	Mon - Thurs 11p / 10c
Islamophobiapalooza
www.thedailyshow.com
Daily Show Full Episodes Political Humor Tea Party

So which one is it?

DeanDad brings this point up in a post today. It's not related to the main point of the post but it is certainly worth noting:

Hell, while we’re at it, let’s make a point of generating enough math teachers so that every state in the country can require four years of math in high school. Get the public K-12 system up to basic competence, and see what happens.

I think, at some level, we need to decide what the priorities in education are. Is there a shortage of competent math teachers? If so then a process in which we make it harder to become a teacher would seem to be counter-productive. After all, we are also grappling with real issues of potential reductions in the number of teachers due to budgetary constraints.

I think that failing to decide on these issues are at the heart of my concerns about education reform. Reformers point to schools like KIPP that have attrtion rates that make them infeasible as a national model. So that doesn't seem to be a way forward. But in the actual education system, we are discussing reducing resources and trying to compensate with higher quality.

Now, add in that the metrics used to evaluate schools have serious concerns (as pointed out by a wide variety of researchers) and it gets hard to see what the road forward looks like. Clearly, if we are reducing resources to education then we can't achieve this with a higher investment. We might gain some efficiency by breaking contracts with current teachers (over tenure and pensions) but such actions tend to increase costs in the long run.

So I think that the real thing that I want to see out of educational reform is specific proposals. Honestly, I suspect that a series of initiatives at the school district level (focusing on the issues in each area) might be the way to go. But I worry that the current approach seems far too focused on test scores and not on the actual process of education.

The appeal of bad ideas -- why do rational executives keep trying ideas that never work?

A while back my friend Brian (of Ultrasonic Remote) and I found ourselves on the subject of TV remakes, shows that kept the title and general premise of an old hit but rebooted it with a new cast. After a while a pattern became obvious, the few successes all had something in common: they were all science fiction or fantasy shows (and even there the odds were not good).

When we went out of that subgenre, we couldn't come up with a single success. Many of the shows were high profile affairs with talented casts (Kojak, Dragnet, the Fugitive) but we are still talking about a failure rate of pretty much 100%.

Eventually, some show will break the streak. Perhaps it will be Hawaii Five-O or the new Rockford Files (after all, how hard could it be to replace James Garner?), but this still begs the question, why do highly paid executives in a tremendously competitive field insist on trying things that have repeatedly and consistently failed?

(Acknowledging) Failure Is Not an Option

Here at OE, we have been spending a lot of time discussing the horrible student attrition rates in Bay area KIPP schools, but when I first came across the statistic, I hesitated before using it in a post. The number was so absolutely damning that I had to wonder about the source. Had some anti-charter school group gone through all the data and cherry-picked the worst thing they could find?

So I looked up the source of the statistic and found a veritable sales brochure for KIPP. The article opened with the headline:

"New Study Finds San Francisco Bay Area KIPP Students Outperform Peers"

followed by the subtitle:

"Combination of Key Features Contributes to Success, Provides Lessons for Other Public Schools"

The rest of the article continues in the same vein before finally getting to this:

As researchers analyzed the student achievement data and KIPP’s approach, they also identified challenges facing Bay Area KIPP schools, including high student attrition rates, teacher turnover, and low state and local funding. For example, 60 percent of students who entered fifth grade at four Bay Area KIPP schools in 2003-04 left before completing eighth grade. Annual teacher turnover rates have ranged from 18 to 49 percent since 2003-04.

Putting aside the human cost paid by the students who fell into that sixty percent and their families (which is huge), the selection/attrition process described in this post results in a school filled with roughly the top quartile of hard-working, dedicated students. All but the most incompetent administrators will have spectacular results under those circumstances. The performance described here ("In most grades, Bay Area KIPP students make above-average progress compared with national norms, and four out of five KIPP schools outperform their host district.") is, if anything, on the low end of what we would expect yet it is written up in the most glowing terms imaginable.

When it comes to charter schools, there's a long tradition of bad news stories with positive or at least neutral headlines. Consider this article from the New York Times. The key paragraph was:

But for all their support and cultural cachet, the majority of the 5,000 or so charter schools nationwide appear to be no better, and in many cases worse, than local public schools when measured by achievement on standardized tests, according to experts citing years of research. Last year one of the most comprehensive studies, by researchers from Stanford University, found that fewer than one-fifth of charter schools nationally offered a better education than comparable local schools, almost half offered an equivalent education and more than a third, 37 percent, were “significantly worse.”

And what was the headline for this grim finding?

"Despite Push, Success at Charter Schools Is Mixed"

I genuinely believe that there are some wonderful charter schools out there and we can greatly improve education by building on these successes, but in order to do that we need to recognize and honestly admit the failures.

Up until now, that honesty has been in short supply.

Substituting metrics for judgement

This episode of This American Life concludes with a fascinating account of the manipulation of crime statistics. Drop by, take a listen, give them a buck.

If reformers are looking for an indictment of our educational system...

I'd like to put some finer shadings on Joseph's recent post. I think I speak for most former and current teachers when I say that my favorite part of teaching was teaching -- explaining concepts, helping students working through problems, convincing them that they can not only do math, they can actually be good at it.

Helping a student who has shown an interest in learning always takes a high priority, but at some point something has to give. When I was teaching I routinely worked fifty plus hours a week when you count grading and making lesson plans. Add being on call ten or more hours a week and you will eventually have to make a choice. You can blow off the kids, you can blow off the grading and lesson plans or you can find out how long you can go without seeing your family or getting over six hours sleep.

As far as I can tell, the underlying assumption of the KIPP business model is an unlimited supply of Erin Gruwells. In case you've forgotten, Ms. Gruwell was the... let's say, inspiration for the character played by Hillary Swank in the movie Freedom Writers. Ms. Gruwell has managed an exceptional career but as a teacher, she followed a very common trajectory. She came into the field with high expectations and tremendous energy, worked incredibly long hours at a fantastic pace, achieved some early successes then bailed in less than five years.

KIPP burns through these teachers at an extraordinary rate ("Annual teacher turnover rates have ranged from 18 to 49 percent since 2003-04"). This is possible in large part because there are less than twenty-seven thousand students in the KIPP system. By charter school standards, this is substantial but there are over sixty-million K through 12 children in the U.S.

A number of people think KIPP is a scalable model for the nation. Many of these people hold advanced degrees from some of America's most prestigious universities. If reformers are looking for an indictment of our educational system, I can't think of a more damning one.

Cost of Education

I was speaking with Mark about this post and an interesting point came up. Teachers already have long and structured hours (pre and post class supervision, class hours and lunch duty) plus work at home (marking -- which at the high school level is never finished during the workday). KIPP has teachers also available by cell phone. In the private sector it generally involves a pretty hefty pay increase to put an associate on a pager.

When we talk about the need to increase our investment in education, are we really prepared to increase either the number of teachers or the median salary of teachers? After all, we are currently talking about teacher layoffs. Perhaps Mark, who has more practical experience in this matter, can comment?

Reflections on the KIPP posts

I want to explicitly emphasize this point in Mark's last post:

One is the eighty/twenty rule. Some students take more time than others and, not surprisingly, the students who lower a school's test average and management metrics are the ones who consume the most time and resources.

In the light of the KIPP website (which he discusses here):

Having teachers available by cell phone after school for homework help

Now, let us consider two effects of this policy. One, the school day is a lot longer for a teacher than it used to be. Add in the rigorous extracurricular activities and the (seemingly 24 hour on call status) and at best you are creating an environment where investing more resources in teaching creates better outcomes.

But I don't think that this is going to be the main effect. If you consider Mark's eighty/twenty rule, you create enormous incentives for teachers to gently nudge out the student who calls for an hour and half to discuss homework every single evening. Even if the teachers at KIPP don't do this (a proposition of which I would be highly sceptical), I would be deeply concerned about what whether this approach was scalable.

It think it would be critical to determine if an approach was scalable before basing massive reforms off of it. I am happy for innovation and improvement in education to continue but I think that we should proceed carefully and based on the evidence. In education, for reasons that evade me, people seem to be very non-critical about the evidence.

Make that most children left behind

In an email, Joseph accused me of burying the lede when I discussed KIPP charter schools in an earlier post. The paragraph in question was this quote from Wikipedia:

"In addition, some KIPP schools show high attrition, especially for those students entering the schools with the lowest test scores. A 2008 study by SRI International found that although KIPP fifth-grade students who enter with below-average scores significantly outperform peers in public schools by the end of year one, "... 60 percent of students who entered fifth grade at four Bay Area KIPP schools in 2003-04 left before completing eighth grade."[7] The report also discusses student mobility due to changing economic situations for student's families, but does not directly link this factor into student attrition. Six of California's nine KIPP schools, researched in 2007, showed similar attrition patterns.[citation needed] Figures for schools in other states are not always as readily available."

Administrators have long known that the simplest and most reliable way to improve a school's performance is by selection and attrition of the student body. This works in the obvious, direct ways -- if you drop the kids who disrupt class and/or can't master the material, test scores and classroom management metrics will go up -- but there are at least a couple of indirect effects that are as, or more, powerful.

One is the eighty/twenty rule. Some students take more time than others and, not surprisingly, the students who lower a school's test average and management metrics are the ones who consume the most time and resources.

Even more significant are peer effects. K through 12 students are particularly sensitive to perceived social norms. By selectively removing certain students from the population, you can easily create a high degree of conformity to an almost ideal set of behaviors and attitudes.

Most educators look at getting rid of students as a last resort. The prevailing attitude is that you are there to help all the kids, not just the easy ones. You will find exceptions, of course, like principals who are a little too eager to expel certain students or find ways to influence which students are assigned to other schools, but they are forced to work around rules that discourage this behavior.

For charter schools, though, these selective factors are built into the system. Here's the deal that college prep charter schools offer, if the students go through an involved application/induction process, take more difficult courses during a longer school day and do more homework, they will have a better chance at academic and professional success. These schools have automatically limited their pool of applicants to kids who consider academic and professional success both desirable and attainable and who come from supportive, involved families that are willing to make a real effort to give their children a chance to succeed.

(quick clarification: we are talking about selection processes that favor certain attitudes and behaviors. We are NOT talking about favoring students with high test scores. Many reporters and, God help us, researchers have failed to grasp this distinction.)

Charter schools segregate out a group of students who tend to be, to put it bluntly, easy to teach. This is not an entirely bad thing. Though there are concerns about this being a zero-sum-game, there is something to be said for making sure that every neighborhood has at least one educational bright spot.

It is, however, an entirely bad thing when dishonest or naive observers evaluate these schools without taking these systemic advantages into account and it is worse still when unscrupulous administrators try to build on those systemic advantages by cooking the data through selective attrition.

If you start with a student body made up almost entirely of kids who want to be in school, avoid gangs and graduate from college, who are supported by families with the same goals, you should expect relatively low attrition. When you see the opposite, you should certainly be suspicious. Even the most inept administrator can look good if you allow him or her to pick the most promising students out of an already select group.

The 'I' stands for 'Ironic'

Check out bullet two. (From the KIPP website)

At KIPP schools:

• Parents are encouraged to be involved and to contribute;
• Teachers have the freedom to innovate and never give up on a child;
• Students work hard and come to school ready to learn.

Update: While you're here, why not take a look at the next post

Rationality

From Econned page 148:

Greenspan was hardly alone in his dogmatic belief in the wisdom of leaving “free markets” to their own devices . . . Judge Frank Easterbrook and Daniel Fishel, then University of Chicago Law School dean, asserted in 1991: “[A] law against fraud is not an essential or necessarily important ingredient of securities markets.”

I find this line of thinking to be rather remarkable. The two classic libertarian roles for government are to prevent force and fraud. Markets can’t be efficient if information is missing or removed – in a world where identities can be re-invented then you simply can’t have a national (let alone) global market if fraud is a permissible business strategy. The only way to trust people with your money or in business deals would be a long history of relationships.

This line of thinking requires one to think that there are no information asymmetries in the real world. Furthermore, a rational agent permitted to engage in fraud might be willing to destroy their relationship for a large enough pay-off. Yves Smith gives an example of how this thinking is leads to odd conclusions:

Easterbrook went so far as blocking a plaintiff from presenting a case that argued that an auditor has assisted in a fraud. The judge claimed it would be “irrational” for the accountant to behave in that way, given his interest in preserving his reputation.

Of course, one could argue that all potential criminals have an interest in preserving their reputation. But one might very well risk their reputation for a sufficiently high pay-off – especially if one is highly unlikely to be prosecuted.

I do not think that this type of thinking is universal among “free market” advocates but it is extremely concerning that these arguments can be advanced without public mockery.

More from Yves Smith

An interesting point in Econned (pages 223-224):

One way to contain compensation is for the central bank to raise interest rates when inflation starts to build. The logic is that increasing unemployment will moderate pay pressures and also discourage business from giving employers pay increases in excess of productivity gains. Ironically, quite a few “free markets” supporters endorse this type of intervention to correct a perceived marker failure (labor having undue bargaining power) but reject a raft of others.

One of the hard things about free markets is that everybody seems to want to meddle with at least one small portion that they see as problematic. However, if you are not careful you can end up with unexpected consequences as markets respond to incentives in unexpected ways. This is not to say that inflation is a good thing (it isn’t) but rather to recognize that real markets have limitations and failures.

This example is particularly interesting as one of the great issues of the moment is how powerless workers have become in the face of tight credit markets.

Fishing expeditions

This has to be a bad idea. Chronic pain is a difficult condition to treat already with an extremely limited set of options; second guessing treatment decisions seems to be likely to lead to more suffering in the long run.

Changing statistical languages

Even when a current programming language has drawbacks, it can be hard to change to a more optimal language due to the investment in the current language. Here is a comment from Julien Cornebise:

But R is here and in everyday use, and the matter is more of making it worth using, to its full potential. I have no special attachment to R, but any breakthrough language that would not be entirely compatible with the massive library contributed over the years would be doomed to fail to pick-up the everyday statistician—and we’re talking here about far-fetched long-term moves. Sanitary breakthrough, but harder to make happen when such an anchor is here.

R is a pretty amazing language and, as a long term SAS user, I must admit that I am delighted by the graphics and the cool packages. Of course, the dark side of such a rich library is needing to learn about the reliability and limitations of all of the different packages.

H/t: Andrew Gelman