Jon Chait has to be pranking us?

I say this because of this review he posts of a book that seems deeply ahistorical:

The new right-wing book of the moment is American Betrayal, by Diana West, which offers the thesis that American foreign policy under presidents Roosevelt, Truman, and Eisenhower was secretly controlled by the Soviet Union. If this sounds like the sort of tract that would be written by somebody who thinks Joe McCarthy was absolutely right about everything, well, that’s exactly what Diana West does believe. She’s also quite the birther.

and

West argues not just that there were communist spies in the government or that American foreign policy failed to take a strong enough line, but that the Soviets controlled the American government in the way they controlled, say, East Germany. For instance, West argues that America provoked Japan into attacking Pearl Harbor — an American working for the Soviets “subverted relations between the US and Japan by inserting ‘ultimatum’ language into the cable flow that actually spurred the Japanese attack.”

So the downfall of the communist US government was due to John F Kennedy, lion of the right wing?  Who managed to squeak out a victory over Soviet pawn Rich Nixon (Eisenhower's Vice President, after all)?  Making JFK and Lyndon Johnson leading the only actual American led government in this 40 year period of subjugation?  Making the Great Society what a non-communist government would introduce in opposition to the communism of Richard Nixon? 

I am not saying that these things aren't possible in the sense of "logically possible".  But the Korean war and the Berlin blockage seem like odd policy decisions if the United States was a soviet satellite in the sense of East Germany.  Look what happened to Hungary in 1956 when it decided to buck Soviet control!  Now, this is completely separate from a claim that these presidents were rabid socialists or a competing strain of communist (kind of like China was).  It is the direct control from Moscow that seems to beg for a very high level of evidence.

So I guess I prefer to believe that Jon Chait got his facts wrong rather than actually imagining anybody would posit this theory of US history. 

The Epidemiology of Laughter

No, really. This post by famed sitcom writer/producer/director Ken Levine illustrate how infectious laughter follows some of the same rules as other infectious phenomena.

Comedy plays better in confined spaces. Laughs are louder when they don’t drift away.

Now you may say this is a superstition and I just want to be near that massage parlor, but (1) they don’t give group on’s, and (2) being in close quarters amplifies the laughter and laughter is infectious.

Whenever a sitcom episode goes into production the first order of business is a table reading. Several large tables are set up, the actors sit across from each other and read the script aloud as the writers and executives sit around them. Many shows I’ve worked on just hold their table readings right on their cavernous sound stages. On shows I’ve produced I insist we hold the table readings in conference rooms. Yes, it’s a little cramped, and chairs are pushed up against walls, but the difference in the reaction is startling. Laughs are so much bigger when you’re not at the bottom of the Grand Canyon.  Jokes are so much funnier when they don't echo.

Lest you think it’s just me, the table readings for CHEERS, FRASIER, WINGS, THE SIMPSONS, and BECKER were all held in conference rooms.

Levine also addresses the question of whether you should initially test a concept under overly friendly or overly hostile conditions.

Do we get an unfair reading as a result? Do the scripts appear funnier than they really are? Sometimes. There are producers who won’t change jokes later if it worked at the table reading. I’m not one of them. If a joke doesn’t work when it’s on its feet I cut it.  Table readings can always be deceiving.

But way more often, I’ve seen bad table readings done on the stage then gone back to the room and changed the shit out of the script. Later that day we'd have a runthrough of the original table draft and 70% of the stuff we planned to cut suddenly worked.

I’d rather err on the side of the table reading going well. Especially since you have the network and studio there as well. The less nervous they all are about the script, the better it is for all concerned.

The Totem Pole

Via Andrew Gelman, Gary Rubinstein has a very good post on what he observed while visiting a KIPP school in NYC (part of a larger series). I'll be getting back to the big issues Rubinstein raises soon, but first I wanted to take a moment on this side issue.

One thing that I see in schools a lot is the most experienced teachers getting to teach the high level class while the newbies have to teach the ninth grade remedial classes.  I suppose that this is some kind of reward for longevity, but it really is unfair.  If it really is about ‘the kids’ the new teachers should teach some of those easier classes.  I’m sorry to report that at KIPP they seemed to have the same sort of totem pole going on.  In the ninth grade wing is where I saw the most first year TFAers.

We hear a lot of complaints about how soft things supposedly are for long-time teachers, but the one place I've actually seen extensive evidence of this is an area that almost no one mentions (except, of course, for Rubinstein).

As a rule, first year teachers are best suited for advanced classes both in terms of strengths and weaknesses. New teachers tend to take a while to stop thinking in college terms. They are generally weak in classroom management and in those special communication skills needed for to handle lower level classes. Talking about concepts with peers and professors is very different from discussing those same concepts with bored ninth graders running behind grade level.

On the plus side, those concepts are fresher for just-out-of-college teachers and they inspire more passion. New teachers generally have more energy and are better able to keep up with classes of academically advanced kids. On top of that, college bound students often respond better to teachers they see as part of the college experience..

There is, as always, a context of bigger issues here, but for now, if you're looking for a reform to get behind, you could do worse than  tearing down that totem pole.

At this point, seeing even inaccurate coverage of OTA television seems like a triumph

This Marketplace story does leave a bit to be desired in terms of accuracy and completeness. In the audio version, the reporter, Sally Herships, like almost everyone covering this story, seems confused by digital versus amplified antennas and TV antennas versus satellite leading her to greatly overstate the price range of the equipment -- think under $10 if you don't need amplification (if not cheaper) -- and she failed to mention that the picture quality is actually better for free TV or that, in the region being discussed, you can get close to one hundred channels over the air or that with an adapter you can watch and digitally record those channels on you laptop.

All of that aside, this is still a story about doing without cable (or at least certain channels on cable), airing on one of the biggest and most respected financial news programs and it lists rabbit ears as the first option. I've been following this story closely for years now and I honestly can't recall this happening before. There have been a few voices in the wilderness on the subject -- Rajiv Sethi being perhaps the first and most notable -- but that was the blogosphere. A story like this in mainstream media is a very promising development, particularly given that the last time I heard the topic broached on Marketplace it was immediately dismissed by their guest expert (from the NYT, of course) who claimed that you can only get "a handful of channels" with an antenna in LA.

The story digital broadcasting has always come down to the question of whether the superior technology could establish a foothold before regulatory capture and the huge imbalance of hype drove it out of business. In that context, this is a good day for the medium.

Metablogging -- stag hunts, misalignment, principal agents and all that jazz

Shortly after Josh Marshall posted this analysis of recent events in the GOP, Joseph called me up to compare reactions. We've been having this conversation for so many years that much of it has devolved into a self-referential shorthand. As an illustration, at one point, after a fairly long-winded comment by me, Joseph simply said "Stag hunt." and tossed the ball back to me and we moved on to the next topic.

We agreed (with, I assume, fingers crossed on both ends of the line) that we'd write some posts on the subject, but I'm starting to think that it might be more useful to step back for a minute and talk about how we've been framing the question of what's going on in the Republican Party (politically, not socially or in terms of policy. Those are entirely different metaposts).

For years now, the two of us have been talking about the post-Tea Party GOP in terms of a multi-player stag hunt. Over the past few years the stakes (particularly the costs of failure) have increased. At the same time, participation rates required to take down the stag have also increased. As a result, progressively smaller groups have gained the power to kill the enterprise. (In a different conversation Joseph pointed out that, in a military context, shooting deserters is also a predictable result of this situation.) We could dig deeper into examples and implications (particularly with respect to the trade off between the power of an alliance vs. its stability) but for now I want to limit the discussion to framing.

(there might also be a place here to talk about symmetry breaking, but I'd need to give that some thought first.)

Another way of looking at the story we've found useful is to look at misalignment of interests, especially what looks to us two non-economists as a particularly nasty two-level principal agent problem where a small group of big donors determine the pool of viable candidates and a relatively small but coherent subgroup of the primary voters make the purchasing decisions for the entire party. You'll notice that, like the stag hunt frame, under this scenario small groups can acquire disproportionate power.

And of course there's the mandatory Influence reference, framing the story in terms of social psychology. If you check out the chapters on commitment and consistency, social proof, and scarcity you'll find all sorts of applicable discussions of the ways groups united by a common belief system deal with ideological challenges and the loss of dominance.

Nothing particularly fresh or profound here, but these idea have proven a pretty good framework recently. I'm not saying they should be the basis of the standard narrative -- I'm not sure there should be a standard narrative -- but they do come in handy. More importantly, I think you can make the case that too little of the public discourse is spent examining underlying assumptions and asking about the different ways to frame our questions.

The real challenge of Libertarian populism

According to Wikipedia, the night-watchman state (a foundation of Libertarianism) is defined as:

A night-watchman state, or a minimal state, is variously defined by sources. In the strictest sense, it is a form of government in political philosophy where the state's only legitimate function is the protection of individuals from assault, theft, breach of contract, and fraud, and the only legitimate governmental institutions are the military, police, and courts.

The bolding was added by me.

I think the real challenge of Libertarianism is not issues like Libertarian populism but rather the requirement that a Libertarian state protect against fraud.  I think that this is a more important the weaker that government gets, the more critical each function becomes.  Yet modern corporations are increasingly being allowed to get away with Fraud (and with changing of contracts).  These are huge problems because if you are going to enforce contracts against the politically weak, you also need to enforce them against the politically strong or you really do get feudalism. 

Trials like that of the S&P ratings schemes (where they argued ratings were puffery) are critical because it may be the case that a reasonable person might well think that misrepresentation was a form of fraud.  Saying that "lies=marketing" seems to be a rather obvious attempt to evade the antifraud portion of the states function. 

In the same sense, changing the rules under which a corporation operates can be an extremely dodgy move.  Changing the rules can be okay, but the optics are terrible when a major corporation shifts the rules when an initial attempt to do something fails.  It doesn't matter that the main victim is Carl Icahn. who is hardly at risk of poverty. 

To me, this is really the central issue of the modern Libertarian approach.  It isn't impossible that you could have a Libertarian populism, but I think the backbone of this approach would have to be that the same rules apply to everyone (even if they have a lot of money). 

Price discrimination at Ralph's

This is a new one on me.

I was in the grocery store last night when the thought hit me that I hadn't had a root beer float in a long time, too long in fact, but as I walked back and forth along the soda aisle I could not find any cheap root beer. There were various Coke and Pepsi products but to get a reasonable price you had to buy four (I don't drink much soda -- eight liters is a bit much). There were also expensive specialty brands, but no house brands. Annoyed, I grabbed a bottle of A&W and went on with my shopping only to see the store brands I was looking for one aisle over mixed in rather incongruously with the sports drinks.

Normally you'd expect to see the house brands near the corresponding name brand products, but I suspect that the previously mentioned "buy four" promotions prompted the shift, that given the choice between paying the inflated list price or dealing with the inconvenience of buying far more than they needed, too many customers were saying to hell with it and grabbing a Shasta or a Big K which had no restrictions and was even cheaper than the discounted name brands.

Given the competition from places like Wal-Mart and Food-4-Less and the 99 cents store, stocking cheap store brands can keep price sensitive customers coming back but making those brands hard to find can keep the less price sensitive customers spending more than they have to.

Worst Statistic of the Day

Via Lawyers, Guns, and Money comes the piece most detached from reality I've seen this year:

When Josh McFarland graduated from Stanford he owed $40,000 in student loans and couldn't fathom a way he'd ever pay it off and have a future for himself - not unusual for the typical young adult these days. Then he went to work for Google.

As a product manager, he got stock options and cashed them in over the five years he worked there. He married a fellow Google employee, so she had stock too. Then she moved on to Yelp, and he quit to launch TellApart, which provides technology solutions for e-commerce sites. Now 33, McFarland has a 3-year-old and a newborn and no longer has to think about his student loan: His company has $17.75 million in venture capital investment. While he doesn't consider himself retire-now rich, his piece of the company affords him what he calls "breathing room" and what other people might call wealth.

McFarland is on the starting end of Generation Y, the cohort born in the United States after 1980 that is typically portrayed as saddled with massive student debt, underemployed and underpaid. More than a third of the 80 million group of so-called millennials live with their parents, according to the Pew Research Group.

But McFarland is part of the sizeable minority that is doing quite well: 12 million Gen Y-ers make more than $100,000, according to the Ipsos MediaCT's Mendelsohn Affluent Survey. Many of them, in technology fields, live frugal work-based lifestyles and are not saddled with the six-digit student debt held by doctors and lawyers.

Paul Campos researched this statistic and discovered where it came from:

The source of the wrongheaded statistic appears to be this: the cited survey claims 59 million Americans live in households with incomes of $100,000 plus, and that 20% of “affluent consumers” are in the Gen Y cohort (this appears to mean that 20% of the adults who live in households with $100,000+ incomes are of this age). The story’s author then did some very bad math to generate the claim that “12 million Gen Y-ers make more than $100,000.” 

But that is totally different than these people making > $100,000 per annum.  That could also be people who are utterly unemployable living in their parent's basement. Or a trophy wife/husband of a hedge-fund billionaire. Or it could be 5 young adults sharing a house together, all making $22,000 a year as Starbucks' employees. 

At the very least it uses a highly no-representative sample and portrays the wealth of young adults in a completely misleading way.  This does not add to the conversation and it appears in the New York Times.  If only they had a prominent economist on staff who could consult over this type of data . . .

Weekend blogging -- Patents and product testing

Of course, these days I'm sure someone has patented the general idea of making a death ray. Perhaps that's an upside of having trolls.

"An astonishing act of statistical chutzpah" -- details on the Tony Bennett scandal

Jordan Ellenberg at Slate and Anne Hyslop at the New America Foundation have posts up explaining exactly how the Indiana Department of Education managed to change the grade of an influential donor's charter school from a C to an A. It appears to be your basic DATA COOKING 101. After seeing results you don't like, go back, try different weightings and look for excuses to drop bad scores then apply these changes selectively.

The 'selectively' part is particularly important and hasn't gotten the attention it deserves:

Two Indianapolis Public Schools might never have been taken over by the state if then-Superintendent of Public Instruction Tony Bennett had offered the district the same flexibility he granted a year later to the Christel House Academy charter school.

The issue was similar in both cases. Christel House had recently added ninth and 10th grades, and IPS’ Howe and Arlington had added middle school grades. The students who filled those seats posted poor enough scores to drag down the schools’ overall ratings.

In the case of Christel House, emails unearthed by The Associated Press show Bennett’s staff sprung into action in 2012 when it appeared scores from the recently added grades could sink the highly regarded school’s rating from an A to a C. Ultimately, the high school scores were excluded and the school’s grade remained an A.

But in 2011, after IPS’ then-Superintendent Eugene White demanded Bennett consider the test scores of high school students separately from those of middle school students so the high schools could avoid state takeover, Bennett was unmoved.

As for the specifics, Hyslop goes into more detail (and has a wonderfully apt movie quote to make her point), but Ellenberg probably does the better job summing it up:

Here’s where Bennett’s team found the loophole big enough to drive a charter school through. A normal person would do exactly what Chief Accountability Officer Jon Gubera did—give Christel the weighted average of its elementary/middle school score, according to the rules for elementary/middle schools, and its high school score, according to the rules for high schools.

But Bennett had a better idea. Christel was, technically speaking, not a high school, so the statutory formula for the high school grades didn’t apply. But it also didn’t have all four high school measures, so, he argued, the rules for combined schools didn’t apply either. There were just 13 schools in the state that had both middle school and high school grades but no seniors. For these schools, Bennett reasoned, the Indiana education poobahs should have a free hand to set the grades however they pleased. You can guess what happened next: Bennett ruled that the ninth- and 10th-graders in these schools didn’t count at all. So it was that the offending algebra grades vanished in a puff of bureaucratic smoke...

This was an act of astonishing statistical chutzpah. Suppose the syllabus for my math class said that the final grade would be determined by averaging the homework grade and the exam grade, and that the exam grade was itself the average of the grades on the three tests I gave. Now imagine a student gets a B on the homework, gets a D-minus on the first two tests, and misses the third. She then comes to me and says, “Professor, your syllabus says the exam component of the grade is the average of my grade on the three tests—but I only took two tests, so that line of the syllabus doesn’t apply to my special case, and the only fair thing is to drop the entire exam component and give me a B for the course.”

Ellenberg then made an observation that echoed some of my earlier points about how the mindset of the reform movement can enable these ethical lapses:

The saddest part is that I’m guessing Bennett sincerely felt he was doing the right thing. In his mind, he knew Christel was a great school, so if the scores said otherwise, the scores had to be wrong. In this respect, ironically, he ends up echoing his policy opponents, adopting the position that a mechanistic testing and scoring procedure can’t be allowed to override firsthand knowledge about teachers and schools.

The Op-ed no one wanted to print

When it comes to the education beat, this guy has one hell of a resume.

John Merrow began his career as an education reporter with National Public Radio in 1974, with the weekly series, “Options in Education.” In 1984, Merrow branched out into public television. He served as host of The Merrow Report, an award-winning documentary series, and currently is the Education Correspondent for PBS NewsHour. Merrow’s work has taken him from community colleges to kindergarten classrooms, from the front lines of teacher protests to policy debates on Capitol Hill. His varied reporting has continually been on the forefront of education journalism.

Of course, all of these qualifications don't necessarily make a person right, but they generally do make a person publishable. That's what makes this story curious, because Merrow can't seem to get anyone to publish the following op-ed. This is the second time in a row that he's found newspapers and magazines reluctant to publish reporting on Micelle Rhee.

As mentioned before, Rhee has gone from feted to indefensible so quickly that it's difficult for most journalists and pundits to cover her current activities without looking comically gullible for swallowing her previous rhetoric. (And journalists don't like looking gullible.)

CAVEAT EMPTOR: MICHELLE RHEE’S EDUCATION REFORM CAMPAIGN

"Today, too many of America’s children are not getting the quality education they need and deserve. StudentsFirst is helping to change that with common sense reforms that help make sure all students have great schools and great teachers." (StudentsFirst press release, emphasis added)

Michelle Rhee created StudentsFirst after leaving her post as Chancellor of Washington, DC’s Public Schools in the fall of 2010. She announced her intentions on “Oprah” that December: to fix America’s schools by enrolling one million members and raising one billion dollars.[2]

Easily America’s most visible education activist, she has been crisscrossing the country lobbying for change and donating money to candidates whose policies she supports. StudentsFirst claims to have helped pass 110 ‘student-centered policies’ in 18 states.

Because Ms. Rhee is trying to persuade the rest of the country to do as she did in Washington, it’s worth asking what her ‘common sense reforms’ accomplished when she had free rein to do as she wished.

She was definitely in charge. Her boss, a popular new mayor, told his Cabinet that trying to block his Chancellor was a firing offense.  The business community, a public fed up with school failure, and the editorial pages of The Washington Post were enthusiastic supporters. Moreover, she had virtually no opposition: the local school board had been abolished when the Mayor took over, and the teachers union, reeling from its own financial scandals, had an untested rookie president. She knew how lucky she was.

"I’m living what I think education reformers and parents throughout this country have long hoped for, which is, somebody will just come in and do the things that they felt was in the best interest of children and everything else be damned. (Interview, fall 2007)"

She lived that dream for 40 months.  She opened schools on time, added social workers, beefed up art, music and physical education, and dramatically expanded preschool programs.  The latter may represent her greatest success, because children who began their schooling in the expanded preschool program tend to do well on the system’s standardized test in later years.

Ms. Rhee made her school principals sign written guarantees of test score increases. It was “Produce or Else” for teachers too. In her new system, up to 50% of a teacher’s rating was based on test scores, allowing her to fire teachers who didn’t measure up, regardless of tenure.  To date, nearly 600 teachers have been fired, most because of poor performance ratings. She also cut freely elsewhere–closing more than two-dozen schools and firing 15% of her central office staff and 90 principals.

When Ms. Rhee departed in October 2010, her deputy, Kaya Henderson, took over. She has stayed the course for the most part, although test scores now make up–at most–35% of a teacher’s rating score.

Some of the bloom came off the rose in March 2011 when USA Today reported on a rash of ‘wrong-to-right’ erasures on standardized tests and the Chancellor’s reluctance to investigate.  With subsequent tightened test security, Rhee’s dramatic test scores gains have all but disappeared. Consider Aiton Elementary: The year before Ms. Rhee arrived, 18% of Aiton students scored proficient in math and 31% in reading. Scores soared to nearly 60% on her watch, but by 2012 both reading and math scores had plunged more than 40 percentile points.

But it’s not just the test scores that have gone down. Six years after Michelle Rhee rode into town, the public schools seem to be worse off by almost every conceivable measure.

For teachers, DCPS has become a revolving door. Half of all newly hired teachers (both rookies and experienced teachers) leave within two years; by contrast, the national average is understood to be between three and five years. Veterans haven’t stuck around either. After just two years of Rhee’s reforms, 33% of all teachers on the payroll departed; after 4 years, 52% left.

It has been a revolving door for principals as well.  Ms. Rhee appointed 91 principals in her three years as chancellor, 39 of whom no longer held those jobs in August 2010. Some chose to leave; others, on one-year contracts, were fired for not producing quickly enough.  Several schools are reported to have had three principals in three years.

Child psychiatrists have long known that, to succeed, children need stability.  Because many of the District’s children face multiple stresses at home and in their neighborhoods, schools are often that rock. However, in Ms. Rhee’s tumultuous reign, thousands of students attended schools where teachers and principals were essentially interchangeable parts, a situation that must have contributed to the instability rather than alleviating it.

Although Ms. Rhee removed about 100 central office personnel in her first year, the central office today is considerably larger, with more administrators per teachers than any of the districts surrounding DC.  In fact, the surrounding districts reduced their central office staff, while DC’s grew.  The greatest growth in DCPS over the years has been in the number of central office employees making $100,000 or more per year, from 35 when she arrived to 99 at last count.

Per pupil expenditures have gone up sharply, from $13,830 per student to $17,574, an increase of 27%, compared to 10% inflation in the Washington-Baltimore region. So have teacher salaries; DC teachers now earn on average more than their counterparts in nearby districts in Virginia and Maryland.

Enrollment declined on Ms. Rhee’s watch and has continued under Ms. Henderson, as families continue to enroll their children in charter schools or move to the suburbs.  The year before she arrived, DCPS had 52,191 students. In school year 2012-13 it enrolled about 45,000, a loss of roughly 13%.

Even students who have remained seem to be voting with their feet, because truancy in DC is a “crisis” situation, and Washington’s high school graduation rate is the lowest in the nation.  The truancy epidemic may be the most telling data point of all, because if young people in this economy are not going to school, something is very wrong. They are not skipping school to work–because there are no jobs for unskilled youth.

Ms. Rhee and her admirers point to increases on the National Assessment of Educational Progress, an exam given every two years to a sample of students under the tightest possible security.  And while NAEP scores did go up, they rose in roughly the same amount as they had under her two immediate predecessors, and Washington remains at or near the bottom on that national measure.

The most disturbing effect of Ms. Rhee’s reform effort is the widening gap in academic performance between low-income and upper-income students, a meaningful statistic in Washington, where race and income are highly correlated.  On the most recent NAEP test (2011) only about 10% of low-income students in grades 4 and 8 scored ‘proficient’ in reading and math. Since 2007, the performance gap has increased by 29 percentile points in 8th grade reading, by 44 in 4th grade reading, by 45 in 8th grade math, and by 72 in 4th grade math. Although these numbers are also influenced by changes in high- and low-income populations, the gaps are so extreme that is seems clear that low-income students, most of them African-American, generally did not fare well during Ms. Rhee’s time in Washington.

English Language Learners in Washington’s schools are also struggling. Title III of ESEA requires progress on three distinct measures: progress, attainment and what ‘No Child Left Behind’ calls ‘adequate yearly progress.’  DC failed on two out of three last year.

DC doesn’t fare well in national comparisons either.  Between 2005 and 2011, black 8th graders in large urban districts gained five points in reading, while their DCPS counterparts lost two points, according to a study by the DC Institute of Public Policy released this spring. Between 2005 and 2011 in large, urban districts, Hispanic eighth-graders gained six points in reading (from 243 to 249), black eighth-graders gained five points (from 240 to 245), and white eighth-graders gained three points (from 270 to 273). In District of Columbia Public Schools, however, Hispanic eighth-graders’ scores fell 15 points (from 247 to 232), black eighth-graders’ scores fell two points (from 233 to 231), and white eighth-graders’ scores fell 13 points (from 303 to 290).

The states that have adopted her approach, and others now being lobbied, might want to make their own data-driven decisions.

A final (?) news wrap-up on the Bennett story

As previously mentioned:

INDIANAPOLIS (AP) — Former Indiana and current Florida schools chief Tony Bennett built his national star by promising to hold “failing” schools accountable. But when it appeared an Indianapolis charter school run by a prominent Republican donor might receive a poor grade, Bennett’s education team frantically overhauled his signature “A-F” school grading system to improve the school’s marks.

You can get Joseph's reaction here and mine here and here.

Despite considerable support from the reform movement, you can add another 'former' to that paragraph.

Bennett said he resigned “because I don’t believe it would be fair to be distracted” by what he characterized as “malicious and unfounded” reports. 

Just yesterday, Gov. Scott told Channel 5 in West Palm Beach that Bennett is “doing a great job.

In what was already a bad news day for Bennett:

In June of 2011, Tony Bennett, then Indiana’s superintendent of public instruction, picked a for-profit education company in Florida to run a group of Indianapolis public schools.

The company, Charter Schools USA, set up operations in Indianapolis soon after the announcement and officially began running Manual High School, T.C. Howe High School and Emma Donnan middle school in the late summer of 2012. Millions of Indiana tax dollars have since flowed to the company, which has received many good reviews for its work in Indianapolis.

But a recent hiring decision by Charter Schools USA is sure to raise eyebrows and questions about conflicts of interest, particularly now that Bennett is embroiled in a massive controversy centering on special treatment given to certain Indiana schools during his tenure.

The decision: Charter Schools USA earlier this year hired Tony Bennett’s wife, Tina, as a regional director based in Florida, where Tony Bennett was hired late last year as commissioner of education. And, so, the bottom line is this: Tina Bennett is now earning a paycheck from the company her husband hand-picked to take over schools in Indiana, a decision that was very good for the company’s financial fortunes.

It’s important to note that Tina Bennett is a longtime educator, a former school administrator and counselor. She is also an advocate of the type of school choice efforts that Charter Schools USA is built on. In Indiana, she faced criticism and sometimes cruel treatment for taking a job with education groups tied to her husband’s former office. But it’s understandable that she would seek work in the education field.

To provide some context (and a bit of schadenfreude) for Bennett's fall, here's a reminder of where Bennett ranked in the reform firmament.

Value added testing without a gold standard outcome

From a Megan Pledger comment on StatChat comes this paper (pdf) on value added testing models for evaluating teachers.  The following concerns were brought up:

In the real world of schools, data is frequently missing or corrupt. What if students are missing past test data? What if past data was recorded incorrectly (not rare in schools)? What if students transferred into the school from outside the system?

The modern classroom is more variable than people imagine. What if students are team-taught? How do you apportion credit or blame among various teachers? Do teachers in one class (say mathematics) affect the learning in another (say science)?

Every mathematical model in sociology has to make rules, and they sometimes seem arbitrary. For example, what if students move into a class during the year? (Rule: Include them if they are in class for 150 or more days.) What if we only have a couple years of test data, or possibly more than five years? (Rule: The range three to five years is fixed for all models.) What’s the rationale for these kinds of rules?

Class sizes differ in modern schools, and the nature of the model means there will be more variability for small classes. (Think of a class of one student.) Adjusting for this will necessarily drive teacher effects for small classes toward the mean. How does one adjust sensibly?

While the basic idea underlying value-added models is the same, there are in fact many models. Do different models applied to the same data sets produce the same results? Are value-added models “robust”?

Since models are applied to longitudinal data sequentially, it is essential to ask whether the results are consistent year to year. Are the computed teacher effects comparable over successive years for individual teachers? Are value-added models “consistent”?

A lot of these concerns have been independently voiced by Mark P.  However, what is especially concerning is the idea that we could iterate through these assumptions to find a school ranking that satisfies some prior.  This can be good under some circumstances -- Thomas Lumley gives an example of a model that clearly mixed up rankings of some kind of sports team (this isn't my area of expertise so I apologize that I don't recognize the teams or sport involved).  But it does show how difficult these models are, even with the best faith involved.  Still, in the case of Dr. Lumley's example there is a universal outcome that has broad agreement (does this team win games) that is being predicted.  In education we lack this very clean outcome which is where it gets tricky -- in a sense we are modeling a latent variable (student outcomes). 

All of this suggests that we should be cautious about these models and perhaps this would be an appropriate time to put some serious effort into student outcomes ascertainment so that it will be easier to calibrate these statistical models (making the outcome the test score seems clever but merely hides the problem rather than solving it unless we are confident that the score is a very good measure of outcomes). 

Stories I should probably be writing about

Thoreau has a glowing review of this book on pedagogical fads. It looks interesting though given the cost and the quantity I assume they are writing them out by hand.




The Bennett scandal continues to reverberate both in Indiana and Florida.

 Democracy Prep and its founder and superintendent Seth Andrew have been media darlings but I'm seeing some things that trouble me, both about the school and the founder, particularly when you read between the lines.


Kevin Drum addresses our old friend, peer effects.


Take a look at this school designed largely to give master's degrees to people who came through Teach for America and similar programs. As you might expect, I see some connections between this and the previously mentioned issue of grooming TFAers for leadership positions.


Which segues nicely into lapsed TFAer Gary Rubinstein's blog and its informative series of posts on a recent visit to a KIPP school in New York. He also often addresses the previously mentioned TFA cultural issues that concern me and many of his other comments (like this one) track with my experience almost perfectly.


I'm a big fan of Kaiser Fung, but the Moneyball analogy strikes me as a bad framework for an analytic approach to education reform, bad enough to cause real damage. (I see from the queue that Joseph is planning to address some of these issues tomorrow)


And, to take a break from the education beat, there's this post from Naked Capitalism on  a shadow credit reporting system.

p.s. Should have included a link to this Washington Post interview with "the world’s most famous teacher." It nicely lays out the tension between some of our best veteran teachers (in this case one of the major models for KIPP) and the education reform movement.

General versus particular cases

Andrew Gelman did a very interesting article in Slate on how being overly reliant on statistical significance can lead to spurious findings.  The authors of the study that he was critiquing replied to his piece.  Andrew's thoughts on the response are here. 

The led to two thoughts.  One, I am completely unimpressed by claims that a paper being in a peer-reviewed journal -- that is a screen but even good test have false positives.  All this convinces me of is that the authors were thoughtful in the development of the article, not that they are immune to problems.  But this is true of all papers, including mine. 

Two, I think that this is a very tough area to take a single example from.  The reason is that any one paper could well have followed the highest possible level of rigor, as Jessica Tracy and Alec Beall claim they have done.  That doesn't necessarily mean that all studies in the class have followed these practices or that there were not filters that aided or impeded publication that might enhance the risk of a false positive.

For example, I have just finished publishing a paper where I had an unexpected finding that I wanted to replicate (that there was an association was a priori, the direction was reversed from the a priori hypothesis).  I found such a study, added additional authors, added additional analysis, rewrote the paper to be a careful combination of two different cohorts, and redid the discussion.  Guess what, the finding did not replicate.  So then I had  the special gift of publishing a null paper with a lot of authors and some potentially confusing associations.  If I had just given up at that point, the question might have been hanging around until somebody else found the same thing (I often used widely available data in my research) and published it. 

So I would be cautious about multiplying the p-values together for a probability of a false positive.  Jessica Tracy and Alec Beall:

The chance of obtaining the same significant effect across two independent consecutive studies is .0025 (Murayama, K., Pekrun, R., & Fiedler, K. (in press). Research practices that can prevent an inflation of false-positive rates. Personality and Social Psychology Review.)

I suspect that this would only hold if the testable hypothesis was clearly stated before either study was done.  It also presumes independence (it is not always obvious that this will hold as design elements of studies may influence each other) and that there isn't a confounding factor involved (that is causing both the exposure and the outcome).

Furthermore, I think as epidemiologists we need to make a decision about whether these studies are making strong causal claims or advancing a prospective association that may led to a better understanding of a disease state.  We often write articles speaking in the later mode but then lapse into the former when being quoted. 

So I guess I am writing a lot to say a couple of things in conclusion. 

One, it is very hard to pick a specific example of a general problem when it is possible that any one example might happen to meet the standards required for the depth of inference being made.  This is very hard to ascertain within the standards of the literature. 

Two, the decision of what to study and what to publish are also pretty important steps in the process.  These things can have a powerful influence on the direction of science in a very hard to detect manner. 

So I want to thank Andrew Gelman for starting this conversation and the authors of the paper in question for acting as an example in this tough dialogue.