Jon Stewart opens up a big ol' can of historical context on Bill O'Reilly

An absolutely beautiful piece of satiric commentary.

The Daily Show
Get More: Daily Show Full Episodes,Political Humor & Satire Blog,The Daily Show on Facebook

Follow-up to yesterday's post

The discovery that Hostess tripled CEO pay (and boosted the pay of other senior executives) while blaming overpaid unions for the end of the company is a great example for yesterday's point.  It would be fine to blame everyone involved.  But some of the issues, like under-funded pension obligations, strike me as more of a management issue than a worker issue.  But, regardless, it is striking that the narrative is all about the striking bakers and not the huge cash grab at the executive level.

EDIT: See Thoreau as well.

Free Market Wages

I often get frustrated by defenses of extremely high pay as needed to be able to attract the incredibly small pool of high skill workers.  This comment is pitch-perfect:

The claim that high pay is necessary to attract and motivate good workers is not merely an economic postulate. If it were, it could apply across the wage distribution. Instead, it functions as a defence of inequality - used to justify higher pay for the rich, rather than for any job. For grunt workers, wages are a cost to be minimized regardless of consequence.

This is a really good point; if wages for managers were treated like an expense for the company (instead of as an investment in talent) then would we have seen manger salaries rise so far in recent years?

Why do we not have the same sort of bidding war for mid-level talent?  I have the same question in a different way: why are managers getting high wages is seen as a sign of paying for talent but union workers are seen as using sleazy tactics to work the system?  Do we see corporate boards setting salaries for CEOs as being fundamentally different than governments setting wages for government workers?  In both cases a group of experts are setting wages based on expert opinion and not on free market principles.

Today's tuition discussion

From Counterparties:

Megan McArdle argues that surging costs are as much a consequence as they are a cause of unprecedented levels of student debt, spurred on by government subsidies. But Mike Konczal flags a Department of Education study that shows that the government earns $1.14 back for every dollar it loans to students and asks, “What’s a good word for the opposite of a subsidy? Whatever it is, student loans are that”.
 

 This really seems to be a major problem with the modern discourse on education.  We are all convinced that there has to be some sort of amazing (clever, counter-intuitive) theory about why tuition is going up.  Bad policy, on the other hand, seems to be completely ignored.  But if the government is making a net profit off of student loans, that seems to be rather concerning.  Not because I object to profit, but because the loans are so high.

There was a period where this blog used to use the word "Canada" a lot.  The reason is that Canada has a lot of interesting existence proofs for US policy.  They have universal health care, acceptable health outcomes, and a much lower percentage of GDP spent on health care.  Their system has flaws and trade-offs, but it shows that a diverse, multi-ethnic and geographically large country could control health care costs. 

In the same sense they seem to have much lower tuition and very high quality Universities.  They may not be the absolute best, but they are surprisingly competetive for a small country.  I am not sure we'd want to adopt all of the problems of Canadian education, but it is worth noting that they seem to be able to deliver high quality education at a much lower price point.  And this should have alerted the clever reader to the problem of Ms McArdle's argument -- why doesn't it happen in other countries, which also have a high return to education? 

Social Security: really, what is going on here?

I keep asking the same question as the commenter on The Incidental Economist asks:

In fact, Social Security is NOT in crisis. If we hiked payroll taxes by two percentage points we would probably take care of the payroll tax deficit. Yes, that’s a regressive tax and we should be cautious about doing it. But our long-run budget problems are mostly in health care. Why the unearthly fascination with cutting Social Security?
 

I mean, seriously, why is it such conventional wisdom that it needs to be reformed?  It is true that a private alternative would be a great revenue center for finance types.  But a lot of things would be great revenue sources that we don't privatize for good reason (Police, Military), either because it would be inefficient or because only the government can guarentee long term contracts like those required for retirement in 45 years. 

Social security is a popular benefit and something like 80-90% of the population will eventually have it as an important piece of their income.  Just what is going on here? 

The biggest political story you probably haven't heard

(At least if you're not on Pacific Time) were California propositions 30, 32, and 36. I don't have time to give this the write-up it deserves, but the LA Times voting guide is a good starting point. The short version is this: under long odds  (including the worst piece of vanity politics since Nader hung it up and a special cameo appearance from the Koch brothers) and by significant margins, the largest state in the union has just made a sharp turn to the left on taxes, education, unions and mass incarceration.

What's more, I suspect that the interest in these issues may have pumped up voter turn out and, as a result, helped Obama's popular vote totals. Add to that a potential supermajority in both houses.

None of this may be as interesting as genetically modified food and condoms for porn stars but it's a damned sight more important.

You really should be reading Andrew Gelman's blog

I considered this point (made by Andrew Gelman) to be really interesting:

All statisticians use prior information in their statistical analysis. Non-Bayesians express their prior information not through a probability distribution on parameters but rather through their choice of methods. I think this non-Bayesian attitude is too restrictive, but in this case a small amount of reflection would reveal the inappropriateness of this procedure for this example.

I had never phrased things like this but it does seem to be a sensible description of what I (essentially a Frequentist) actually do in real life.  This may be a very good teaching point and it nicely illustrates that all analysis involves priors -- just that some are more explicit than others.

More for blogger's to-do list

Really swamped this week, which invariably means that all sorts of interesting topics have been popping up. Here's a partial list. I'll try to get to as many as I can, but if any fellow bloggers want to beat me to the punch please go for it.

1. The ethics (and advisability) of psychoanalyzing your opponents
(when is it acceptable, wise and not overly sleazy to try to explain your opponents' actions using psychology and sociology and how do you avoid using those explanations to dismiss valid arguments on the other side)

Link to Pauline Kael's "Raising Kane" (no, really)


2. Implications of a submerging Republican majority
A. Scarcity and cognitive dissonance -- link to Cialdini
B. The looting phase (or Karl Rove is not your friend) -- link to Red State, Rick Perlstein, David Frum
C. Waiting for the Big Dog  -- link to Ed Kilgore (at some length)
D. Nate Silver and the Sandy Hypothesis  -- link to Talking Points and 538

3. The killer app for the driverless car -- it's not what you think

4. Daniel Engber doesn't seem to get the subtleties of statistics debates -- link to this and this

5. Nudge cars (I'll explain later)

6. Homemade Pi -- a classroom exercise for geometry, spreadsheets and Monte Carlo techniques

7. More Groupon -- link to Felix Salmon

8. Trivial thoughts -- athletic actors

9. The annual Toys-for-Tots post

Podhoretz mans up

Lots of pundits did shoddy work covering the past election. Very few have actually owned up to it.

From NPR:

FOLKENFLIK: But if Florida stays blue, Silver will have picked every state correctly, along with the president's margin of victory. Conservative columnist John Podhoretz of the New York Post and Commentary magazine had earlier argued pollsters were getting it wrong by ignoring the high turnout by Republicans in the 2010 elections that swept the GOP into control of the U.S. House. The 2012 race would be the same, Podhoretz argued, quite mistakenly, as it turned out.

JOHN PODHORETZ: That view was strengthened and amplified by what I wanted to happen, which I freely confess. People don't ordinarily cast a skeptical eye on data and information that supports their opinions. They're happy to take it.

p.s.  Add Unskewed Polls' Dean Chambers to this list.:

“Most of the polls I ‘unskewed’ were based on samples that generally included about five or six or seven percent more Democrats than Republicans, and I doubted and questioned the results of those polls, and then ‘unskewed’ them based on my belief that a nearly equal percentage of Democrats and Republicans would turn out in the actual election this year,” Chambers wrote on The Examiner website. “I was wrong on that assumption and those who predicted a turnout model of five or six percent in favor of Democrats were right. Likewise, the polling numbers they produced going on that assumption turned out to be right and my ‘unskewed’ numbers were off the mark.”

That certainly was a close one

Neck and neck

A dead heat

Too close to call

Virtually tied

Plan on a long night




One of the things I hope people will take away from the election is a reminder of how bad journalistic group think has become. Narratives are converged upon instantly. treated as axioms no matter how many non-journalists reach a different conclusion, and defended with an appalling level of nastiness (just ask Nate Silver).

The Colbert Report
Get More: Colbert Report Full Episodes,Political Humor & Satire Blog,Video Archive

The Colbert Report
Get More: Colbert Report Full Episodes,Political Humor & Satire Blog,Video Archive

What's even more depressing is the realization that these pundits will pay no penalty for being willfully, arrogantly, childishly wrong.

The Daily Show
Get More: Daily Show Full Episodes,Political Humor & Satire Blog,The Daily Show on Facebook

When it comes to the horse race, what do the people say?

It's possible to find exceptions but the general consensus among political scientists, poll aggregators and betting markets has generally been that Obama held a small but respectable lead. Journalists, on the other hand, seem to be contractually required to describe the race with some synonym for tied (only breaking the pattern occasionally to talk about Romney's momentum).

With that in mind, this quote from Andrew Kohut of Pew is particularly interesting...

RAZ: The popular vote. Yeah. Interestingly, when you asked voters who they think will win, 52 percent say Obama, just 32 percent say Romney.

KOHUT: Yes. That has been pretty consistent through this election. Obama is seen by average voters as the likely winner of the next election. That's another measure that's been a pretty good indicator of who will win the election. That is the candidate that the electorate thinks will win generally does win.

Life on 49-49

[Following up on this post, here are some more (barely) pre-election thoughts on how polls gang aft agley. I believe Jonathan Chait made some similar points. Some of Nate Silver's critics also wandered into some neighboring territory (with the important distinction that Chait understood the underlying concepts)]

Assume that there's an alternate world called Earth 49-49. This world is identical to ours in all but one respect: for almost all of the presidential campaign, 49% of the voters support Obama and 49% support Romney. There has been virtually no shift in who plans to vote for whom.

Despite this, all of the people on 49-49 believe that they're on our world, where large segments of the voters are shifting their support from Romney to Obama then from Obama to Romney. They weren't misled to this belief through fraud -- all of the polls were administered fairly and answered honestly -- nor was it a case of stupidity or bad analysis -- the political scientists on 49-49 are highly intelligent and conscientious -- rather it had to do with the nature of polling.

Pollsters had long tracked campaigns by calling random samples of potential voters. As campaign became more drawn out and journalistic focus shifted to the horse race aspects of election, these phone polls proliferated. At the same time, though, the response rates dropped sharply, going from more than one in three to less than one in ten.

A big drop in response rates always raises questions about selection bias since the change may not affect all segments of the population proportionally (more on that -- and this report -- later). It also increases the potential magnitude of these effects.

Consider these three scenarios. What would happen if you could do the following (in the first two cases, assume no polling bias):

A. Convince one percent of undecideds to support you. Your support goes to 50 while your opponent stays at 49 -- one percent poll advantage

B. Convince one percent of opponent's supporters to support you. Your support goes to 50 while your opponent drops to 48 -- two percent poll advantage

C. Convince an additional one percent of your supporters to answer the phone when a pollster calls. You go to over 51% while your opponent drops to under 47%-- around a five percent poll advantage.

Of course, no one was secretly plotting to game the polls, but poll responses are basically just people agreeing to talk to you about politics, and lots of things can affect people's willingness to talk about their candidate, including things that would almost never affect their actual votes (at least not directly but more on that later).

In 49-49, the Romney campaign hit a stretch of embarrassing news coverage while Obama was having, in general, a very good run. With a couple of exceptions, the stories were trivial, certainly not the sort of thing that would cause someone to jump the substantial ideological divide between the two candidates so, none of Romney's supporters shifted to Obama or to undecided. Many did, however, feel less and less like talking to pollsters. So Romney's numbers started to go down which only made his supporters more depressed and reluctant to talk about their choice.

This reluctance was already just starting to fade when the first debate came along. As Josh Marshall has explained eloquently and at great length since early in the primaries, the idea of Obama, faced with a strong attack and deprived of his teleprompter, collapsing in a debate was tremendously important and resonant to the GOP base. That belief was a major driver of the support for Gingrich, despite all his baggage; no one ever accused Newt of being reluctant to go for the throat.

It's not surprising that, after weeks of bad news and declining polls, the effect on the Republican base of getting what looked very much like the debate they'd hoped for was cathartic. Romney supporters who had been avoiding pollsters suddenly couldn't wait to take the calls. By the same token. Obama supporters who got their news from Ed Schultz and Chris Matthews really didn't want to talk right now.

The polls shifted in Romney's favor even though, had the election been held the week after the debate, the result would have been the same as it would have been had the election been held two weeks before -- 49% to 49%. All of the changes in the polls had come from core voters on both sides. The voters who might have been persuaded weren't that interested in the emotional aspect of the conventions and the debates and were already familiar with the substantive issues both events raised.

So response bias was amplified by these factors:

1. the effect was positively correlated with the intensity of support

2. it was accompanied by matching but opposite effects on the other side

3. there were feedback loops -- supporters of candidates moving up in the polls were happier and more likely to respond while supporters of candidates moving down had the opposite reaction.

You might wonder how the pollsters and political scientists of this world missed this. The answer that they didn't. They were concerned about selection effects and falling response rates, but the problems with the data were difficult to catch definitively thanks to some serious obscuring factors:

1. Researchers have to base their conclusions off of the historical record when the effect was not nearly so big.

2. Things are correlated in a way that's difficult to untangle. The things you would expect to make supporters less enthusiastic about talking about their candidate are often the same things you'd expect to lower support for that candidate

3. As mentioned before, there are compensatory effects. Since response rates for the two parties are inversely related, the aggregate is fairly stable.

4. The effect of embarrassment and elation tend to fade over time so that most are gone by the actual election.

5. There's a tendency to converge as the election approaches. Mainly because likely voter screens become more accurate.

6. Poll predictions can be partially self-fulfilling. If the polls indicate a sufficiently low chance of winning, supporters can become discouraged, allies can desert you and money can dry up. The result is, again, convergence.

For the record, I don't think we live on 49-49. I do, however, think that at least some of the variability we've seen in the polls can be traced back to selection effects similar to those described here and I have to believe it's likely to get worse.

Elasticity, Plasticity and Creep in polls

I've been thinking quite a bit about polls lately, about what can go wrong with them, and how they might react and even how some of Nate Silver's critics have almost stumbled onto a couple of valid points (albeit ones that Silver has acknowledged).

I'm working on a full post on the subject but in the meantime, here's some background complete with sound track. (hit play now)


One of the things I've been coming up against is what can happen when something moves a poll.

1. Elasticity

Let's say a candidate gives a rousing speech that gets base fired up. It makes supporters more likely to talk to pollsters but it doesn't really change anyone's mind. After a short period the excitement fades and the polls return to their previous position.

2. Plasticity

Another speech, but this this time with enough substance to win over some undecideds. The polls move to another position and stay there.

3. Creep

This starts with an event that would normally cause an elastic shift, but the deformation of the polls is drawn out, either because it's followed by other trivial events or because the timing of the polls makes it seem drawn out or because the media simply decides to dwell on it or (most likely) some combination. Under these circumstances we can easily get a feedback loop where bad polls create the impression of eroding support which causes supporters to become discouraged and money to dry up which causes support to actually erode.

I suspect Romney was experiencing creep in the last week of September.

I'll try to flesh this out more in the next 24. In the meantime, does anyone have any thoughts on the subject?

A very good perspective on the latest Star Wars news

Star Wars has been sold to Disney for 4.05 billion dollars.  George Lucas's high risk decision to push through the difficult task of making the first Star Wars movie appears to have paid off.  While some people are worried about the new owners of the property, I think this piece has a very good perspective on the whole transaction and is definitely worth reading.

When a stag/hare hunt is not a stag/hare hunt

This speculation from a TPM reader got me to thinking...

Regarding dumping money into non-competitive states, here in Dallas nearly every other commercial is an anti-Obama ad from the superPAC, “Restore our Future.” I’m trying to figure out why they’re dumping so much money into the Texas ad market when I’d assume Texas is about as solidly red as it gets. I have two theories: 

1. It’s to build up GOP enthusiasm for down-ticket races.

2. They need to spend some of their donations on something other than lining their own pockets to help keep people from realizing they’re nothing but grifters who have been lining their own pockets with all that political donation money.

This reminded me of the Stag Hunt game where players have to decide whether to stick with a group in hopes of a big pay-off that's dependent on cooperation or go off on their own for a smaller but more likely pay-off.

I don't want to get bunch of emails saying I don't know anything about game theory (which is true but hopefully not relevant here), so to be clear I'm not saying this is a stag hunt, just that this raises some similar tactical questions about diverting resources from a big but unlikely pay-off (the Electoral College) to smaller ones (the popular vote, better down-ticket results).