The cost of being wrong – – political polling edition

Note: in this context, when I talk about the accuracy of the polls, I mean the correlation between the election results and where the polling averages were at the time the polls were being discussed. If I write a post today and ask if the presidential polls are accurate, I mean is what we're seeing in May a good indicator of what the results will be in November.

This is what pretty much everybody means when they talk about the accuracy of the polls except for pollsters and political scientists who generally mean the correlation between the election results and what the polls will be in the final three or so weeks of the campaign. These are very different concepts often producing very different numbers and those of us who work with or talk about numbers for a living should really clean up her act on this one.

Let's start with the recent primary in Maryland.

 

There were some very strange things about the polling around this election and probably the election itself, but the one aspect are interested in here is how it affected the decision to do this...

Trone spent over $60 million on his failed Senate bid. Will he dig deep to help his rival beat Hogan? https://t.co/q35QrvE3X3

— The Baltimore Banner (@BaltimoreBanner) May 17, 2024

I have to admit I have not read the article (my queue is full and you'll notice there's no revenue coming in from this blog) so if I'm wrong about this, please let me know in the comments, but I doubt that Trone would have spent what is for most of us a substantial fortune had the polls consistently shown him behind by 10 to 15 points instead of ahead by that margin.

Of course, in the grand scheme of things, the impact wasn't that big a deal one way or the other. Billionaires often spend their money on even more wasteful things. At least this stimulated the state media economy. For a case of inaccurate polling causing truly disastrous results we would need to go back about eight years.

This article by Amy Chozick is essential reading whether you're interested in the history of the 2016 election or in the workings of the New York Times. In the case of the latter, this is the most honest and unflinching writing I've seen about the paper from anyone still in its employ since they lost Margaret Sullivan.

[Emphasis added]

It’s dizzying to realize that without even knowing it, you’ve ended up on the wrong side of history. Months after the election, every time I heard the words “Russia” and “collude,” this realization swirled in my head, enveloping everything.

...

 Editors and reporters huddled to discuss how to handle the emails. Everyone agreed that since the emails were already out there — and of importance to voters — it was The Times’s job to “confirm” and “contextualize” them. I didn’t argue that it appeared the emails were stolen by a hostile foreign government that had staged an attack on our electoral system. I didn’t push to hold off on publishing them until we could have a less harried discussion. I didn’t raise the possibility that we’d become puppets in Vladimir Putin’s master plan. I chose the byline.

...

A few weeks before Election Day, I was stuck in my cubicle poring over John Podesta’s emails. I wanted to be on the road. “I just feel like the election isn’t happening in my cubicle,” I said. “But it’s over,” an editor replied, reminding me that the Times’s Upshot election model gave Mrs. Clinton a 93 percent chance of winning. The ominous “they” who would keep the glass ceiling intact didn’t look that powerful then.

We can go back and forth as to whether the polls were wrong or the NYT model was wrong (I'm inclined to say more the second than the first), but either way, if the editors and reporters at the paper had believed that Hillary Clinton had a 60% chance of winning or worse yet a 50-50 chance, it is unlikely they would have made those same decisions. As the account spells out explicitly, they knew they were at best in that ethically gray area, helping a foreign power influence a national election in exchange for revelations of limited news value. I am a bit of a cynic, especially when it comes to the New York Times, but even I don't believe they would have been so reckless had they known what they were likely to unleash.

Journalism professor and press critic Jay Rosen says reporters should focus on the the stakes of the election, not the odds, but when you focus on the odds and get them wrong, that's the worst of both worlds.