Track records matter. Like it or not, unless you're actually working with the numbers, you have to rely to some degree on the credibility of the analysts you're reading. Three of the best ways to build credibility are:
1. Be right a lot.
2. When you're wrong, admit it and seriously examine where you went off track, then...
3. Correct those mistakes.
I've been hard on Nate Silver in the past, but after a bad start in 2016, he did a lot to earn our trust. By the time we got to the general election, I'd pretty much give him straight A's. By comparison, there were plenty of analysts who got straight F's, and a lot of them are playing a prominent role in the discussion this time around..
I was going back and forth on whether or not to revisit this
critique of the New York Times' Nate Cohn in the wake of the Walker implosion. If you remember, little over a month ago, Cohn made the
argument that:
In the end, Mr. Trump almost certainly won’t win the Republican
nomination; the rest of the party will consolidate around anyone else.
He can influence the outcome only if his support costs another candidate
more than others. But for now, he seems to be harming all candidates
fairly equally.
This was never a convincing claim (at the time I called it "strained and
convoluted"), and it has gotten even less defensible in the light of
recent events, but that's not necessarily enough reason to dredge things
up. There's a difference between keeping score and piling on (and the
last post on Cohn was a bit on the harsh side).
I was leaning toward dropping the thread until I read Cohn's piece on
the collapse of the Walker campaign and again saw things that bothered
me. It was better than the Trump pieces, but it was improvement without
progress.
[Yeah, I know, it sounds garbled but let me unpack the oxymoron. If the
most recent outcome is better than the previous one, that's an
improvement; if conditions change so that we can expect future outcomes
to be better than previous outcomes, that's progress. (My blog, my
definitions.)]
Over the past three months, Nate Cohn and many of his colleagues have
not only failed to anticipate major developments; they've also made a
string of predictions that haven't come to fruition. This wouldn't be
worrisome if it were leading to a critical examination of the analogies
and assumptions that led to these errors, or at least a little less
self-assurance.
What we don't want to see is yet is yet another simple narrative presented as the explanation.
Which brings us to..
Mr. Walker faltered so quickly because he simply was not skilled enough
to navigate the competing pressures of appealing to the party’s
establishment at the same time as arousing its base. It was much like
the story of Rick Perry.
Though the entry of Donald Trump into the race made things harder for
all the Republican candidates, Mr. Trump can’t be blamed entirely for
Walker’s troubles. Mr. Walker was tied with Mr. Bush for second place in
national polls heading into the first debate, long after Mr. Trump took
a lead in those polls. By the time he dropped out, Mr. Walker had the
support of less than one-half of 1 percent of Republican primary voters,
according to the most recent CNN survey.
The Walker campaign — or perhaps the candidate personally — felt
pressure from the rise of Mr. Trump on his right, especially once Mr.
Walker started slipping a bit in the polls. This sort of pressure isn’t
unusual and was inevitable — he would have felt it at some point, if not
from Mr. Trump, then from Ben Carson or Ted Cruz.
Mr. Walker, to put it gently, did not handle this pressure well. His
instinct was to move to the right as fast as possible at any point of
vulnerability. He staked out a conservative position on birthright
citizenship and a fringe position on considering a wall at the Canadian
border. These moves alienated party elites and weren’t credible to
conservative voters. He quickly reversed positions; in the end, he
reassured no one.
First off there's the argument itself. It seems to be a reasonable, if
not all that convincing, little story (not all that different than the
one Nate Silver
tells,
but more an that in a minute) until you consider magnitude of the event
being discussed. Walker did have notable missteps (to use Silver's
term) but they were relatively minor. If we were talking about a
campaign losing momentum or dropping a few points they might make sense
as a possible cause, but we're talking about a complete implosion with a
well-funded candidate going from near the top of the field to less than
one percent support in shockingly little time.
The important part here is the speed of the collapse. Cohn himself was discussing Walker's
weaknesses as a campaigner back in July, but he also said Walker had "plenty of time to assuage these concerns."
We now know Walker didn't. A month later, he would already be in
free fall.
Scott Walker has sought to reassure jittery donors and other supporters
this week that he can turn around a swift decline in the polls in Iowa
and elsewhere by going on the attack and emphasizing his conservatism on
key issues.
And, if you check the dates, you'll notice that the donors got jittery
before Walker started talking about that
Canadian wall.
Furthermore, probably Walker's most high-profile move to the right was
the anti-labor position. I suppose this could have alienated the party
elites (though we are talking about Koch brothers here) but if Scott
Walker can't make an attack on unions credible to conservative voters, I
honestly can't imagine anyone who could.
But the argument itself isn't what really bothers me. What's troubling
here is the way that Cohn deals with the failure of his predictions.
Even those with a good track record on the GOP race, such as Josh
Marshall, admitted to being surprised at just how rapidly the collapse
transpired. As a good rule of thumb, if you did not expect something,
you should be careful about offering explanations about how it happened.
As far as I can tell, Marshall and company are doing their best to
follow this rule.
Cohn, by comparison, has perhaps the worst track record of any of the
analysts I've been following when it comes to the GOP primary. After
calling a premature end to the Trump surge a number of times and making
the just-another-Herman-Cain analogy on no less than five separate
occasions, he then made the previously mentioned claim about Trump not
having any effect on the race.
Despite this, Cohn appears to be one of the most confident commentators
when putting forward his theories about the causes of the implosion. The
comparison to Nate Silver here is instructive. Silver actually opens
his piece by acknowledging just how wrong he and his team were on Scott
Walker. In addition to caution, Silver also offers a great deal more in
terms of complexity and counter arguments. Cohn actually uses the word
"simply" to describe his version of events.
But what's important here is not just the unwarranted confidence. We all
have our moments. What merits attention is what Cohn's confidence in
this situation tells us about his process. Anyone who works with data
long enough will have occasion to see their models break down and their
predictions go so far awry that they are no longer even directionally
accurate.
When journalists looking in from the outside describe these disasters,
they almost invariably use the phrase "go back and check your numbers,"
but in complex situations that is relatively seldom the source of the
problem. More likely and far more difficult to catch are problems with
robustness and modeling assumptions.
I realize I may be making too much of this, but there's a bigger issue
here that has been bothering me for a long time. When Nate Cohn says
this is what happened to Scott Walker, he is displaying a
this-is-how-the-world-works tone and mindset that is very common in
places like the Upshot, even when it is not at all appropriate. If you
read carefully the work of journalists inspired by Nate Silver (though
not so much Silver himself), you pick up the implicit belief that the
standard methods and assumptions being employed are as true and as
reliable as the laws of mathematics , that they have always worked and
will always work.
This is a dangerous way to approach the social sciences, particularly
when you start running into range of data issues (and between Trump,
Carson, and the rise of the tea party, I think we are definitely in new
territory now).
As mentioned before, I strongly suspect that the theory that Walker
collapsed because his move to the right offended the elites and yet was
not credible to the base is wrong, but I would be much more comfortable
with it and would certainly have not written this post if it had been
clearly framed as a theory.
