Unfortunately, one side effect of all this attention has been to create the impression of implicit comparisons. When we talk about the weaker articles in 538 because we think the direction of the website is important, we can leave people thinking that weak articles are disproportionately found in 538. That is by no means a sound conclusion.
With the obvious exception of Roger Pielke Jr., my least favorite 538 hire is probably Walt Hickey (though concerned, I'm reserving judgement on Emily Oster for the moment). Hickey seems like a nice, well-intentioned fellow, but from what I've seen, he's an excellent example of one of those data journalists who understand the math but not the statistics, getting the procedures right but missing the point (this is somewhat analogous to Feynman's comments about textbook writers missing the nuances of math and science).
I decided to Google him. It turns out that Hickey (going under the slightly more businesslike 'Walter') was a prolific contributor to Business Insider (among other sites). Since he seemed to be doing a lot of entertainment reporting for 538, I looked for something similar on BI and came up with this:
Here's Where All The Miley Cyrus Haters Live
The metric used was the addresses (five-digit zip only) of the 158 complaint letters sent to the FCC after Miley Cyrus's performance at MTV's VMA award show. This is not a good data set but it is possible to do some mildly interesting demographic breakdowns. It's not as good as it would have been with nine-digit zips (those open up a lot of useful information), but you could, for example, look at things like city size.
But what you would never want to do with 158 addresses is a state-by-state breakdown.
This was followed by a list of "the top ten most irate states, based on letters sent per capita" with the sparsely populated South Dakota coming in at number four based on just one letter to the FCC.
(as a side note, when I went back to the article to write this I tried to find it again by searching Business Insider for Miley Cyrus. Big mistake. You would not believe how many posts came up.)
Feel free to discuss this graph, but the point I want to make is that based on this and the other articles I looked at, Hickey appears to have improved considerably when he moved to 538. I'm still not impressed with the work he's doing now, but that's an absolute, not a relative statement. Furthermore, this case raises some real questions about Noah Smith's claim that "In sum, this so-called “data-driven” website is significantly less data-driven (and less sophisticated) than Business Insider or Bloomberg View or The Atlantic."
A problem with 538 is that it can appear expert if you don't know about the area being written about ... and silly if you do. For example, they ran a piece about bike lanes and congestion. I know a fair amount about traffic congestion, so the piece was silly. But if you don't know much about traffic, it probably seemed deeper than it was.
ReplyDeleteThis is always a problem but it really, really helps to acknowledge in the piece what its limits are rather than choosing to make it look more authoritative than it is.