What would be really bad is if this turns into a Freakonomics-type exercise, all contrarianism without any appreciation for the importance of actual expertise. And Michael Mann reminds me that Nate’s book already had some disturbing tendencies in that direction.
In response...
Silver, for his part, said he doesn't shun the negative reviews that FiveThirtyEight has drawn in its infancy, telling TPM that much of the criticism will help the site improve. He just doesn't think Krugman's assessment has been on the mark.For the record, if someone accuses you of having analysts bungee jump into areas they know nothing about then crank out a bunch of contrarian findings, the fact that you just hired Emily Oster should not feature prominently in your defense.
"His comment about experts was particularly strange given that (i) we publish lots of articles by experts, e.g. academic economists like Emily Oster and political scientists like Dan Hopkins and that (ii) Krugman has himself been very critical of Very Serious People and experts in economics and other fields," Silver wrote in the email.
Dan Hopkins is a good guy. He's posted on the Monkey Cage too. I assume that 538 pays better (we pay close to zero) and will give him more exposure, at least in the short run. A few years ago I was posting on 538 for free, so it would seem to make even more sense for Dan to post there for a reasonable rate of pay.
ReplyDeleteRegarding the site in general, the real point is that brand names aren't magic. Nate Silver is excellent, but slapping his name on a website doesn't automatically instill quality in the other contributions. It's not like a restaurant where you could imagine successful franchising with careful quality control, training, etc. It's more like one of these movies where they say "Steven Spielberg presents..." which means that Spielberg gave it his seal of approval but didn't really direct it. From that perspective, I think people's expectations of Nate's site were never realistic.
Consider this: Nate is an unusually talented combination of analyst, writer, and entrepreneur. This is obvious: if Nate weren't unusually talented, we wouldn't have heard of him. But the flip side of this is that it's not like he can just snap his fingers and clone himself.
To me, the most interesting thing Nate could do with his budget is not to hire Emily Oster or even Dan Hopkins to write independent pieces--ultimately, hiring someone to write you their own article is not much different from just linking to existing articles, e.g. Nate could just regularly link to the Monkey Cage, Marginal Revolution, and even Krugman (!) as approppriate--but rather if he were to hire people to collaborate with, both in the nitty gritty (for example, hiring some people who are experts on economic and political data who can find him the numbers he needs, right away) and in the interpretation (so that when Nate or his collaborators come up with an idea, they'll have someone to bounce it off of).
I don't know how Nate is running his group. To the extent that the people writing the articles are working independently, I don't really see the point (except to supply a steady flow of content, which is fine, but it doesn't really speak to Nate's strengths as an analyst and writer). But if Nate is sitting in a room collaborating with people, that could make a difference.
There are a few issues I've been chipping away at (a questionable business model, poor response to critics, a fundamental misreading of his own strengths -- the last in the upcoming hedgehog post), but I think the underlying question is do the incentives encourage someone like Silver to make the greatest contribution to the discourse and the science?
DeleteMost of the new wave of critics (very much including Krugman) are complaining, not that the site is bad but that Silver is not making good use of his talents. The trouble is that this is one of those points where social good and standard career metrics deviate. The social good is best served Silver by hiring a bunch of assistants to maximize his productivity as a researcher and a writer, but revenue, traffic, fame and popular influence will be maximized by taking the Freakonomics path.
Huffington Post may be a better analogy than Freakonomics and I probably should have said "collaborators and assistants."
DeleteMark:
ReplyDeleteI've actually thought a lot about this. One issue is that social good might be maximized by Nate doing more analyses, but that's a lot of work for Nate! I imagine that one motivation of the new website, from Nate's perspective, is that he is not under the gun to produce a new analysis every day or two Producing a column ever few days (as Krugman does) or even every day (as I do on my blog) is not so much effort (for those of us who can write fluidly). Doing a new analysis every couple days is a huge amount of work, especially if your motivation is not internal but rather is an external need for content to fill a slot. From Nate's point of view, it's gotta be a lot more pleasant to have Dan Hopkins and Emily Oster do the work and for him to just relax a bit. I think Nate's earlier model, in which he produces original analyses on a regular basis, is not easily sustainable on his part.
LIke Andrew, I also have been thinking about this, and I come out on the side of Nate. Individually, the critique stands but taken together, they don't call for any coherent vision of how his critics would run an operation such as his. The level of rigor that Krugman and others demand requires years, perhaps decades, of research to write one piece; meanwhile, the other critique is the content is not timely. Think about the full-time journalists he has hired - there isn't a way to pay them enough to do the kind of pieces that are being imagined. As we all know, data collection, cleaning and analysis take a huge amount of time. It may be months of work to get one article out. Further, I'd like to judge them relative to competitors rather than in some kind of abstract universe. Compared to the Freakonomics blog, for example, 538 has a much better orientation. Compare to Huffington Post - when did HP have any real data journalism? Compare to Buzzfeed, don't even want to talk about it.
ReplyDeleteKaiser:
DeleteOne might say that in this case the role of the new 538 is not to raise the quality of data journalism by adding revolutionary new analyses, rather it would raise the quality of data journalism by increasing the supply, in two ways:
(1) By paying people such as Emily Oster, Dan Hopkins, and Carl Bialik to do data journalism, you're get them to spend more of their time and effort on data journalism and less on research or teaching (in the case of Oster and Hopkins) or softer journalism (in the case of Bialik).
(2) By paying people to work at 538, you're freeing up new slots for data journalism, for example now the WSJ will want to hire someone to replace Bialik, and we at the Monkey Cage will want to get someone to write the sorts of posts that Hopkins was writing for us. It's sort of like if Harvard went and hired a bunch of professors from Yale. Then Yale would have to replace the hires, etc etc. It's a trickle-up effect.
As an economist, Krugman (and Tyler Cowen, another highly-visible critic of the new 538) should be well aware of these processes. That said, it seems like their criticism is more of specific pieces (in particular, some of 538's econ articles) and in this case their argument, I think, is that Nate is burning up some of 538's hard-won reputation. This could also be true. The new 538 could be a plus for data journalism in the aggregate even if some of its pieces are weak. As you say, at least they're better than Buzzfeed!