Abstracts as pitches – – how a Wansink paper is like Will Ferrell's Land of the Lost reboot

Picking up from Andrew Gelman's recent post on why so much of what we hear about the replication crisis seems to center on social psychology, My take is that we should talk more about our old friend hype and the way it distorts discussions and corrupts important processes.

When I started to write out a reply in the comment section, it struck me that there was a pretty good analogy here with movie pitches (and obviously, if I come up with a pretty good analogy, I'm going to save it for a blog post). Researchers in today's environment have a strong incentive to generate buzz around their work.

In a sense, you can think of a paper's abstract as a pitch directed at reporters. Though the scale is very different, the basic dynamic is similar to that of a producer approaching a studio. Both Disney and the New York Times are looking for story ideas that will bring in large audiences, both have limited time and attention, and both are going to make the initial decision to proceed based on a quick summary of the story.

That intermediate step is very important. It means that the primary focus of both producer and researcher needs to be on the pitch more than on the quality or appeal of the larger story. Something that is easily pitched has an enormous advantage.

It is almost certainly not a coincidence that, when you see a big, underperforming movie come out, it's generally easy to imagine how the initial presentation went. "Look at those numbers for Night at the Museum. big-budget action comedy with lots of special effects and dinosaurs. We'll do the same thing with that 70s kids show, Land of the Lost . Old TV shows are big. Will Ferrell is big. We'll just put them all together." .

If you were a researcher trying to get a reporter to write up your study, what areas would "pitch well"? Three that come to mind are broad statements about human nature, dietary findings, and most of all "news you can use"/self-help. You'll notice that studies that fall into one or more of these categories tend to do very well in the hype market. As a consequence, there's also more of an incentive to push these through despite low-quality,
.
Of course, along with the hype comes increased scrutiny and a greater chance that someone will try to replicate the findings.

While I don't want to push the analogy too far or over into size one aspect of a complicated process, this might be a useful way of thinking about at least part of the problem.