Thursday, January 18, 2018

Double blind peer review

This is Joseph

I was reading this piece by Andrew Gelman and this led me to this other article in the comments.  The discussion was a journal being annoyed by preprints, and one reason that people wondered if it might be so was double blind peer review.  So the comments on the challenges of double blind peer review are well worth thinking about:

A related problem with mandatory DBPR, if the journal wants to actually attempt to enforce it (in my experience, many problems in any form of professional life start when someone creates a rule and then tries to be consistent in enforcing it, despite the messiness of the world), is that in addition to the assumption that the manuscript is not available through Google, it also assumes, more completely, that it has not previously been seen by the reviewers in an unblinded state.  That seems like a rather untenable assumption, especially in specialised fields.  PSPB is a well-respected journal by any measure, but like any journal ("Cell wouldn't take it? Let's try Nature!") it may not always be the first port of call for the authors who submit there.  Should the reviewer who has already seen the manuscript unblinded on behalf of another journal recuse herself because she knows who the author is, thus depriving the editor of an expert opinion (which, as a bonus, could presumably be provided very quickly

I am actually pretty good at guessing who the authors are when I review a double blinded paper, even if I am not specifically trying.  Part of it is that some pieces are informative -- a paper on the Framingham Heart Study has a limited pool of typical authors.  Journals ask people to include notes about ethical review (gives the institution).  And the citations are a pretty big clue if there is any building upon previous work.

Does this mean that I shouldn't review in areas where I know the field well?

It is not a trivial problem.  But I think I might err on the side of free information above strict blinding if I had to make a call.  But it's definitely an issue I want to think more about.  

No comments:

Post a Comment