The new peer review ecosystem

← back to the blog

I've blogged previously about the problems I have with peer-review, especially about how lower publication costs would seem to make pre-publication review unnecessary. But others have put it much better than me:

From Michael Eisen's blog:

[Peer review] is conservative, cumbersome, capricious and intrusive. It slows down the communication of new ideas and discoveries, while failing to accomplish most of what it purports to do. And, worst of all, the mythical veneer of peer review has created the perception that a handful of journals stand as gatekeepers of success in science, ceding undue power to them, and thereby stifling innovation in scientific communication.

From the journal Breast Cancer Research:

Peer review ... is an ineffective, slow, expensive, biased, inefficient, anti-innovatory ... easily abused lottery.

These are just the opinions of the authors, of course, but I think few would argue that the system needs a shake-up, at the very least. Here are some the ideas being tried out in the wild:

  • F1000Research switches the ordering of peer-review and publication. Papers are accepted after a simple "sanity check" by the editors, and published (i.e. put online) immediately. Reviews are published alongside the paper as they come in. However, the paper is not indexed in PubMed until the reviewers are satisfied, so it does bare some similarity of pre-publication review, as well.
  • ScienceOpen also reviews after publication. They additionally focus on giving reviewers credit for their work, which should increase the quality of review in at least two ways. First, clever reviewers can be identified by their insight, and the removal of anonymity should discourage unhelpful "review trolls".
  • PLOS One practices pre-publication peer-review, but asks reviewers not to consider the "impact" or "importance" of publications, instead focusing solely on the technical soundness of the paper. This sounds like common sense, actually. Even for glamour journals, I had always believed that the appropriateness of the paper should be the editors' call, and I've been continually surprised in my professional life at how much sway reviewers have in this area.
  • PubPeer is not a journal at all, but indexes publications so that readers can leave comments on them. As Eisen points out, serious technical problems are usually discovered long after publication, not before. More than any other group, PubPeer has made it clear that scientific communication is now a two-way street: readers talk back. In a startling inability to accept this reality, a prominent cancer researcher recently sued PubPeer's anonymous readers for defamation.
  • PubMed Commons is a forum for comments, just like PubPeer, but reviewers are not anonymous (in fact, they must have a published paper in the PubMed database). A possible advantage is that comments are displayed next to the PubMed abstract, the most commonly-used literature search tool in the biomedical field.
  • Axios Review has perhaps the most innovative idea: they are a private company that will charge authors a fee to solicit reviews. By performing the review first, they can better judge which journals are likely to accept the paper. Then they pass the paper and the reviews along to various target journals. This speeds up the publication process by wasting less time.

One last idea which deserves special mention is Nature Publishing Group's recently announced plan to fast-track peer review for authors that pay a fee. For sure, both Axios and Nature have identified a pain-point and found a solution driven by market incentives. The fact that both are making a buck should not cause us to dismiss them out-of-hand. While this promises to solve only one problem with peer-review in its current form, a greater diversity of ideas is probably for the best as we figure out what will work going forward. Because the one thing we know won't work anymore is what we have now.

I apologize if I missed anyone: please chime in in the comments.