The peer review process can be frustrating to researchers eager to get their work published. Changes to the process might be warranted—but only if they are based in fact, not conjecture.
Confidential peer review is a cornerstone of the publication process in science, but is not without its drawbacks. A recent open letter (http://www.eurostemcell.org/commentanalysis/peer-review/) signed by 14 stem cell researchers to the editors of major scientific journals cites “unreasonable or obstructive reviews” and the fact that “publication of truly original findings may be delayed or rejected” as reasons to refine our current practices. To increase transparency, the signatories suggest appending reviewers' comments and editorial correspondence alongside published manuscripts.
This is not the first time scientists have voiced concerns over the peer review process, and, consequently, at Nature Medicine a number of mechanisms are in place to prevent bias. The editorial team discusses every paper before referees are chosen and articles are sent out for external review. In a given year, we draw upon hundreds of reviewers to assess manuscripts. Including both established researchers and young investigators, our pool of reviewers is in constant flux, preventing a too-small number of individuals from directing what is ultimately published. A common fear among authors is that rival scientists will make unreasonable demands to intentionally delay or halt publication of data that may 'scoop' or contradict the reviewers' own findings. We recognize such conflicts of interest and permit a list of three excluded reviewers. We also ask other referees to weigh in on conflicting reviews or bring in additional expertise to evaluate the disputed aspect of the manuscript. We also attend meetings and talk with scientists one on one, which allows us to stay abreast of developments in the field and to avoid sending manuscripts to investigators with conflicting interests. Our goal is to actively engage with the communities whose work we publish so that we can better serve them.
We also encourage a frank dialogue on papers after publication and are willing to discuss ways to improve our decision-making process. Several times a year, editors from across Nature Publishing Group identify a topic that our journals publish on regularly and convene a meeting at our offices. Several scientists are asked to identify strong and weak studies that were published in our journals and are invited to discuss the merits and drawbacks of the studies and to outline standards of excellence for publication in the field. This discussion allows us to refine our editorial guidelines and identify what to expect from future submissions (both in technical and conceptual terms).
The authors of the open letter advocate for more profound changes, including publication of supplementary files containing anonymous referees' comments and relevant editorial communication. These measures might allow readers to appreciate how the review process has shaped a paper and might provide insight into the peer review process to younger investigators. These benefits notwithstanding, a number of lingering concerns prevent us from endorsing this strategy.
Publication of referees' comments in full may affect the quality of the reviews, leading to more cautious and restrained comments. It is difficult to ascertain how much the quality of reviews would be compromised by adopting these measures; however, previous attempts with open peer review suggest that referees are less likely to provide a direct and detailed evaluation of the report. Authors may also be reluctant to adopt this strategy, as publication of earlier reviews may expose flaws that were addressed in later submissions (or never addressed satisfactorily) but nonetheless color interpretation of the findings. Finally, most articles undergo numerous rounds of review. As such, it is difficult to evaluate the importance of comments made in the first round of review without reference to that earlier version of the manuscript. Simply publishing the reviewers' comments as a whole removes them from the context in which the decisions were made and may place undue emphasis on specific points of concern.
We do not want our readership to feel that the editors are indifferent to the concerns voiced by the stem cell researchers. The peer review process might be improved, but it remains unclear whether publishing reviewers' comments would provide a measurable advantage that outweighs the drawbacks of adopting this approach. When properly managed, confidential peer review allows for a fair and unbiased assessment and ensures that we publish papers of the highest quality. We invite you to continue this discussion at http://blogs.nature.com/nm/spoonful/.
Rights and permissions
About this article
Cite this article
Peering into review. Nat Med 16, 239 (2010). https://doi.org/10.1038/nm0310-239
Issue Date:
DOI: https://doi.org/10.1038/nm0310-239