Although peer review is far from perfect, it is currently our best tool to evaluate the robustness and validity of scientific research. However, the peer review process is a black box for many scientists, especially early-career researchers. We take a look at popular but untrue conceptions of the process at our journal.

Fiction: ‘Who I am will have a role in whether my research is sent out to review or not’.

Fact: We evaluate manuscripts, not people. Regardless of whether you are a Nobel laureate or a PhD student submitting your first manuscript, we use the same criteria to make peer review decisions (see our previous Editorial). We are editors by trade and no longer hold academic positions: this removes potential conflicts of interest with scientists who would otherwise have influence over our careers (for example, in grant and job applications or our own manuscript submissions). We disregard cover letters in which well-established scientists extensively discuss their credentials or past work if this information is unrelated to the work at hand.

Fiction: ‘It’s pointless to suggest or exclude reviewers — editors ignore these suggestions/requests’.

Fact: We honour all requests for reviewer exclusions — no questions asked — provided that these comprise no more than four individuals or research groups. We want your work to be evaluated fairly, so there is no point in inviting reviewers who you have reason to believe are unable to do so. However, if your list of exclusions is extensive, includes whole institutions or makes it otherwise impossible for us to recruit well-qualified reviewers, we will contact you to discuss this and ask for a more circumscribed list of exclusions.

We also value reviewer suggestions, especially of experts who are not immediately obvious (that is, avoid recommending the best-known researchers in a specific field), experts who come from diverse backgrounds that are underrepresented in science and, in all cases, experts with whom you have no professional or personal relationship. Typically, no more than one of the reviewers we secure to review your manuscript will have been recommended by you; however, all of your suggestions are valuable, as they allow us to enrich our pool of potential reviewers in the future.

Fiction: ‘Editors take all reviewer comments at face value’.

Fact: Editors are trained to critically evaluate reviewer reports in the same way we evaluate manuscripts. Reviewer reports vary in their quality, depth of engagement with the work, and strength of reasoning and evidence underlying their arguments or recommendations. Very brief, superficial reviews that suggest the reviewer did not engage with the manuscript in sufficient detail have little if any bearing on editorial decisions, no matter how positive or negative they may be. A review that only says ‘This is groundbreaking — accept’ or ‘This is rubbish — reject’, without detailed evidence and arguments to support the recommendation, is rarely helpful. Although we do not suppress reviews (except if we have discovered a disqualifying competing interest that the reviewer failed to disclose when they accepted the review), we base our decisions on reviewer feedback that is thoughtful, detailed, constructive and provides strong evidence in support of all comments made.

Fiction: ‘Editors count votes when making decisions’.

Fact: We do not ask reviewers to make recommendations for editorial outcomes: although reviewers may provide recommendations within their comments (and they frequently do), we do not provide a drop-down menu of recommendations. Instead, we rely on the content of reviews, the issues raised and how these relate to journal criteria for publication. For example, some reviewers may recommend rejection despite the fact that the issues are in principle addressable; others may recommend revision but the issues raised reveal fundamental flaws in the project that would require the authors to go back to the drawing board. The recommendations themselves are not reliable signals for decision-making — the substantive comments of the reviewers and issues raised are.

Frequently, what reviewers perceive as an important advance does not align with journal criteria of what constitutes a significant contribution suitable for publication in the journal. For instance, a reviewer may be concerned about the lack of conceptual novelty in the manuscript; however, we consider advances in evidence or advances with practical or policy significance equally important to publish even if they lack conceptual or methodological novelty, and we frequently overrule novelty concerns raised by the reviewers. On the other hand, a fundamental flaw identified by a methods expert will override glowing recommendations by reviewers who lack in-depth methodological expertise.

Fiction: ‘You can accurately guess the identity of a critical reviewer’.

Fact: You cannot. In our experience, author guesses of critical reviewers are almost always inaccurate. Occasionally, a very negative reviewer may be a reviewer you recommended. There is little point trying to guess the identity of an unsigned reviewer report. You are better off focusing on the substance of the feedback and what evidence the reviewer has provided to support it.

Fiction: ‘Editors always share reviewer reports exactly as they were originally submitted’.

Fact: We do not edit or alter reviewer reports — reviews are the intellectual property of reviewers, who hold copyright over their work. However, on occasion, we do ask reviewers to amend their reviews before we share them with the authors. For example, sometimes confidential comments to the editors and comments to the authors are not aligned. If the content of confidential comments is crucial for decision-making, we ask reviewers to amend their comments to the authors to include this information, so that decision-making is transparent. On other rare occasions, the content of a review may cross the line into being offensive: in these cases, we again ask reviewers to edit their reviews before we share them with the authors.

Are there issues surrounding peer review we haven’t covered? Contact us at humanbehaviour@nature.com to let us know.