In February 2022, we announced1 that we would begin considering a new article type known as the Registered Report. Although this format is relatively well-known in the social sciences, it is not in mainstream use in the life sciences. Inspired by our colleagues at Nature Human Behaviour2, we saw an opportunity to use this format to solve some of the issues that we have commonly encountered when peer-reviewing ‘Analysis’ papers, which describe the results of method or tool performance comparison studies. We are so pleased to publish our first Registered Report in this issue, describing a quantitative comparative analysis of near-infrared fluorescent proteins, from Kiryl Piatkevich and colleagues. (To read about the authors’ inspiration and experience with this format, see their Behind the Paper post on the Springer Nature Protocols & Methods community.)

A published Registered Report in Nature Methods will look very similar to an Analysis. But the two formats undergo radically different peer review processes. Whereas an Analysis undergoes the typical review process that we are all familiar with, the review process for a Registered Report is shifted to the experimental design stage. Authors may submit a detailed experimental proposal for a method performance comparison study for peer review by experts (of course, if it first clears the editorial bar for novelty, interest and scope). If deemed technically sound, the editors may offer this ‘Stage 1’ manuscript an ‘accept in principle’ (AIP) decision before any experiments are actually performed by the authors. Once the authors have completed their experimental study, they submit a ‘Stage 2’ manuscript (now including Results), which the editors will send back to the reviewers for a technical check. However, the editors will not reject the Stage 2 manuscript for reasons of scooping or the perceived importance of the results.

When we first learned about the Registered Report format, we mused that Analysis submissions could benefit from this unique approach to peer review. We hoped that the focus of the reviewers would be shifted away from the subjective value of the study findings, and instead focused on the technical soundness of the research question and experimental design. We anticipated that the format would encourage greater transparency in data reporting and support the publication of negative results. We also felt that our ability to provide authors with editorial and expert reviewer feedback before beginning experiments would ensure the success of their study and help to avoid wasted time and effort. Now, having peer-reviewed approximately 10 Registered Report manuscripts over the past 2 years, we are even more convinced that the Registered Report is highly beneficial for method comparison studies. We also strongly believe that they should be more broadly embraced by the life sciences community in general.

Overall, we have received overwhelmingly positive feedback from authors, reviewers and readers, many of whom appreciated our initiative in trying something new in scientific publishing. However, the road to publishing Registered Reports has not been without some bumps. We have learned quite a lot and received much valuable feedback from our authors and reviewers.

From the perspective of the authors, we understand that the peer review process for a Registered Report can feel frustratingly slow, as our guidelines stipulate that authors should not begin any data collection until they have a Stage 1 AIP decision in-hand. Even in the best-case scenario, the process of Stage 1 peer review, manuscript revision and re-review is still likely to take several months, so authors need to have realistic expectations. But the format does not work if the authors have already begun their experiments and therefore cannot modify their experimental plan to adequately address the comments of the reviewers.

The original Registered Report article type was designed for psychology research, and journals that publish such studies have very strict guidelines that state that authors must follow their experimental design plan precisely or risk rejection at Stage 2. We realized early on, however, that we needed to allow for a little flexibility for authors to modify their experimental plan in cases in which it is unclear exactly what experimental conditions will be required to complete the project. We do encourage pilot studies (which should be described in the Stage 1 manuscript), and we wrote our guidelines to allow for some minor experimental optimization. However, the Registered Reports format requires that the experimental plan be by and large in place by Stage 1 AIP. Should new research questions arise during the course of the study, authors can feel free to pursue these routes of investigation, but the Stage 2 manuscript must label these as ‘exploratory analyses’.

Peer-reviewing a Registered Report requires a different mindset from that of a typical paper — something that some of our reviewers struggled with. We understand that it can be quite challenging to evaluate a paper based solely on the experimental design rather than the results. One piece of feedback we received suggested that reviewers might introduce their biases at the experimental design stage, for example by insisting that authors use a particular approach. Another common comment was that reviewers might ask for too many unnecessary experiments — something we have indeed observed during the review process of several Registered Report submissions. We have found that the role of the editor in evaluating the reviewer reports and communicating which experiments are essential and which are optional is crucial for the success of this format.

We also want to make the point that submitting an underdeveloped Stage 1 proposal in the hopes of having expert reviewers largely shape the study is by no means a realistic route to a published paper. We have a stringent editorial bar and will reject underdeveloped proposals without peer review, as we have learned that their review process will not be productive.

Interest in the life sciences community for Registered Reports is growing. Multiple Nature titles now publish or are considering publishing Registered Reports. We look forward to seeing the broader adoption of Registered Reports and encourage all potential Nature Methods authors who are interested in carrying out method performance comparison studies to strongly consider using this format.