Science is in flux, with the rise of big science and big data, the breakdown of traditional disciplinary walls, and an ever-increasing presence of machine learning. What do these developments mean for peer review? This is the question asked by Peer Review Week 2023, happening this September. We ponder how physics referees and editors may need to adapt to a changing world.

Physics — especially high-energy, nuclear and astrophysics — increasingly happens in large collaborations, which are the only way to tackle the great complexity of big experiments and observations. If a result arises from the analysis of a petabyte-scale dataset that comes from the only instrument in the world capable of gathering it, what does it mean to peer review the publication? Unlike in the traditional model of peer review, where the important checks happen at the journal, much of the scrutiny happens internally to the collaboration, before the manuscript is even submitted. This practice does not make external peer review superfluous, but allows it to focus on the big picture rather than inspecting the methods, a job that requires a high level of familiarity with the details of the project. But the wider community doesn’t have to take the internal checks on blind trust: the procedures and analysis pipelines are all published, providing a layer of transparency. This approach may be most prevalent in big science, but its existence shows that peer review doesn’t always have to happen — and indeed cannot happen — in the same way.

“peer review doesn’t always have to happen — and indeed cannot happen — in the same way”

Even areas of physics that tend to happen in smaller research groups need to adapt, as researchers increasingly publish new types of outputs such as code and data. Peer reviewing such research objects poses practical problems. If the code has millions of lines or can only run on a supercomputer, or the dataset is much larger than a referee’s hard drive, can they test it beyond commenting on the authors’ description of the algorithm or data-collection method? In response to these kinds of questions, researchers are starting to call for open workflows, which go beyond just making code and data available.

These issues are less relevant for us, as a journal that publishes review articles, compared to journals that publish primary research. But we are also confronted by changes in how physics is done. For example, many of our review-type articles are interdisciplinary, reflecting the increasingly interdisciplinary nature of physics. If a manuscript is relevant for several fields, we want the advice of experts in all those areas. These referees must assess a manuscript across differences in jargon and ways of thinking about science; often referees from different disciplines will have very different ideas about how a manuscript could be improved. As editors, our job is to take these sometimes contradictory suggestions, and synthesize them into actionable guidance for our authors. But, as interdisciplinary researchers attest, it is essential to have referees who value work that crosses disciplinary boundaries.

Given all these challenges, it is clear that peer review cannot remain static, but must adapt. Indeed, although the modern peer review process feels like a bedrock of science, it is actually quite recent1 and has adapted before. It is possible that some of the adaptation will be technological — many are asking if artificial intelligence (AI) could help. But AI is not yet ready to assess research output2. There are also ethics questions that need resolving, such as the potential privacy issues involved in feeding a manuscript into a machine learning model, which has led the National Institutes of Health to ban the use of AI in grant reviewing3. Rather than jumping to technological fixes, there needs to be a conversation across physics about what different fields want from peer review, and what the best way is to achieve it. At Nature Reviews Physics, we want to be part of that conversation, and value hearing from our readers how they would like to see peer review continue to evolve.