PeerRef and the Future of Preprint Peer Review

By Elliott Lumb (Founder CEO, PeerRef) <elliott@peerref.com>

Introduction

Academic publishing has been in a state of constant change over the past decade. Open Access has become the primary way to publish research. A growing number of national mandates are driving the increase in open data sharing. The use of preprints continues to rise. Traditional publishers are continuing to grapple with the new business models associated with open research. Recently, peer review has become a focus for innovation. Several platforms, including PeerRef, are striving to improve the peer review process.

What is Peer Review

Researchers and funders agree that peer review is an integral part of the scholarly communication process. However, there is limited consensus on what constitutes peer review. We believe that traditional, journal-based peer review has three main facets: reviewers provide insight into the rigour and validity of a manuscript, give feedback on how to improve a manuscript, and assess a manuscript’s fit for a particular journal. Journal fit can derive from a range of standards including scope, perceived impact, and novelty. Repeated peer review at multiple journals is a common occurrence due to high levels of rejection following peer review. We believe the traditional journal-based approach is wasteful and creates more value for publishers than it does for the research community.

The Rise and Utility of Preprints

The publication of preprints has grown rapidly over the past few years. Over 450,000 preprints were published in 2022, representing approximately 10% of scholarly publications. The growth in preprints accelerated during the COVID-19 pandemic when it became more important than ever to rapidly share results. The number of preprints continues to grow year-on-year. We expect this growth to continue as researchers, particularly early career researchers, find unique value in preprints. Funders are also driving this growth as they continue to encourage the use of preprints.

The rise of preprints has provided benefits and new opportunities across scholarly communication. Preprints can accelerate discovery, be used as evidence of productivity in grant or job applications, and are associated with higher citations for the Version of Record. Preprints are also enabling innovation in peer review.

What is PeerRef

We launched PeerRef in September 2021. Our goals are to make peer review open, provide researchers with greater choice in how their research is shared and evaluated, and reduce waste in the academic publishing process.

PeerRef is a journal-independent open peer review platform. It helps authors by organising open peer review for their preprints. The peer review reports are published on the PeerRef platform, assigned a DOI, and posted alongside preprints on several preprint platforms. Authors can use the feedback in these reviews to improve their research. The reviews can signal to readers that the research is verified by expert peers, without journal publication. Readers also benefit from the extra context provided in the open reviews.

Journal-independent peer review of preprints gives authors more choice in how they share research. They can use the package of their preprint and its peer reviews as the final product, or they can send the preprint and reviews to a journal editor, who can use the reviews to inform or make rapid publication decisions. The reuse of PeerRef’s open assessment creates a single point of peer review, which reduces waste by eliminating the need for repeated peer review at successive journals.

PeerRef is field agnostic, serving all research communities. Its journal-independent open peer review of preprints can revolutionise the academic publishing process and benefit authors, reviewers, publishers, and research funders.

Sourcing Peer Reviewers

Journal editors note that it is difficult to source peer reviewers. It is also reported that 10% of reviewers do 50% of global peer review. We hypothesise that the root cause for both phenomena is that a relatively small group of researchers are repeatedly asked to conduct peer review. This may be caused by journal editors only asking their established pool of reviewers to evaluate articles.

PeerRef has a unique approach to peer reviewer selection. It does not have an editorial board. Instead, it has a tool-first approach to sourcing reviewers, using a mix of internal and third-party tools to identify suitable expert reviewers for all types of research at scale. In addition to improving efficiency and precision, we intend for this approach to broaden the pool of reviewers to include more early-career researchers and researchers from middle and low-income countries.

A Better Approach for Authors

For authors, peer review at a single journal can take 4-12 months. There is a global average rejection rate of 45% , and the reviewer reports are not publicly shared. Therefore, almost half of the research must undergo several rounds of peer review at multiple journals. With 5 million publications in 2022, it is estimated that repeated peer review wasted 45 million hours of researcher time. This process is delaying research and distracting researchers from their priorities. We can solve this problem by organising peer review of preprints outside of a journal and publishing the peer review reports. Peer review can be conducted once with journal-independent peer review and used by anyone, including journal editors. Journal editors will no longer need to organise a full round of peer review themselves. Instead, they can use existing open peer reviews to inform or make publication decisions. This accelerates publication, reduces repeated peer review, and gives researchers more time to focus on generating knowledge.

Typically, authors can only access peer review by submitting their research to a journal for publication, which is arguably not necessary for all research. Journal publication can be a timeconsuming and expensive process. And the research must meet a journal’s criteria for acceptance, which often entails being perceived as novel or potentially impactful and communicating positive results. These requirements tend to preclude many types of research, such as iterative studies, replication studies, or studies with negative results, that may benefit from peer review. By decoupling peer review from journals, we can empower authors to seek evaluation for more types of research and to choose which of their peer-reviewed outputs should go through the journal publication process.

Recognition for Reviewers

In traditional peer review, the reviewer provides a service to a journal. They help to protect the journal’s brand by advising an editor on whether a piece of research meets a subjective set of standards. This is not the case with journal-independent open peer review of preprints. In this model, the reviewer is solely focused on assessing the rigour and validity of the work without acting as a gatekeeper. By putting the research at the centre of the assessment, the reviewer can provide constructive feedback from a collegial, rather than adversarial, position. Ultimately, when conducting open peer review on preprints, the reviewer provides their service and expertise to the author and research community, which benefits from the open assessment.

Reviewers can also gain from conducting open preprint peer review. Based on 5 million articles published in 2022, there was an estimated 100 million hours of peer review conducted. This enormous amount of scholarly work is not in the public domain. These reviews could be used to provide more context to research articles, used to train reviewers, and be used in research assessment. PeerRef publishes signed peer reviews and assigns DOIs to them. This enables reviewers to share their work and add it to their CVs and allows others to cite their reviews. Funders and institutions are beginning to signal that they will consider published peer reviews in grant proposals and promotion and tenure decisions. Services such as Reviewer Credits, a partner of PeerRef, help with this. Reviewer Credits enables reviewers to keep track of all their open peer review activity in one place and easily share their peer review contribution with funders and institutions.

Future Role of Publishers

In this new workflow where authors can freely publish their research on preprint platforms and request peer review from platforms such as PeerRef, we believe there remains a role for publishers. Publishers have existed for centuries and are deeply entrenched in scholarly communication. We do not believe that peer review is the core value proposition of a publisher; rather, publishers create most of the value for the author after publication. This value can be high-quality article production, community building, or helping research to be as discoverable and societally impactful as possible. Publishers can influence discovery and impact in a variety of ways. The primary tools to discover research are Google and Google scholar, so it is important for journals to have strong SEO. In some communities, expert human curation is essential. For specific types of research, the impact may depend on the publisher getting that research to policymakers, journalists, or patient groups. As peer review is decoupled from journals, publishers can focus on these roles where they add the most value.

A New Ecosystem

Several platforms are innovating in the space of preprint peer review, each offering its unique service. Review Commons, Peer Community in, and Biophysics Colab organise peer review of preprints for specific research communities. PREreview allows authors to request peer review for their preprint from the PREreview community. It also offers resources and training on how to write a high-quality peer review. eLife has also recently transitioned from a selective journal to a publisher of peer-reviewed preprints.

Other platforms help the preprint review ecosystem to grow and thrive. Sciety, an eLife product, aggregates peer reviewed preprints so users can discover and curate lists of evaluated preprints. JMIR, a PeerRef partner journal, created Plan P. This initiative supports certified preprint peer review services and allows authors to publish a Version of Record of peer reviewed preprints.

Funders are also beginning to recognise the utility of preprint peer review. In 2022, cOAlition S announced that they consider peer reviewed preprints to be of equivalent merit to peer-reviewed journal publications. Several research funders have also individually committed to including peer-reviewed preprints in their evaluation processes.

The preprint peer review ecosystem is in its infancy. Over the next decade, we expect preprint peer review to become the primary way research is evaluated. As PeerRef develops, we will drive progress towards an ecosystem of scholarly communication in which open peer review is embraced and waste is reduced, allowing researchers and publishers to focus on where they create real value for their community and society.