The New World of Preprint Review

By Michele Avissar-Whiting (Program Officer, Howard Hughes Medical Institute) <michavissar@gmail.com>

On the first two days of December, I had the pleasure of visiting the ultramodern, ultra-picturesque Janelia Research Campus for the first time. The occasion: an intensive two-day workshop with ~200 colleagues from across the research, publishing, and funding spaces to dive deep on the topic of preprint review. The timing was ideal because I had just recruited writers for a preprint-focused issue of Against the Grain, and the peer review angle seemed to have a special gravity. This signaled to me that the novelty of preprints has worn off. Just a few years ago, it was not unusual for a journal to refuse to review a paper that had been posted on a preprint server. Now, at least one journal will only review preprinted submissions. We’ve come a long way in a short time on preprints, and we are moving on to the next phase: what does peer review look like in a preprint world?

There is a reluctance to refer to this practice as post-publication review — lest we admit that posting a preprint counts as “publication” — but that is effectively what it is. Preprint review refers to the act of layering scrutiny, analysis, and context onto a piece of scholarship that has been made publicly available by its authors. It happens directly on preprint servers, on specialized preprintreview platforms and (arguably) on social media — although to wildly varying degrees between disciplines and communities. Amidst the feverish proliferation of preprints in the first year of the COVID-19 pandemic, a paradoxical status became more prevalent. Preprints — which had been consistently defined as “unreviewed manuscripts” and emblazoned with cautionary text to reflect this fact — were getting reviewed. And the reviews were publicly accessible to all readers. Some, in fact, were reviewed more thoroughly within a few days of publication than most journal articles have ever been reviewed.

Still, most preprints are not publicly peer reviewed. In fact, as far as anyone can tell, they do not get reviewed at all until a subsequent version ends up published in a journal. This is unfortunate, as sharing new findings openly should be considered as part one of a multi-part open process. Public scrutiny and scholarly discourse between the authors and the community should follow. But just as it has taken time for researchers — particularly in the life science fields — to come around to the idea of posting preprints, it will take time to accept the idea of public peer review.

Going (ahem) against the grain and breaking with norms is hard, but the discussions at the Janelia meeting and the content of this issue’s features give me hope that change is underway. Elliott Lumb from PeerRef and Stuart King from eLife first discuss the myriad of benefits of preprint peer review and how their organizations are helping — in different ways — to normalize and operationalize the practice. Next the PREreview team looks at peer review of preprints through the lens of equity and explains how PREreview aims to educate and expand opportunities for participation in the practice. Ben Mudrak from ChemRxiv provides a view into preprint adoption in chemistry and touches on attitudes toward preprint review in this more recent entrant into the preprint space. Lastly, Michael Parkin gives us a tour of EuropePMC’s approach to surfacing peer review of preprints.