and Editorials Op Ed — To Err is Human: Part 2 Retractions in Scholarly Journals

By Daniel S. Dotson (Professor, The Ohio State University, Columbus, OH 43210) <dotson.77@osu.edu>

Introduction

Mistakes happen. We all make them from time to time. But what does it say when our most respected sources — peer reviewed journals — publish mistakes? This is the second in a series of three articles on issues related to errata and retractions in scholarly journals. This article will focus on the frequency of retractions.

Methodology

What happens when a submitted and accepted article goes beyond error correction? In such cases, this may rise to the level of a retraction, which is also tracked by Scopus using the document type “Retracted” for such items. Scopus was chosen for its wider title coverage than Web of Science and its ability to export data for analysis.

For the same time period 2012-2021 as the previous article on errata, there were 9,206 items listed as Retracted. The number of retractions is a much smaller number compared to errata (over 229,000), with only 94 journals containing more than ten retractions for this period. This document type in Scopus thus allows users to identify these and in the case for this article, see what journals, disciplines, and publishers tend to have higher number of retractions.

In order to find these retractions, all that needs to be done is to enter Scopus and conduct search for the source title and then limit to the document type Retracted. Finding all retractions is a bit more work, but a search for all fields using the following search should get every document indexed: a* OR b* OR c* OR d* OR e* OR f* OR g* OR h* OR i* OR j* OR k* OR l* OR m* OR n* OR o* OR p* OR q* OR r* OR s* OR t* OR u* OR v* OR w* OR x* OR y* OR z*

For the purposes of this study, the items were limited to 2012-2021 content prior to search. After this point, to examine top titles:

1. The source type was limited to Journal and data about the types of document types was recorded.

2. The Retracted document type was chosen and refined.

3. Only journals with ten or more retractions were examined more closely.

4. A separate search was conducted to obtain total document counts for each journal. The totals were used to identify the percentage of documents that were retractions for each journal.

5. Journal Citation Reports was examined to determine each journal’s impact factor and its highest listed rank. Some journals were ranked in multiple fields, the highest ranking from these was recorded.

6. The current publisher for each journal was recorded.

7. I assigned broad discipline (e.g., medicine, physics) to each journal based upon the journal’s content coverage.

8. Journals with less than 10 retractions were excluded.

This resulted in 6,791 retractions for closer inspection. These items’ journals were examined to see how many total titles of all document types were indexed during this year range and to discover what percentage of their titles published were retractions.

Results

Examining these titles, we see a similar pattern to errata in that medicine journals dominated the items examined. However, in this case, interdisciplinary titles took second place. But looking at the percentage of the retractions rather than title distribution, medicine’s percentage of retractions is more in the middle. Several disciplines had over 1% of their journal content titles indexed as retractions, but note these disciplines had only one or two journals examined. Biology was the area with more than two journals with the highest retraction rate from the journals examined. Medicine, a heavy hitter for errata, had a lower percentage than several areas, but still had the largest number of titles (37, or 39.4% of the journals). See Figure 1 page 27.

Examining the publishers of these titles, a smaller number of publishers was seen compared to the errata and commercial large publishers dominated. However, a few smaller publishers had three or more journals on the list, as seen in Figure 2 on page 28.

Examining the journal titles with the highest percentage of retractions is quite a different experience from examining the errata. In this case (see Table 1 page 28), one sees more:

• Niche titles than general titles

• Large commercial publishers

Similar to errata, the impact factor and rankings of journals were examined by using Journal Citation Reports and selecting the highest listed ranking. While not as high as the errata titles, these journals also had fairly high rankings and impact factors, indicating that these are, for the most part, not low-quality journals. Note only one title had no impact factor. See Figure 3 page 28.

Takeaways

In summary, examining the data for these journals with the most retractions published 2012-2021 revealed:

Many reputable journals have published content requiring retractions.

Publishers of all types are publishing content requiring retractions.

Medicine has a lot of journals with retractions, but also medicine has a lot of journals in general.

Medicine is not as dominant in the top retraction percentage titles compared to errata. What cannot be determined via the data for this study, or were not examined closer, but are potential areas to explore further:

Why are these retractions happening?

Are journals with less retractions doing something different?

Are other journals doing a better job at catching problems prior to publication?

Do they get less items requiring retractions to begin with?

Are they simply removing retracted items, but not notifying databases of changes?

Are they doing something different with how retractions are labeled?

Are items that should be retracted just not retracted (whether by authors or the journal)?

Are some journals treating these as errata or some other category rather than labeling as a retraction?