and Editorials Op Ed — To Err is Human: Part 1 Errata in Scholarly Journals?

By Daniel S. Dotson (Professor, The Ohio State University, Columbus, OH 43210) <dotson.77@osu.edu>

Introduction

Mistakes happen. We all make them from time to time. But what does it say when our most respected sources — peer reviewed journals — publish mistakes? This is the first in a series of three articles on issues related to errata and retractions in scholarly journals. This article will focus on the frequency of errata.

Methodology

Scopus assigns a document type of “Erratum” to documents which are information about errors in a previous publication. This document type in Scopus thus allows database users to identify items with errors and in the case for this article, see what journals, disciplines, and publishers tend to have higher numbers of errata. Scopus was chosen for its wider title coverage than Web of Science and its ability to export data for analysis.

In order to find these errata, all that needs to be done is to enter Scopus and conduct search for the source title and then limit to the document type Erratum. Finding all errata is a bit more work, but a search for all fields using the following search should get every document indexed: a* OR b* OR c* OR d* OR e* OR f* OR g* OR h* OR i* OR j* OR k* OR l* OR m* OR n* OR o* OR p* OR q* OR r* OR s* OR t* OR u* OR v* OR w* OR x* OR y* OR z*

For the purposes of this study, the items were limited to 2012-2021 content prior to search. After the search, to examine top titles:

1. The source type was limited to Journal and data about the categories of document types was recorded.

2. The Erratum document type was chosen, and the results refined. Note Erratum is used for all erratum categories (e.g., corrigendum), but not for retractions.

3. The journals with the highest quantities of errata were selected. Thus, 99 titles were selected to review further (100 and 101 were a tie, otherwise 100 would have been chosen).

4. A separate search was conducted to obtain total document counts for each journal. The totals were used to identify the percentage of documents from each journal that were errata.

5. Journal Citation Reports was examined to determine each journal’s impact factor and its highest listed rank. Some journals were ranked in multiple fields, the highest ranking from these was recorded.

6. The current publisher for each journal was identified and recorded.

7. I assigned broad discipline (e.g., medicine, physics) based upon the journal’s content coverage.

Results

After the source type was limited to journal (Step 1 in methodology), the document types breakdown as shown in Figure 1.

Figure 1: All document types found in journals indexed in Scopus 2012-22

By far, most items in journal articles are unsurprisingly of the Article document type, followed by Review, Note, Editorial, and so on. Erratum is in seventh place, with only 0.92% of the titles being of this type. There was a total of over 229,000 errata for this time period for all journals (compared to over 20 million articles). So overall, most journals should not have more than about 1% of their content, on average, being of this type. So does this reflect reality and how far beyond this number are some journals?

Once the Erratum document type was selected and results refined (Step 2 in Methodology), the 99 journals with the most errata were examined closer. I then took these titles through the steps in the Methodology section.

In terms of disciplines, most titles were in the sciences. This is at least partially due to both the high number of science journals and Scopus’s strength in this area. Then, the 99 titles with the highest number of errata were examined closer. As seen in Figure 2, Medicine by far outpaced all disciplines for these 99 titles, followed distantly by biology, chemistry, and physics.

Figure 2: Top 99 Errata Journals: Number of Titles Per Discipline

A direct comparison to overall distribution would not be perfect as Scopus assigns some titles to multiple subjects. However, examining raw numbers using the Source area in Scopus, medicine titles comprise 15,421 of the 40,878 journals indexed in Scopus . So medicine titles being the most common subject area of the 99 makes sense as over a third of the journals indexed in Scopus cover medicine topics.

Looking at individual titles, 20 journals out of these 99 have over 5% of their indexed titles being errata. Four titles have over 10% of their indexed titles be errata. Table 1 shares the ten journals from this group with the highest percentage of indexed errata. As can be seen, most of these titles were in medicine.

Table 2 includes publishers with higher errata percentages across all of their journals from these 99 titles (publishers with just one title in the list were excluded). As can be seen, this list includes a mix of publisher types, with commercial publishers not dominating. Commercial, university presses, and society publishers are all represented.

Some may wonder if the quality of the journals may be a part of the equation. Impact factors were used as a proxy for journal quality due to being well known and easily discoverable. This of course comes with flaws, as there are many potential metrics that can speak to journal quality in diverse ways. Impact factor focuses on how well a journal is cited, but does not take into account disciplinary differences in the way in which the literature is used. Comparing impact factors thus can be spotty for some disciplines or inappropriate to compare items from different disciplines.

As seen in Figure 3, the 99 examined journals often had high impact factors and ranking. In fact, several journals had the highest ranking in at least one area. So many of these journals are of top quality per these criteria. (See Figure 3 located on the next page.)

So finally, what are the errata rates for the highest impact journals from this group? Table 3 listed the examined titles with the ten highest impact factors (again, with the caveat that impact factors from different disciplines is not a great comparison). This list has titles that many will readily recognize. (See Table 3 located on the next page.)

Takeaways

In summary, examining the data for these 99 journals with the most errata published 2012-2021 revealed:

• Many reputable journals have a higher percentage of errata than the overall journal errata rate.

• Publishers of all types are publishing errata.

• Medicine has a lot of journals with higher errata rates, but also medicine has a lot of journals.

Table 1: Ten Journals Highest Errata Percentage

Table 2: Publisher Errata Percentages

Figure 3: Impact & Best Rank for the 99 Errata Journals

Below are some questions to ponder that either I could not explore with the existing data or due to time limits. Some may not even have straightforward answers.

• Why are these errors happening?

• Are there differences between the errata? For example, Scopus puts all errata together and does not distinguish between types in the document types (e.g., corrigendum not separate).

• Why do some journals have less errata?

— Are other journals doing a better job at catching errors prior to publication?

— Do they have errors that get published and never issue a correction?

— Are they correcting errors in the published version without notifying of a correction?

— Are they just getting submissions without errors in the first place?

— Is nobody noticing errors when they do occur?

— Do they have errata, but they do not get indexed in databases?

— Do the way disciplines handle errors vary?

• Could the editorial or peer review process do anything to lower the rate of errors?

• Are there journals with even larger percentage of errata (but with lower overall errata counts)?

• How does this compare to books and conference papers?

• Is labeling a problem that affects the data?

— Did Scopus mislabel items as errata? Mislabel errata as other types?

— How do publishers present errors and corrections on their own platforms?

° How does that affect the visibility of errata in indexing?

— Do publishers do a bad job of labeling errata when publishing them?

° Does how these are published affect indexing?

• Are publishers handling errata differently than they previously handled them?

— Are some correcting without making a notation?

— Are some not putting errata as a separate “publication” as it was traditionally done in print titles?

• How (well) are other databases handling errata?

• Why does medicine seem to dominate the errata so much? Is it simply due to the number of journals?

On the last point, it’s likely due to the number of medical titles. Some may point out correcting medical journals errors is vital to prevent harm to patients. But errors in physics, engineering, or chemistry can also have quite catastrophic potential. Regardless, the ideal is to always correct errors.

Next time, I will be sharing content related to the next level of “problems” facing journals — retractions.

Table 3: Ten Highest Impact Factors for Examined Journals