More haste, less science?

Sir

One widely recognized trend in scientific research is the development of the ‘publish or perish’ academic environment. A second trend is the increasing complexity and technical sophistication of science. Third, the amount of science that is being published is increasing rapidly, and scientists complain about the difficulty of keeping abreast of the literature. One consequence of these trends has not, to my knowledge, been adequately addressed: namely, how has the accuracy of the literature been affected by these pressures?

I sampled a leading international journal and quantified the number of original articles that contained at least one error. I read correspondence, errata and technical articles in one issue per month (usually the first week's issue), and recorded all published errors by authors or editors. I did not count disputes between authors and correspondents unless authors admitted that critics had identified a legitimate mistake.

I classified errors as: ‘trivial’ (typographical, grammatical or production errors of little or no impact on interpretation of the paper); ‘errors of scholarship’ (failures to acknowledge priority, mistaken claims of novelty, misuse of terminology, and improper or missing citations); and ‘technical errors’ (author- and journal-generated errors that could cause confusion, such as errors of fact or omission, statistical errors, chemicals misidentified, errors in mathematical equations or DNA sequences, figure legends transposed, use of improper methodology, data misinterpreted, and, in extreme cases, the retraction of parts or all of a paper).

I also attempted to distinguish whether the error originated with the authors or with the journal, although this was possible in only 81% of the papers. Finally, I counted the total number of articles and reports in each issue of the journal to estimate error rates (total number of papers containing errors per year/total number of papers that year). I sampled from 1960 (when professorial life was more leisurely, according to my old PhD supervisor) to 1997. Because I depended on authors, correspondents or editorial staff to identify errors in papers, my estimates of error rates are almost certainly conservative.

I found 183 papers with errors: 56 (31%) trivial, 35 (19%) errors of scholarship, and 92 (50%) technical errors. The error rate has been increasing since the early 1960s, when there were few reported errors (Fig. 1). By the 1970s, errors began to appear regularly, almost all of which were, as far as I could judge, the authors' responsibility. But by the 1980s and 1990s, both authors and the journal have contributed to the error rate, although authors remain responsible for most.

Figure 1: The yearly error rate in articles and reports in a major scientific journal from 1960 to 1997.

Based on a sample of 12 issues per annum. a, Total rate and the rates judged to be due either to authors or to the journal. b-d, Errors broken down by type and severity.

Although a third of errors can be considered trivial, the number of such errors is increasing rapidly, suggesting that production standards are becoming more difficult to maintain. There is no strong temporal trend in errors of scholarship, at least since the 1970s, suggesting that authors have not yet lost track of the literature. By contrast, technical errors have been increasing.

What factors are likely to be contributing to this state of affairs? First, authors may be less careful during manuscript preparation owing to increasing career pressures to publish. Second, reviewers and editors may also be less thorough. Third, the increasing complexity of science may mean that more errors escape notice until papers are published. Fourth, journals may increasingly be contributing to errors in the literature. An alternative line of reasoning is to assume that the innate error rate has not changed but that we are more likely to criticize others' work than in the past, or that authors are generally more willing to accept criticism than in the past.

I believe that the observed increase is real, and that the academic environment is generating sources of error that are getting more severe. I doubt that this problem will be solved unless there are significant changes in the criteria by which many of us are evaluated by our universities. We might also ask ourselves about the consequences for scientific progress of the apparent increase in the publication error rate.