Patterns of text reuse in a scientific corpus

Daniel T. Citron; Paul Ginsparg

doi:10.1073/pnas.1415135111

Edited* by William H. Press, University of Texas at Austin, Austin, TX, and approved November 6, 2014 (received for review August 7, 2014)

Significance

In the modern electronic format it is both easier to reuse text and easier to detect reused text. This is the first comprehensive study of patterns of text reuse within the full texts of an important large scientific corpus, covering a 20-y timeframe. It provides an important baseline for what is regarded as standard practice within the affected research communities, a standard somewhat more lenient than that currently applied to journalists, popular authors, and public figures.

Abstract

We consider the incidence of text “reuse” by researchers via a systematic pairwise comparison of the text content of all articles deposited to arXiv.org from 1991 to 2012. We measure the global frequencies of three classes of text reuse and measure how chronic text reuse is distributed among authors in the dataset. We infer a baseline for accepted practice, perhaps surprisingly permissive compared with other societal contexts, and a clearly delineated set of aberrant authors. We find a negative correlation between the amount of reused text in an article and its influence, as measured by subsequent citations. Finally, we consider the distribution of countries of origin of articles containing large amounts of reused text.

Footnotes

↵¹To whom correspondence should be addressed. Email: ginsparg{at}cornell.edu.

Author contributions: P.G. designed research; D.T.C. and P.G. performed research; D.T.C. and P.G. analyzed data; and D.T.C. and P.G. wrote the paper.
The authors declare no conflict of interest.
↵*This Direct Submission article had a prearranged editor.
^†Commercial resources, such as Ithenticate, use a much larger dataset. See in particular CrossCheck (13), implementing Ithenticate for research publications, and used by member publishers to screen journal submissions (14, 15). That coverage is still far from as comprehensive as that available via commercial search engines, as assessed by comparing to results from the Google custom search API.
^‡The number of article pairs with at least 10 or more 7-grams in common is of order 600,000, about 2 per million of the total possible (757,000)2/2≈278 billion total article pairs.
^§Recall that the vast majority of arXiv submissions appear in the conventional peer-reviewed literature, with the primary exceptions being theses, conference proceedings, lectures, and other “review-type” materials discussed earlier (and excluded from subsequent analysis).
^¶Review articles pose an additional challenge, because standard software used to include pdf figures from other articles sometimes carries along ”hidden” text surrounding the figure from its original context, invisible to the author and reader in the new context but nonetheless seen by the pdf-to-text converter and flagged as a large text overlap.
^‖This happened historically when users inadvertently created a submission with a new identifier rather than using the replace function to create a new version of an existing submission, with the same identifier. This problem has been largely eliminated by the daily overlap screening, with submitters now instructed to replace an existing submission if excessive overlap is detected.
^††As discussed earlier, there is no systematic scan for text copied from sources outside of arXiv, and no attempt to detect “plagiarism” as more generally defined, as unattributed use of ideas independent of copied text. The exceptions described earlier for review articles, theses, conference proceedings, book contributions, multipart articles, and so on, are respected, so that common-authored overlaps are not flagged in cases that seem to be accepted as common practice.
^‡‡After the completion of this work, we discovered significantly higher rates of text reuse specifically in computer science articles published in predatory open-access journals (articles largely received after the mid-2012 timeframe of the dataset analyzed here). We defer to any later work a more discipline-specific assessment of the issues.
This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.1073/pnas.1415135111/-/DCSupplemental.