Massive covidization of research citations and the citation elite

From January 1, 2020 until August 1, 2021, a total of 5,728,015 items were published and indexed in Scopus (7), including 210,183 (4%) items related to COVID-19. The number of total citations that they received until August 1, 2021 was 9,174,336, of which 1,832,477 citations (20%) were to the published items related to COVID-19. Therefore, even though COVID-19 items were a minority, they accounted for a five-times larger share of the citations received to very recently published items.

Table 1 shows the 36 scientific disciplines (of a total of 174 fields across all science) where more than 30% of citations received in 2020 to 2021 to work published in these 2 y were to COVID-19 work. For three scientific fields, more than two-thirds of the citations received in 2020 to 2021 were for COVID-19–related work: General and Internal Medicine 79.3%, Virology 76.7%, and Emergency and Critical Care Medicine 66.8%. Stated differently, less than one-third of citations in these fields during 2020 to 2021 referenced non–COVID-19 literature, including literature from all other diseases.

Across all disciplines, 98 of the 100 most-cited publications published in 2020 to 2021 were COVID-19 related. Similarly, 97 of the 100 most-cited publications published in 2020 were COVID-19 related. The proportion declined to 76 of the 100 most-cited among publications published in 2021.

As shown in Fig. 1, the proportion of papers receiving very high numbers of citations by August 1 of the next calendar year increased only slightly between 2017 and 2019. However, papers published in 2020 had a major shift, with much larger proportions of papers receiving very high numbers of citations. The shift was entirely attributable to COVID-19–related publications. Of the 3,183,277 publications in 2020, the 96,351 COVID-19–related publications received 8.4-fold more citations than the non–COVID-19 publications. The fold difference was 20.9-fold for General and Internal Medicine; that is, on average a COVID-19–related paper received more than 20 times the number of citations received by a non–COVID-19 paper. The average citations per paper were higher for COVID-19 papers than for non–COVID-19 papers for 128 of the 129 scientific disciplines that published more than 50 COVID-19–related papers in 2020 (with the exception of Computational Theory and Mathematics) (SI Appendix, Table S1).

A total of 84,757 scientists had received ≥100 citations to their work published in 2020 to 2021 by August 1, 2021 (among a total of 4,183,909 Scopus IDs that had published at least one paper in that time period and five or more papers in their entire career). Among these 84,757 scientists, 35,358 had received ≥300 citations, 5,773 had received ≥1,000 citations, 240 had received ≥5,000 citations, and 110 had received ≥10,000 citations for such very recent work.

Of the 84,757 scientists with ≥100 citations to very recent work, 53% had published at least some COVID-19 papers and of the 5,773 scientists with ≥1,000 citations to very recent work, 65% had published some COVID-19 work. Table 2 shows the number of scientists who had received high numbers of citations to very recent work overall, COVID-19–related work, and non–COVID-19 work. As shown, the number of authors who received ≥100 citations to very recent work was almost double for non–COVID-19 work than for COVID-19 work, but the difference was eliminated at the ≥1,000 citations threshold. Greater than or equal to 5,000 citations were received only for COVID-19 work, with the exceptions of six scientists (three authors of an annual cancer statistics reference and three authors of cardiology guidelines). At the ≥10,000 citation threshold, all 110 scientists conducted COVID-19 work; only 15 of these scientists had also received ≥100 citations for their very recent non–COVID-19 work.

Almost all of the scientists who had received >10,000 citations to their very recent work were from China, because the eight most-cited papers of 2020 to 2021 were all papers from China published in the early days of the pandemic and describing clinical characteristics and preliminary epidemiological features of COVID-19.

Among the 84,757 scientists with ≥100 citations to their very recent work (published in 2020 to 2021), for n = 11,767 scientists the citations to their COVID-19 work already accounted for more than half of their total career citation count. Correspondingly, for n = 5,071 of the 84,757 scientists the citations to their non–COVID-19 work published in 2020 to 2021 already accounted for more than half of their total career citation count.

Using a composite citation indicator for ranking the citation impact of scientists, among the top-300 ranked scientists for their COVID-19 work, 117 were among the top-100,000 ranked science-wide for their entire career impact as of August 1, 2021, and 54 were among the top-20,000 ranked science-wide for their career impact. Fig. 2 shows the trajectory for the ranking of these 54 scientists across science according to the composite indicator, considering the citations received in a single year. As shown, in 2019 versus 2017, improvements in ranking were as common as worsening ranking: 13 of 54 scientists improved their ranking by a third or more and 10 of 54 worsened their ranking by as much. Conversely, in the 2020 versus 2019 comparison, 47 of 54 scientists improved their ranking by a third or more, while no scientists worsened their ranking by this margin. Six scientists improved their ranking more than sixfold. These six scientists worked largely independently, and the extremely highly cited papers on COVID-19 that they authored were typically different, except for one paper that introduced the RT-PCR for diagnosis and where two of them were among the authors. Three of them were virologists and made major contributions in identifying the virus and its receptors and developing testing. One was a respiratory medicine specialist who coauthored early papers on COVID-19 clinical features. Another one was a clinician who published a highly cited single-authored viewpoint on African Americans and COVID-19. The sixth scientist was an immunologist who wrote a highly cited single-authored early review. The most impressive improvement was for a long-time coronavirus virologist who went from rank 48,045 in 2019 to rank 362 in 2020, a 133-fold improvement.

Among the top-100 ranked scientists according to the composite citation indicator for their work published in 2018 to 2019, 48 had not been among the top-1,000 ranked for their work published in 2016 to 2017. Among the top-100 ranked scientists according to the composite citation indicator for their work published in 2020 to 2021, 70 had not been among the top-1,000 ranked for their work published in 2018 to 2019. Among the top-100 ranked scientists across science in 2018 to 2019, only 36 focused on Health Sciences subfields and only 14 of the 36 had risen to such extremely high ranks even though they did not belong to the top-1,000 ranked in 2016 to 2017. Conversely, among the top-100 ranked scientists across science in 2020 to 2021, 70 focused on Health Sciences subfields and most (57 of 70) had risen to such extremely high ranks even though they did not belong to the top-1,000 ranked in 2018 to 2019. Twelve of the 70 were editors or journal staff who published profusely in their journals, mostly on COVID-19. These 12 authors were all highly qualified in editor roles and/or journalism, but none of them were COVID-19 specialists themselves, with their own research agenda on COVID-19 investigation and, more generally, none were active, practicing researchers.

Across the 84,757 scientists, the performance for the recent work in 2020 to 2021 correlated strongly with the respective performance during their entire career, for the number of papers (r = 0.69, ρ = 0.79), the ranking according to the composite citation indicator (r = 0.55, ρ = 0.56), the h-index (r = 0.56, ρ = 0.59), and the hm-index (r = 0.59, ρ = 0.76), but not for the number of citations (r = 0.16, ρ = 0.31). The number of citations to recent non–COVID-19 work correlated strongly with the number of citations for the entire career (r = 0.46, ρ = 0.54), but the number of citations to very recent COVID-19 work did not correlate with the number of citations to the entire career (r = −0.03, ρ = −0.29).

The lack of correlation between performance metrics on COVID-19 work and performance for the entire career was seen also for all other metrics besides citations. Fig. 3 shows the strong relationship between the hm-index for entire career and the hm-index for very recent work overall, but weak relationship between the hm-index for entire career and the hm-index for COVID-19 work.

We used a copy of the Scopus database (7) extracted on August 1, 2021. We identified all publications published and indexed in 2020 and 2021 as of that date. We then separated COVID-19 publications from all publications. Similar to previous work (2), COVID-19–related publications were retrieved by searching in articles published in 2019 or later for any of the following terms in the title, abstract, or key words: ‘sars-cov-2’ OR ‘coronavirus 2’ OR ‘corona virus 2’ OR covid-19 OR {novel coronavirus} OR {novel corona virus} OR ‘2019-ncov’ OR ‘covid’ OR ‘covid19’ OR ‘ncovid-19’ OR ‘coronavirus disease 2019’ OR ‘corona virus disease 2019’ OR ‘corona-19’ OR ‘SARS-nCoV’ OR ‘ncov-2019’, with further filtering through the Elsevier International Center for the Study of Research Lab infrastructure to limit to publications indexed (loaded) in Scopus in 2020 or 2021 only, and with the publication year of 2020 or later.

Scopus citations to all publications in 2020 to 2021, to COVID-19 publications in 2020 to 2021, and to non–COVID-19 publications in 2020 to 2021 were counted as of August 1, 2021. Preprint publications from ArXiv, SSRN, BioRxiv, ChemRxiv, and medRxiv account for 9% of COVID-19 publications (2). Citations from or to preprints are not systematically recorded in Scopus, thus preprints are excluded from all citation analyses.

Publications in 2020 to 2021 were assigned to a discipline based on their journal of publication and according to the Science Metrix classification of science, which is a standard mapping of all science into 21 main fields and 174 subfield disciplines (33, 34). For each of the 174 subfields, we estimated the share of citations received in 2020 to 2021 by COVID-19 publications published in 2020 to 2021 against all publications in the same time frame.

We also examined specifically the top-100 most-cited publications published in 2020 to 2021, in 2020 alone, and in 2021 to identify how many of them were COVID-19–related. We used citation counts in Scopus as of August 1, 2021.

We also generated plots of the proportion of publications receiving different numbers of citations for papers published in 2017, 2018, 2019, and 2020 considering citations received until August 1 of the following calendar year. For 2020, we considered separately COVID-19–related and non–COVID-19 publications. For each of the scientific fields that had published at least 50 COVID-19 publications in 2020, we assessed the ratio of mean citations to COVID-19 publications over mean citations to non–COVID-19 publications. This analysis is slightly biased in favor of non–COVID-19 publications, since very few COVID-19 publications appeared in the first 2 mo of 2020 and thus non–COVID-19 publications had slightly more time available to be cited on average.

We identified all authors who had received by August 1, 2021 at least 100 citations to their work published in 2020 to 2021. We noted how many of them had published at least one COVID-19–related publication. We also noted how many authors had received by August 1, 2021 at least 100 citations to their COVID-19 versus non–COVID-19 work published in 2020 to 2021. Numbers of authors passing higher citation thresholds (≥500, ≥1,000, ≥5,000, ≥10,000) for these categories were also noted. We examined the country of the authors at the highest citation levels. For all analyses of authors, similar to prior work (2), we only considered those that have published at least five papers (articles, conference papers, or reviews) in their career. This allows the exclusion of authors with limited presence in the scientific literature and of author IDs that may represent split fragments of the publication record of some more prolific authors.

For each author with ≥100 citations to their work published in 2020 to 2021, we also recorded the number of published items in 2020 to 2021 (publications including preprints, citations to preprints are not recorded), and additional citation indices limited to the impact of the 2020 to 2021 published work, the Hirsch h-index (35), and the coauthorship adjusted Schreiber hm-index (36). Using a previously published and validated composite citation indicator (37–39; see URLs for the individual versions in ref. 39) that combines total citations, h-index, hm-index, and three indicators of citations to works as single, single/first, single/first/last author, we generated a ranking of scientists based on their 2020 to 2021 work alone. We did the same calculations and generated the respective and rankings limited to COVID-19 work published in 2020 to 2021.

For each author with ≥100 citations to their work published in 2020 to 2021, we also calculated the same citation metrics and overall ranking across all science as of August 1, 2021 for the work published during their entire career (39). We evaluated for how many authors their COVID-19 work accounted for at least half of the citations they had received in their entire career; and for how many authors their non–COVID-19 work published in 2020 to 2021 accounted for more than half of the citations they had received in their entire career.

We had previously generated (2) a list of the top-300 ranked scientists for their COVID-19 work based on the composite citation indicator. We investigated how many of those were also among the top-100,000 ranked science-wide for their entire career impact as of August 1, 2021 and how many were among the top-20,000 ranked science-wide for their career impact. For the scientists who were among the top-20,000 ranked science-wide for their career impact, we also noted their science-wide ranking for their annual citation impact in the single years 2017, 2019, and 2020; data were extracted from previously published, publicly available datasets that use the composite citation indicator for the ranking (38, 39). This allowed us to assess the evolution of the trajectory of the ranking of these scientists before the pandemic and during the pandemic. The annual assessments consider all the citations received in a single year to all work published in the scientist’s career. Therefore, they reflect the recent attention not only to the recent work, but also to all past work.

For each of three periods 2016 to 2017, 2018 to 2019, and 2020 to 2021, we calculated the composite citation indicator for all scientists in Scopus for their work published in the respective years. We excluded books and book chapters because Scopus assigns to the latest edition of books also the citations to previous editions from past years. We evaluated how many scientists rose to the top-100 ranked according to the composite citation indicator while they were not even among the top-1,000 ranked in the previous 2-y period. We also report such major rises specifically for scientists whose Science Metrix classification of their primary and/or secondary subfield was in the 60 subfields of Health Sciences.

Finally, we calculated Pearson and Spearman correlation coefficients for the productivity and citation metrics of the scientists for their entire career as of August 1, 2021 and the respective metrics for COVID-19 work and non–COVID-19 work published in 2020 to 2021. This allowed us to evaluate whether the career impact tracked with their recent COVID-19 work, non–COVID-19 work, or both.

All calculations throughout the paper include self-citations. No statistical tests were used and no P values are reported, since analyses are descriptive.