Abstract
COVID-19-related research redistributed citations toward pandemic topics, but the author-level contribution of this redistribution to individual h-index values remains insufficiently documented in lower-volume national research systems. This cross-sectional bibliometric study used OpenAlex metadata from 2015–2025 to evaluate highly productive Peruvian health researchers. We assembled a non-random upper-productivity cohort of 500 authors; the primary cohort included 456 authors with fractional Peruvian affiliation of 0.5 or higher. Observed h-index values were compared with COVID-19-excluded h-index values recalculated after removing COVID-19-related works identified through an audited classifier. In the primary cohort, the median observed h-index was 12.0, the median COVID-19-excluded h-index was 11.0, and median Delta h was 0.00 [0.00, 1.25]. Overall, 210 authors (46.1%) had Delta h > 0; the observed total Delta h after COVID-19 removal was 450. A matched non-COVID removal benchmark produced a lower median simulated total Delta h of 297 [292, 303], and none of the 500 simulations reached the observed COVID-removal total. The proportional h-index change was larger among authors with lower baseline h-index, although this indicator is denominator-sensitive. COVID-19-related works had higher two-year citation velocity than non-COVID-19 works within the same authors. These findings support a cautious interpretation of h-index values from the pandemic period according to the thematic and temporal composition of the works contributing to the index.
1. Introduction
The evaluation of researchers and institutions increasingly relies on bibliometric indicators derived from citation counts. Among these, the h-index has become one of the most widely used measures for summarizing an author’s output and citation impact in a single number (Hirsch, 2005). Because of its widespread adoption and apparently simple calculation, the h-index has also been critically examined with respect to its properties, limitations, and use (Bornmann & Daniel, 2007; Koltun & Hafner, 2021). Its application extends beyond academia, informing decisions on funding, hiring, promotion, and researcher classification; consequently, systematic changes in the works contributing to the index can affect the allocation of resources and recognition within scientific systems.
The COVID-19 pandemic produced an unprecedented increase in scientific publishing (Else, 2020) and rapid growth in the number of researchers contributing to this literature (Ioannidis et al., 2021). Globally, COVID-19 articles received a share of citations far exceeding what would be expected from their publication volume: they represented approximately 4% of articles published in 2020 and 2021, but nearly 20% of citations received, and 98 of the 100 most-cited papers from that biennium addressed COVID-19 (Ioannidis et al., 2022). This thematic concentration of citations placed numerous researchers in citation-elite positions and made COVID-19-related works substantially more likely to rank among highly cited papers than the rest of the literature (Ioannidis et al., 2022; Liu et al., 2023).
Latin America participated in this dynamic through accelerated growth in COVID-19-related output (Gregorio-Chaviano et al., 2020), and Peru also showed a marked increase in national research on the disease after 2020 (Vasquez-Uriarte et al., 2021; Estrada, 2022). This context is relevant for researcher evaluation systems that use citation indicators. In lower-volume national research systems, a small number of highly cited papers can modify individual indicators proportionally more than in systems with larger baseline citation volume. However, that possibility requires direct author-level measurement rather than broad inference from global citation patterns.
Available evidence on COVID-19 citation concentration comes mainly from global analyses focused on the citation elite and using data through the early pandemic period (Ioannidis et al., 2022; Liu et al., 2023). These studies describe the rise in indicators, but not their author-level composition in a defined national health-research cohort. Specifically, no prior study has quantified how much the h-index of highly productive Peruvian health researchers changes after COVID-19-related works are removed, nor whether the relative change differs according to pre-pandemic citation trajectory. Likewise, within the same author, the early citation velocity of COVID-19-related output has not been compared with that of non-COVID-19 output.
The contribution of this study is threefold. First, it provides an author-level recalculation of observed and COVID-19-excluded h-index values in a defined cohort of highly productive Peruvian health researchers. Second, it evaluates whether relative h-index change differs according to pre-pandemic citation trajectory, operationalized by baseline h-index. Third, it implements an OpenAlex-based workflow that can be inspected and adapted to other national or disciplinary settings. The study does not estimate whether COVID-19 citations were deserved or undeserved, nor does it compare Peru directly with other research systems.
Therefore, this study aimed to quantify the observed h-index change after removing COVID-19-related output among highly productive Peruvian health researchers, analyze whether this relative change is greater among those with a lower baseline h-index, and compare within-author two-year citation velocity for COVID-19-related versus non-COVID-19 output.
2. Materials and Methods
2.1. Study Design
We conducted an observational bibliometric study with an analytical cross-sectional author-level design, using publicly available secondary metadata. The unit of analysis was the author and, nested within authors, their scientific works. The publication period considered spanned 2015 to 2025. The manuscript was reported using STROBE as the primary framework for the observational author-level design (von Elm et al., 2007). Because the data source and outcomes are bibliometric, the BIBLIO checklist was used as a complementary reporting aid for bibliometric-specific items; the completed checklist is provided as Table S4.
2.2. Data Source and OpenAlex Fields
Data were obtained from OpenAlex, a fully open scientific knowledge graph that compiles metadata on works, authors, institutions, and fields of study, and operates as a replacement for the discontinued Microsoft Academic Graph (Priem et al., 2022). OpenAlex provides author disambiguation, institutional linkage and ORCID identifiers, automated field classification, and citation counts for each work, accessible through its application programming interface (API). Data were extracted from the OpenAlex API in June 2026; because citation counts correspond to a specific date, all citation and h-index figures refer to that extraction point.
The OpenAlex fields used for variable construction included work identifier (works.id), publication year (works.publication_year), document type (works.type), title (works.title), abstract reconstructed from works.abstract_inverted_index when available, authorship information (works.authorships), author identifier (works.authorships.author.id), ORCID (works.authorships.author.orcid), institutional country (works.authorships.institutions.country_code), work-level citation counts (works.cited_by_count), and OpenAlex topic, field, or subfield labels used to identify health-related works and dominant author fields. The query scripts, extraction metadata, and field mappings are provided in Supplementary Data Package S1 (Complete reproducibility package).
2.3. Population and Sample
The operational universe was defined by querying works indexed in OpenAlex between 2015 and 2025 that had at least one institution with country_code = “PE” and a health-related primary field; this query retrieved 37,252 works. From this universe, authors with evidence of Peruvian affiliation in their own authorship records within those works were retained. An authorship record was defined as one OpenAlex authorships entry linking one author profile to one work. The extraction identified approximately 49,967 candidate authors with Peruvian authorship records. A non-random upper-productivity cohort was constructed comprising the 500 authors with the largest number of health-related works with Peruvian affiliation in this universe, ordered in descending order by that count; ties were resolved using the OpenAlex author identifier, and extraction was performed in batches of 50. This cohort is neither a census of Peruvian health authors nor a random sample, but rather an upper-productivity segment within the defined operational universe.
Author selection and h-index calculation were performed on two different analytical corpora. The selection corpus consisted of OpenAlex works published from 2015 to 2025 with at least one Peruvian institution and a health-related primary field. This corpus was used only to identify and rank candidate authors. After the author cohort was defined, h-index variables were calculated at the OpenAlex author-profile level using all works linked to each selected OpenAlex author profile and available at the extraction date, regardless of country affiliation or field. The baseline h-index was calculated from the subset of those author-profile works published between 2015 and 2019. The COVID-19-excluded h-index was recalculated from the same author-profile corpus after removing works classified as COVID-19-related.
For each author, fractional Peruvian affiliation (pe_share) was calculated as the number of authorship records in which the focal author had at least one institution with country_code = “PE” divided by the total number of authorship records linked to that OpenAlex author profile in the retrieved author-profile corpus. Authorship records with both Peruvian and non-Peruvian institutions were counted as Peruvian in the numerator. Records without institutional country information were retained in the denominator but were not counted as Peruvian. The primary analysis cohort included authors with pe_share ≥ 0.5; this threshold retained authors for whom Peruvian affiliation represented at least half of their OpenAlex authorship records and reduced inclusion of internationally based collaborators with episodic Peruvian participation. Additional sensitivity cohorts used pe_share ≥ 0.3 and an unfiltered 500-author reference cohort. Authors excluded by the primary threshold are shown in Table S5.
Author disambiguation relied primarily on OpenAlex author identifiers and available ORCID links. All 500 selected profiles were screened for risk signals, and 301 profiles were flagged for review based on one or more of the following indicators: missing or inconsistent ORCID information, abrupt changes in country or institutional affiliation, implausible field switching, duplicate or near-duplicate names within the candidate list, unusually large gaps between active publication years, or publication/citation patterns suggesting merged or fragmented profiles. For flagged profiles, names, ORCID records when available, affiliation histories, coauthor patterns, and topic continuity were reviewed. Supplementary Data Package S1 (complete reproducibility package) reports the operational flags and author-level audit variables used in this process. The objective author-level risk indicators are summarized in Supplementary Table S7.
2.4. Variables
Each author’s observed h-index was calculated using all works available in the OpenAlex author profile at the extraction date, according to the original definition of the indicator (Hirsch, 2005). The baseline h-index was calculated analogously, restricted to works published between 2015 and 2019. The COVID-19-excluded h-index was obtained by recalculating the h-index after removing works classified as COVID-19-related. Delta h was defined as the difference between the observed h-index and the COVID-19-excluded h-index. The proportional h-index change was defined as Delta h divided by the observed h-index. Because this proportional indicator is mechanically sensitive to the observed h-index denominator, absolute Delta h was treated as the primary descriptive measure, and the proportional indicator was used as a secondary heuristic measure of relative h-index change. It is not an additive decomposition of the h-index and not a causal estimate.
COVID-19-related works were identified using an explicit dictionary applied to titles, reconstructed abstracts, and OpenAlex keyword and topic fields when available. The dictionary included the exact regular-expression terms COVID-19, COVID19, SARS-CoV-2, 2019-nCoV, coronavirus disease 2019, and pandemia por COVID. Missing abstracts or keywords were treated as empty fields rather than as exclusion criteria; titles and available topic fields were still evaluated. Because OpenAlex topic and keyword labels can be retrospective, a pre-2020 audit was performed: works published before 2020 were excluded from the COVID-19-related set unless an exact COVID-19 or SARS-CoV-2 term appeared in the title. This audit removed 378 of 381 pre-2020 flagged unique works, and none of the three retained explicit-title matches belonged to the 2015–2025 author-linked population subset. The complete classifier dictionary, audit rule, and sensitivity schemes are provided in Supplementary Methods S1 and Tables S1–S3.
The number of previous works corresponded to the author’s work count between 2015 and 2019. The primary field of study was taken from the OpenAlex classification. The citation velocity of each work was defined as the number of citations accumulated during the two complete calendar years after the year of publication; this fixed window was adopted to make works from different annual cohorts comparable and avoid distortion by differences in available citation accrual time. Works published in 2024 or 2025 were retained for h-index analyses but excluded from the citation-velocity analysis because two complete post-publication calendar years were not available at the June 2026 extraction date. For analysis, subgroup A consisted of authors with three or more previous works, and subgroup B consisted of authors with at least one COVID-19 work and at least one non-COVID-19 work published during 2020 and 2021. To describe the relative contribution gradient, baseline h-index was categorized into quartiles.
2.5. Statistical Analysis
The descriptive analysis summarized continuous variables using medians and interquartile ranges, and categorical variables using frequencies and percentages, by cohort. The statistical methods, analysis populations, outcomes, and interpretations are mapped explicitly in Supplementary Table S8.
The association between baseline h-index and the proportional h-index change was analyzed using Spearman’s correlation coefficient and a fractional logistic regression model estimated by quasi-likelihood with HC1 robust standard errors (Papke & Wooldridge, 1996). The model was adjusted for the logarithm of the number of previous works and field of study. Because the proportional outcome combines the occurrence of any h-index change with the magnitude of relative change and is sensitive to denominator effects, a complementary two-process analysis was added. The first process was modeled using logistic regression for any h-index change (Delta h > 0). The second process summarized the magnitude of Delta h among affected authors only. Absolute Delta h and the binary indicator Delta h > 0 were prioritized descriptively.
Citation velocity for COVID-19 and non-COVID-19 works was compared at the document level using medians, means, and the ratio of means, and through a paired within-author comparison. For the paired analysis, work-level two-year citation velocities were first aggregated within each author and thematic category using the median citation velocity of COVID-19-related works and the median citation velocity of non-COVID-19 works. The paired author-level difference was then calculated as COVID-19 minus non-COVID-19 citation velocity and tested using the Wilcoxon signed-rank test; the confidence interval for the median difference was obtained by resampling. The paired within-author design reduces confounding by stable author-level factors such as productivity, seniority, and coauthorship network, although it does not control for work-level characteristics such as document type, journal venue, collaboration size, international collaboration, article format, or topic urgency.
A matched non-COVID removal benchmark was added as a mechanical sensitivity analysis. For each primary-cohort author, the simulation removed the same number of non-COVID works as audited COVID-19 works, matching first on publication year and document type and relaxing the match if needed. Each author was simulated 500 times. The observed total Delta h after removing COVID-19-related works was compared with the distribution of simulated total Delta h after matched non-COVID removals. The main analyses were repeated under the three cohort definitions to assess stability and upper-tail sensitivity. Analyses were performed in Python 3.12.13 using pandas, NumPy, SciPy, statsmodels, and matplotlib; the executable scripts and software-environment documentation are provided in Supplementary Data Package S1 (Complete reproducibility package).
2.6. Ethical Considerations
This study was based exclusively on publicly available secondary bibliographic metadata from OpenAlex. The information used corresponds to work and authorship data made public through scientific publication; it does not contain clinical data or sensitive personal information, and no primary data collection or intervention involving individuals was conducted at any stage. Because this was a secondary analysis of publicly available data, the study did not require review by an institutional ethics committee (Tripathy, 2013). The analysis was conducted in accordance with applicable standards for research using publicly available secondary data.
3. Results
3.1. Cohort Selection and Characteristics
The OpenAlex query identified 37,252 health-related works with Peruvian affiliation from 2015 to 2025 and approximately 49,967 candidate authors with Peruvian authorship records. The 500 most productive authors in this operational universe were extracted. Applying the fractional Peruvian affiliation criterion excluded 44 authors with pe_share < 0.5; the primary cohort therefore comprised 456 authors. Subgroup A included 383 authors with at least three previous works, subgroup B included 331 authors with both COVID-19 and non-COVID-19 work during 2020–2021, their intersection included 294 authors, and the paired citation-velocity sample included 395 authors. The expanded flow diagram is shown in Figure 1, and excluded authors are listed in Table S5.
Figure 1.
Expanded cohort flow with parallel analytical branches. Candidate-author count is the approximate operational production-batch count documented in Supplementary Data Package S1 (complete reproducibility package).
The cohort characteristics according to the three Peruvian-affiliation definitions are summarized in Table 1.
Table 1.
Cohort characteristics according to the definition of Peruvian affiliation.
3.2. Annual COVID-19 Publication and Citation Context in the Author-Linked Work Set
Among the 13,247 unique author-linked population works represented among the extracted top-500 authors, the pre-2020 audit removed retrospective COVID-19 flags before 2020. COVID-19-related works represented 19.4% of works in 2020 and 27.3% in 2021, while accounting for 39.2% and 32.9% of citations to author-linked works in those years, respectively. This table describes the author-linked analytic work set represented among the extracted top-500 authors and should not be interpreted as a complete annual census of all 37,252 Peruvian health-related OpenAlex works. Annual COVID-19 publication and citation concentration in the author-linked work set is shown in Table 2.
Table 2.
Annual COVID-19 publication and citation concentration among author-linked works represented in the top-500 analytic author set.
3.3. Observed H-Index Change After Removing COVID-19-Related Works
In the primary cohort, the median observed h-index was 12.0 and the median COVID-19-excluded h-index was 11.0. Median Delta h was 0.00 (IQR 0.00–1.25), and 210 authors (46.1%) had Delta h > 0; 246 authors (53.9%) had no h-index change. The total observed Delta h after removing COVID-19-related works was 450. The maximum Delta h was 10, 114 authors (25.0%) had Delta h ≥ 2, 63 authors (13.8%) had Delta h ≥ 3, and 18 authors (3.9%) had Delta h ≥ 5. Thus, the absolute h-index change was small for most authors and concentrated in a smaller upper tail. The distribution of absolute h-index change after removing COVID-19-related works is shown in Table 3.
Table 3.
Absolute h-index change after recalculating the h-index without COVID-19-related works, primary cohort (n = 456).
In the matched non-COVID removal benchmark, the observed total Delta h after removing COVID-19-related works was 450. Across 500 simulations removing matched non-COVID-19 works by author, publication year, and document type, the median simulated total Delta h was 297 (IQR 292–303). None of the 500 matched non-COVID removal simulations reached the observed COVID-removal total; equivalently, the empirical Monte Carlo probability was <0.2%. The benchmark indicates that COVID-19-related works had a larger h-index effect than expected from removing the same number of matched non-COVID-19 works, but it remains a mechanical sensitivity analysis rather than a causal estimate.
3.4. Relative H-Index Change and Baseline H-Index
The proportional h-index change was higher among authors with lower baseline h-index. The mean proportional h-index change was 0.157 in the first baseline h-index quartile, 0.088 in the second, 0.071 in the third, and 0.029 in the fourth. Spearman’s correlation between baseline h-index and proportional h-index change was rho = −0.288 (p < 0.001; n = 456). In the fractional logistic regression model adjusted for the logarithm of previous works and field of study, each additional baseline h-index point was associated with lower proportional h-index change (OR = 0.888, 95% CI 0.849–0.930; p < 0.001).
Because the proportional indicator is denominator-sensitive, we added a two-process analysis. Baseline h-index was inversely associated with the occurrence of any h-index change (Delta h > 0; OR = 0.928, 95% CI 0.889–0.968; p = 0.0005), whereas the number of pre-pandemic works was not significantly associated with this outcome. Among affected authors, the median Delta h was 2.0 (IQR 1.0–3.0). These results suggest that the relative gradient should be interpreted jointly with the absolute distribution of Delta h and not as an additive decomposition of the h-index. Proportional h-index change according to baseline h-index quartile is summarized in Table 4. Full regression results and the affected-only magnitude summary are provided in Supplementary Table S9.
Table 4.
Proportional h-index change according to baseline h-index quartile, primary cohort.
3.5. Citation Velocity by Topic
Within the fixed two-year citation window, COVID-19-related works had a median of 1 citation (IQR 0–7) compared with 0 citations (IQR 0–3) for non-COVID-19 works. The respective means were 6.38 and 3.93, with a rate ratio of 1.62. In the paired within-author comparison, restricted to 395 authors with at least one work from each thematic category and a complete two-year citation window, the median author-level difference was +1.0 citation (resampling 95% CI 0.5–1.0; Wilcoxon signed-rank test, p < 0.001). Citation velocity for COVID-19-related and non-COVID-19 works in the fixed two-year window is presented in Table 5.
Table 5.
Citation velocity of COVID-19 and non-COVID-19 works in a fixed two-year window.
3.6. Sensitivity Analyses
Sensitivity analyses showed that the central tendency of Delta h was stable or lower across cohort definitions, while the upper tail was sensitive to the fractional Peruvian affiliation threshold. In the unfiltered 500-author reference cohort, the maximum Delta h was 24 and the five largest Delta h values were 24, 22, 10, 8, and 7. In the primary pe_share ≥ 0.5 cohort, the maximum Delta h was 10 and the five largest values were 10, 8, 7, 6, and 6. Therefore, the affiliation threshold did not materially alter the central tendency but reduced the influence of high-Delta h profiles with lower fractional Peruvian affiliation (Tables S6 and S10). Classifier sensitivity analyses also showed that stricter text-field classifiers yielded lower but directionally consistent estimates (Tables S2 and S3).
4. Discussion
4.1. Main Findings
In this non-random upper-productivity cohort of highly productive Peruvian health researchers, COVID-19-related works produced a modest absolute h-index change for most authors after pre-2020 classifier auditing. Median Delta h was 0.00 [0.00, 1.25], more than half of authors had no h-index change, and the total observed Delta h after COVID-19 removal was 450. The proportional h-index change was larger among authors with lower baseline h-index, but this result must be interpreted with the denominator sensitivity of the proportional indicator in mind. COVID-19-related works also showed higher two-year citation velocity than non-COVID-19 works by the same authors.
4.2. Comparison with Other Studies
The existence of a citation advantage for COVID-19-related work is consistent with international evidence. Globally, these works have been shown to concentrate a share of citations far greater than their publication volume and to place numerous researchers in citation-elite positions (Ioannidis et al., 2022), within a context of rapid and massive expansion of authors and publications (Ioannidis et al., 2021; Fassin, 2021). Similarly, COVID-19-related work has been reported to have a higher probability of ranking among the most cited (Liu et al., 2023). Our findings are more limited and more specific: they quantify author-level h-index change in a defined Peruvian health-research cohort and show that the absolute effect was modest for most authors, even though COVID-19-related works had higher early citation velocity. This interpretation is also consistent with evidence that the rapid growth of COVID-19 publications affected citation indicators and was accompanied by citation polarization during the early pandemic period (Sjögårde, 2022; You et al., 2024).
The inverse association between baseline h-index and proportional h-index change is consistent with known properties of the h-index, whose sensitivity to a small number of highly cited papers is greater when the starting value is low (Bornmann & Daniel, 2007; Waltman & van Eck, 2012). This pattern should not be read as proof that lower-volume or peripheral systems were affected more strongly than consolidated systems, because no comparator countries were included. Instead, the results show that, within this Peruvian cohort, authors with lower baseline h-index had larger relative changes when COVID-19-related works were removed.
The paired within-author comparison, which compares citation velocity for works by the same researcher, reduces confounding by stable author-level characteristics and provides a clearer thematic comparison than an independent-document analysis. However, it does not adjust for work-level differences in document type, journal venue, collaboration size, international collaboration, article format, or topic urgency. The higher within-author citation velocity of COVID-19-related work is therefore compatible with thematic citation concentration described globally (Ioannidis et al., 2022; Liu et al., 2023), but should not be interpreted as a fully causal estimate of topic effect.
4.3. Implications for Research Evaluation and Public Health
These results have implications for researcher evaluation systems that rely on citation indicators. Interpretation of h-index increases during the pandemic should consider the temporal and thematic composition of the works supporting the index, because part of the observed indicator value may reflect concentration of citations in a time-sensitive emergency topic. Principles for responsible use of metrics recommend complementing indicators with expert judgment and measures normalized by field and period (Hicks et al., 2015), a precaution that is particularly relevant for indicators calculated over the pandemic period.
From the perspective of science policy and public health, the concentration of output and citations in a single topic during the pandemic illustrates how a health emergency can abruptly redirect research activity. Understanding the extent to which this redirection translates into durable citation recognition is relevant for designing incentives that sustain priority research lines beyond the immediate emergency context, including endemic and neglected diseases that account for a substantial share of disease burden in settings such as Peru.
4.4. Limitations
This study has limitations. Because of its cross-sectional bibliometric design, the associations described do not establish causal relationships. Delta h is a descriptive recalculation and not a causal estimate of inflation; it does not determine whether citations to COVID-19-related work were artificial, undeserved, or would have occurred under alternative publication conditions. The cohort was purposively constructed from the most productive authors in the operational universe and is not representative of all Peruvian health researchers. In addition, the analysis was restricted to Peruvian health researchers and did not include comparator countries; therefore, the findings should not be interpreted as evidence that peripheral or lower-citation systems were affected more strongly than consolidated research systems.
Several limitations concern interpretation of the recalculated h-index measures. First, the proportional indicator Delta h/observed h-index is sensitive to denominator effects, so larger relative changes among authors with lower h-index values may partly reflect arithmetic scaling. Second, the COVID-19-excluded h-index assumes that citations to non-COVID-19 works were unaffected by the author’s COVID-19-related visibility and therefore cannot capture possible halo effects whereby highly visible COVID-19 papers increased attention to older or unrelated work. Third, Delta h is an integer measure with limited sensitivity at the center of the distribution.
Data-source and classification limitations also remain. The data source was OpenAlex rather than Scopus, the database used by some national evaluation systems; differences in coverage and citation linkage across bibliographic sources may affect counts (Visser et al., 2021), and automated author disambiguation remains imperfect despite screening with ORCID and profile-level audit signals. COVID-19 classification was based on an explicit dictionary and OpenAlex topic fields; because topic and keyword labels can be retrospective, we added a pre-2020 audit and classifier sensitivity analyses, but residual misclassification is still possible. Finally, works published in 2024 and 2025 were excluded from citation-velocity analysis because a complete two-year citation window was not available.
5. Conclusions and Recommendations
In this non-random upper-productivity cohort of Peruvian health researchers, COVID-19-related works produced a modest absolute change in author-level h-index values for most authors, with median Delta h of 0.00 [0.00, 1.25] and no h-index change for 53.9% of the cohort. Relative h-index change was larger among authors with lower baseline h-index, although this pattern partly reflects the denominator sensitivity of proportional indicators. COVID-19-related works also showed higher two-year citation velocity than non-COVID-19 works by the same authors. These findings support the need to interpret h-index values from the pandemic period in relation to the thematic and temporal composition of the works contributing to the index, without assuming that such citations were artificial or undeserved.
Evaluation and classification processes that use the h-index should consider field- and year-normalized indicators together with expert judgment (Hicks et al., 2015; DORA, 2012). Future research should replicate this analysis using Scopus data and complete national researcher population frames, and should use comparative designs to test whether lower-volume research systems experience larger relative bibliometric shifts during health emergencies.
Supplementary Materials
The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/publications14030054/s1: Supplementary Methods S1: COVID-19 classification dictionary, pre-2020 audit rule, and classifier sensitivity; Table S1: COVID-19 classifier dictionary and operational rules; Table S2: COVID-19 classification sensitivity checks; Table S3: Pre-2020 COVID-19 flag audit summary; Table S4: BIBLIO checklist for reporting bibliometric studies of biomedical literature; Table S5: Authors excluded from the primary cohort because of fractional Peruvian affiliation below 0.5; Table S6: Distributional stability of Delta h according to cohort definition; Table S7: Objective author-level risk indicators; Table S8: Analytical questions, samples, variables, methods, and interpretation; Table S9: Regression models and affected-only magnitude summary; Table S10: Upper-tail Delta h sensitivity by cohort definition; Supplementary Data Package S1: Complete reproducibility package.
Author Contributions
Conceptualization, V.J.V.P., J.B.-C. and F.E.Z.-M.; methodology, V.J.V.P., J.B.-C. and F.E.Z.-M.; formal analysis, V.J.V.P.; investigation, all authors; data curation, J.B.-C., F.E.Z.-M., J.C.B.-R. and H.E.D.-T.; visualization, V.J.V.P., J.C.B.-R. and H.E.D.-T.; supervision, V.J.V.P. and C.I.G.D.C.; writing—original draft preparation, V.J.V.P., J.B.-C. and F.E.Z.-M.; writing—review and editing, all authors. All authors have read and agreed to the published version of the manuscript.
Funding
This research received no external funding.
Institutional Review Board Statement
Ethics committee approval was not required because this study analyzed publicly available secondary bibliographic metadata from OpenAlex and did not involve primary data collection, intervention, clinical information, or sensitive personal data.
Informed Consent Statement
Informed consent was not required because the study used publicly available bibliographic metadata and did not collect data from human participants.
Data Availability Statement
The OpenAlex source metadata are publicly available through the OpenAlex API and full data snapshot (https://openalex.org; accessed on 12 July 2026). The curated data and executable code required to reproduce the analyses are provided with this article in Supplementary Data Package S1. The package includes the canonical author-level dataset (authors_master_v2_FINAL.csv), extraction metadata, data-processing and analysis scripts, software-environment documentation, validation files, generated outputs, README_REPRODUCIBILITY, and a SHA-256 manifest. Because OpenAlex citation counts and metadata may change over time, all reported results correspond to the June 2026 extraction.
Acknowledgments
The authors acknowledge the institutional support of Universidad Nacional Toribio Rodríguez de Mendoza de Amazonas (UNTRM), Chachapoyas, Peru, during the conduct of this research.
Conflicts of Interest
The authors declare no conflicts of interest.
References
- Bornmann, L., & Daniel, H. D. (2007). What do we know about the h index? Journal of the American Society for Information Science and Technology, 58(9), 1381–1385. [Google Scholar] [CrossRef] [Scilit]
- DORA. (2012). San Francisco declaration on research assessment. Available online: https://sfdora.org/read/ (accessed on 12 July 2026).
- Else, H. (2020). How a torrent of COVID science changed research publishing—In seven charts. Nature, 588(7839), 553. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Estrada, E. (2022). Produccion cientifica sobre la COVID-19 en la revista Anales de la Facultad de Medicina. Anales de la Facultad de Medicina, 83(4), 364–365. [Google Scholar] [CrossRef] [Scilit]
- Fassin, Y. (2021). Research on COVID-19: A disruptive phenomenon for bibliometrics. Scientometrics, 126(6), 5305–5319. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Gregorio-Chaviano, O., Limaymanta, C. H., & Lopez-Mesa, E. K. (2020). Analisis bibliometrico de la produccion cientifica latinoamericana sobre COVID-19. Biomedica, 40, 104–115. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature, 520(7548), 429–431. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Ioannidis, J. P. A., Bendavid, E., Salholz-Hillel, M., Boyack, K. W., & Baas, J. (2022). Massive covidization of research citations and the citation elite. Proceedings of the National Academy of Sciences of the United States of America, 119(28), e2204074119. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Ioannidis, J. P. A., Salholz-Hillel, M., Boyack, K. W., & Baas, J. (2021). The rapid, massive growth of COVID-19 authors in the scientific literature. Royal Society Open Science, 8(9), 210389. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Koltun, V., & Hafner, D. (2021). The h-index is no longer an effective correlate of scientific reputation. PLoS ONE, 16(6), e0253397. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Liu, W., Huangfu, X., & Wang, H. (2023). Citation advantage of COVID-19-related publications. Journal of Information Science, 51(6), 1529–1535. [Google Scholar] [CrossRef] [Scilit]
- Papke, L. E., & Wooldridge, J. M. (1996). Econometric methods for fractional response variables with an application to 401(k) plan participation rates. Journal of Applied Economics, 11(6), 619–632. [Google Scholar] [CrossRef] [Scilit]
- Priem, J., Piwowar, H., & Orr, R. (2022). OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and concepts. arXiv, arXiv:2205.01833. [Google Scholar] [CrossRef] [Scilit]
- Sjögårde, P. (2022). The effect of the rapid growth of COVID-19 publications on citation indicators. arXiv, arXiv:2205.01010. [Google Scholar] [CrossRef] [Scilit]
- Tripathy, J. P. (2013). Secondary data analysis: Ethical issues and challenges. Iranian Journal of Public Health, 42(12), 1478–1479. [Google Scholar] [PubMed]
- Vasquez-Uriarte, K., Roque-Henriquez, J. C., Angulo-Bazan, Y., & Ninatanta Ortiz, J. A. (2021). Analisis bibliometrico de la produccion cientifica peruana sobre la COVID-19. Revista Peruana de Medicina Experimental Y Salud Pública, 38(2), 224–231. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Visser, M., van Eck, N. J., & Waltman, L. (2021). Large-scale comparison of bibliographic data sources: Scopus, Web of Science, Dimensions, Crossref, and Microsoft Academic. Quantitative Science Studies, 2(1), 20–41. [Google Scholar] [CrossRef] [Scilit]
- von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gotzsche, P. C., & Vandenbroucke, J. P. (2007). The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: Guidelines for reporting observational studies. The Lancet, 370(9596), 1453–1457. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Waltman, L., & van Eck, N. J. (2012). The inconsistency of the h-index. Journal of the American Society for Information Science and Technology, 63(2), 406–415. [Google Scholar] [CrossRef] [Scilit]
- You, T., Lee, J. Y., Park, J., & Yun, J. (2024). Auditing citation polarization during the early COVID-19 pandemic. Quantitative Science Studies, 5(4), 906–928. [Google Scholar] [CrossRef] [Scilit]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.