Research Trends of Vaccination-Related Systematic Reviews, 2011–2023: A Bibliometric Analysis

Abstract

Systematic reviews (SRs) reflect the best available evidence for informing vaccination recommendations. This study presents a bibliometric analysis of vaccination-related SRs aiming to uncover research trends. Vaccination-related SRs published from 2011 to 2023 in MEDLINE, Embase, the Cochrane Library and the Living Overview of Evidence database were considered. Trends over time, disease/pathogen, topic, population, geographical location, accessibility, methodological quality, and overlap were descriptively analyzed using R. A total of 2275 SRs were identified, most of which were freely accessible (n = 2083, 91.7%). The annual number of published SRs increased more than twelvefold from 2011 to 2023. COVID-19 (n = 861, 37.8%), influenza (n = 328, 14.4%), Human papillomavirus (n = 248, 10.9%), and pneumococcal disease (n = 152, 6.7%) were the most frequently addressed diseases/pathogens. Efficacy/effectiveness (n = 1066, 46.9%) and safety of vaccines (n = 812, 35.7%) were the most common topics. The methodological quality of SRs on intervention topics (n = 1376) was mostly critically low (n = 1155, 84.0%). Several SRs were identified that covered similar diseases/pathogens, topics and populations, indicating duplication and overlap, particularly for COVID-19. Our analysis showed a large increase in the number of published vaccination-related SRs. The results provide a basis for understanding the current state and priorities in vaccination research and decrease the overlap potential in newly developed SRs.

1. Introduction

Vaccinations are amongst the most crucial and effective means of preventing and controlling infectious diseases (Immunization Agenda 2030 Partners, 2024). To ensure that immunization policies are based on best available evidence, the World Health Organization (WHO) recommends that countries establish National Immunization Technical Advisory Groups (NITAGs) (Immunization Agenda 2030 Partners, 2024). As reemphasized in the Immunization Agenda 2030 (IA2030), NITAGs provide expert advice to governments on vaccine-related decisions, such as introducing new vaccines, setting immunization schedules, and ensuring vaccine safety (Duclos, 2010; Immunization Agenda 2030 Partners, 2024). When developing recommendations, NITAGs are encouraged to use systematic reviews (SRs) reflecting the best available evidence. By applying rigorous and transparent methods, SRs synthesize results of multiple studies. In recent years, the body of evidence has expanded rapidly, challenging NITAGs in accessing technical resources to effectively synthesize and assess the relevant literature (Fernandes et al., 2018; Steffen et al., 2021).

This paper presents a bibliometric analysis of the vaccination-related SRs. Bibliometric analysis is a quantitative method used to evaluate the inter-relationships and impacts of publications (Donthu et al., 2021). Unlike SRs, bibliometric analyses summarize publication details such as author information, journals, and keywords (Ellegaard & Wallin, 2015), and highlight shifts in research focus and emerging areas of interest. However, to our knowledge, only one bibliometric analysis has been published to date, analyzing trends of vaccination-related SRs published from 2008 to 2016 (Fernandes et al., 2018).

The objectives of this paper were (i) to provide an overview of the vaccination-related SR literature by describing trends in time, journal, accessibility, topic, disease, population, geographic region, and methodological quality published between 1 January 2011 and 31 December 2023 and (ii) to identify the extent of duplication and overlap in vaccination-related SRs.

2. Methods

2.1. Data Source: Systematic Reviews from SYSVAC Registry

SRs from the freely accessible SYSVAC registry were considered in this study (WHO, n.d.-a). The following section describes the standard operation used to populate the SYSVAC registry with SRs. In this registry, both SRs and systematic umbrella reviews are included (hereafter referred as SRs). Systematic searches for SRs were carried out in MEDLINE (via Ovid), Embase, the Cochrane Library of Systematic Reviews, and the COVID-19 Living Overview of Evidence (L-OVE) repository (Epistemonikos, n.d.). The last literature search was conducted in June 2024, using search terms related to “vaccination” and “immunization” (see Supplementary Materials for complete search strategy for each database). Vaccination-related SRs published from 01 January 2011 to 31 December 2023 were included. Further eligibility required articles (1) to state if they are a SR or follow minimum criteria defined by Robinson et al. (2016), (2) focus on preventive vaccines for humans, (3) not be a pre-print, (4) not be withdrawn, and (5) be written in English language. The inclusion and exclusion criteria can be found in Table S1, Supplementary Materials.

EndNote (Version 21.3) was used for record management. After removing duplicates and irrelevant records, the remaining records were screened for eligibility by one author based on the title and abstract of the published SRs. The eligibility of full texts of the remaining records was assessed by two reviewers independently. Disagreement between reviewers was resolved by discussion or by consulting with a third reviewer.

Eligible SRs were indexed using keywords according to a keyword coding protocol (see Supplementary Materials, Table S2 for keyword categorization). Keywords enabled SRs to be assigned to categories in seven areas: (1) accessibility of full texts; (2) vaccine-preventable disease or pathogen, e.g., COVID-19 (if applicable); (3) vaccine topic, e.g., safety of vaccines; (4) target population, e.g., adults, health care workers; (5) focus on low and middle-income countries (if applicable); (6) geographical location according to WHO regions (WHO, n.d.-b) (if applicable); and (7) methodological quality using A Measurement Tool to Assess systematic Reviews 2 (AMSTAR 2) (Shea et al., 2017), e.g., high. Keywords were not mutually exclusive. SRs could be classified with multiple keywords or none, except for disease/pathogen or geographical region. Any uncertainties during coding process were resolved by consensus between two reviewers or by consulting a third reviewer.

2.2. Data Analysis

An analysis protocol was developed in advance and is available upon request. For this analysis, the registry content (as of June 2024) was exported to Excel 2019. Data categorizations were checked in Excel, providing complete information of each publication, including the literature title, author list, journal name, publication year, and keywords. The dataset was descriptively analyzed by year of publication, journal, and keywords. Time series analysis on the disease/pathogen and methodological quality keyword category were used to explore trends over time. Subgroup analysis was performed to describe trends for umbrella reviews. To identify SRs covering identical or overlapping topics, the keywords disease/pathogen, topic, and population were examined in a bubble plot. In this plot, the bubble size corresponded to the number of SRs published on each unique keyword combination. Larger bubbles indicated higher productivity of SRs and allowed assessment of several intersections, while smaller bubbles indicated less frequent productivity, reflecting potential gaps in the literature. Both nominal and metric data were described by calculating frequencies with corresponding percentages. Bibliometric analysis and data visualization were conducted using R (version 4.4.1).

3. Results

3.1. Journals and Accessibility of Full Texts

A total of 2275 SRs were included in the bibliometric analysis, 28 of which were umbrella reviews (see Figure 1 for PRISMA flowchart). The 2275 unique SRs were published in 653 different academic journals. The most productive journals were Vaccine (n = 213, 9.4%), Vaccines (n = 197, 8.7%), and Human vaccines & Immunotherapeutics (n = 136, 6.0%) (see Supplementary Materials, Table S3 for top ten of productive journals). Most SRs were published open access (n = 2083, 91.6%), while for the remaining SRs subscription was needed to access full text of publications (n = 192, 8.4%).

Regarding umbrella reviews (n = 28), PLOS One and Vaccine (n = 3 each, 10.7%) were the most productive journals (see Supplementary Materials, Table S4 for journals). The majority of umbrella reviews were freely accessible in full text (n = 26, 92.9%).

3.2. Trend over Time (2011–2023)

The number of vaccination-related SRs published per year increased continually from 40 in 2011 to 495 in 2023, a more than twelvefold increase (mean number of annual SRs = 175; see Figure 2 for bar chart). The highest number of SRs (n = 614) was published in 2022. Notably few SRs were published from 2011 to 2013 (n = 154, 6.8%), while more than half of the SRs (n = 1392, 61.2%) were published during the last three years (2021–2023). A similar time trend was shown for umbrella reviews (see Supplementary Materials, Figure S1 for bar chart by year).

3.3. Trend for Disease/Pathogen

In total, 43 disease/pathogen areas were covered in the SRs (see Supplementary Materials, Table S2 for alphabetical order of disease/pathogen areas). Among 2275 SRs, 1918 (84.3%) focused on specific disease/pathogen areas, of which 1779 (78.2%) focused on one disease/pathogen and 139 (6.1%) on two or more. For eight disease/pathogen areas more than 50 SRs were identified: COVID-19, influenza, Human papillomavirus (HPV), pneumococcal disease, hepatitis B, rotavirus, measles, and pertussis (see

Table 1. Vaccination-related systematic reviews by disease/pathogen, published from 2011–2023.

Disease/Pathogen	n	%
COVID-19	861	37.8
Influenza	328	14.4
HPV 1	248	10.9
Pneumococcal disease	152	6.7
Hepatitis B	93	4.1
Rotavirus	66	2.9
Measles	60	2.6
Pertussis	53	2.3
Tuberculosis	47	2.1
Rubella	36	1.6
Tetanus, Varicella (each)	35	1.5
Poliomyelitis	33	1.5
Herpes zoster	31	1.4
Diphteria, Meningococcal disease, Mumps (each)	28	1.2
Haemophilus influenzae type b	25	1.1
Hepatitis A	18	<1.0
Cholera	16	<1.0
Yellow fever	13	<1.0
Dengue	12	<1.0
Rabies	10	<1.0
HIV/AIDS	9	<1.0
Japanese encephalitis, Mpox (each)	7	<1.0
Typhoid	6	<1.0
Streptococcus group B, RSV 2, Tick-borne encephalitis (each)	5	<1.0
Hepatitis C, Smallpox (each)	4	<1.0
Malaria	3	<1.0
Escherichia coli (ETEC), Ebola, Salmonella (each)	2	<1.0
Coxiella burnetii (Q fever), Hepatitis E, Herpes simplex, Lyme disease (Borreliosis), Shigella, Streptococcus group A, Zika (each)	1	<1.0
Summary estimates
All systematic reviews	2275	100
No disease/pathogen-specific area	357	15.7
One disease/pathogen area	1779	78.2
Two or more disease/pathogen area	139	6.1

¹ Human papillomavirus; ² RSV: Respiratory syncytial virus.

for number of SRs for each disease/pathogen). These eight diseases/pathogens fluctuated in the annual number of SRs over time (see line chart Figure 3 and Supplementary Materials, Table S5 for absolute numbers). The sharpest increase in the number of SRs was for COVID-19, followed by influenza, HPV and pneumococcal disease. For COVID-19 and influenza, there is a peak in 2022, shortly after the early 2020 SARS-CoV-2 pandemic. The number of SRs for hepatitis B, rotavirus, measles, and pertussis seems to have plateaued in the past three years.

In contrast, umbrella reviews covered only five disease/pathogen areas (see Supplementary Materials, Table S6 for number of umbrella reviews for each disease/pathogen). The highest number of umbrella reviews focused on influenza with nine umbrella reviews.

3.4. Trends for Vaccination Topic

The majority of SRs addressed the efficacy/effectiveness (n = 1066, 46.9%) and safety of vaccines (n = 812, 35.7%) (see Figure 4 for trend by vaccination topic). Further vaccination topics were acceptance (n = 524, 23.0%), coverage (n = 455, 20.0%) and administration (n = 337, 14.8%). Less frequently addressed topics were ethical issues (n = 86, 3.8%), modelling (n = 27, 1.2%), and logistics (n = 15, 0.7%). All nine topic areas exhibited a general trend upwards in the annual number of SRs over time (see line chart Figure 5 and Supplementary Materials, Table S7 for absolute numbers). The sharpest increase in the number of SRs was for efficacy/effectiveness, safety, acceptance, coverage, and administration.

Similar trends were observed for umbrella reviews, with efficacy/effectiveness, safety, and acceptance being the leading topic areas (see Supplementary Materials, Figure S2 for bar chart).

3.5. Geographic Location

Most SRs did not focus on a specific country or WHO region (n = 1951, 85.8%). Among the 324 SRs with a geographical focus, 307 (95%) focused on one WHO region and 17 (5%) on more than one WHO region. The most often addressed WHO region was the Americas with 94 SRs (4.1%), followed by Africa, Europe, and the Western Pacific with 87 (3.8%), 64 (2.8%) and 53 (2.3%) SRs, respectively (see Supplementary Materials, Figure S3 for details on distribution by WHO region). A smaller number of SRs focused on South-East Asia and the Eastern Mediterranean region (n = 26, 1.1% and n = 19, <1.0%), respectively. Only a small portion (n = 96, 4.2%) of the SRs targeted low and middle-income countries.

The majority of SRs were not restricted to a specific country (n = 2084, 91.6%), while 187 (8.2%) SRs had a single-country focus and 4 (0.2%) SRs had a multi-country focus. The United States of America (n = 54, 2.4%), followed by China (n = 29, 1.3%) and Ethiopia (n = 24, 1.1%) were the countries with the most focused SRs, respectively (see Supplementary Materials, Table S8 for other country foci). Of the 190 SRs focusing on a specific country, in 173 (91.1%) SRs the country focus was identical to the country affiliation of the corresponding author.

3.6. Target Population

A total of 1019 (44.8%) SRs targeted one particular age or population group, 794 (34.9%) focused on two groups, and 462 (20.3%) on more than two age or population groups. More than a third of the SRs included all age groups (n = 878 SRs; 38.6%), followed by adults aged ≥18–64 years with 837 SRs (36.8%) (see Figure 6 for distribution of further age or population groups). Less often addressed age groups were older adults aged ≥65 years (n = 527, 23.2%), adolescents aged 10–17 years (n = 518, 22.8%), children aged 1–9 years (n = 363, 15.6%) and newborns aged ≤1 year (n = 301, 13.2%). Other population groups were healthcare workers (n = 300, 13.2%), parents/caregivers (n = 209, 9.2%), pregnant women (n = 195, 8.6%), and travellers (n = 13, 0.6%).

A similar distribution was observed for umbrella reviews, with all age groups being the most often targeted population group (see Supplementary Materials, Figure S4 for bar chart).

3.7. Methodological Quality

In the majority of the 1376 SRs where AMSTAR 2 was applicable, methodological quality was assessed as critically low, indicating flaws in more than one of seven critical items of the 16-item AMSTAR 2 instrument (n = 1155, 84.0%). The remaining SRs were assessed as low (n = 139, 10.0%; flaw in one critical item), high (n = 75, 5.5%; flaw in no or one non-critical item) or moderate (n = 7, 0.5%; flaws in more than one non-critical item). The methodological quality of SRs remained stable over the time, with the exception of 2018 when the percentage of high-rating SRs increased to 22% (see Figure 7 for stacked bar chart of methodological quality by year and Supplementary Materials, Table S9 for relative frequencies). For information on journals of high-quality SRs, refer to Supplementary Materials, Table S10.

3.8. Repetition and Overlap of Systematic Reviews

There were several overlapping topics among the SRs (see Figure 8 for bubble plot). Of the 861 SRs on COVID-19, 389 focused on COVID-19 vaccination for adults, with 258 addressing safety outcomes, 155 the efficacy/effectiveness, 50 addressing administration, and 41 focusing on vaccine acceptance. In 2022, seven SRs (Elberry et al., 2022; Kim et al., 2022; Kolahchi et al., 2022; Matar et al., 2022; Palaiodimou et al., 2022; Saluja et al., 2022; Zheng et al., 2022) were published on thrombosis after COVID-19 vaccination. A further eight SRs published in 2022 focused on COVID-19 vaccine hesitancy and acceptance among pregnant women (Azami et al., 2022; Bhattacharya et al., 2022; Bianchi et al., 2022; Galanis et al., 2022; Halemani et al., 2022; Nikpour et al., 2022; Nindrea et al., 2022; Sarantaki et al., 2022). Of 328 SRs on influenza, 97 targeted older adults aged ≥65 years, with 70 on efficacy/effectiveness and 30 on safety outcomes. Among the 248 SRs addressing HPV, 51 targeted caregivers/parents of which 46 focused on barriers and factors associated with HPV vaccination and 36 SRs on determinants of uptake. Among 152 SRs on pneumococcal disease, 56 SRs focused on pneumococcal vaccination of newborns, 42 SRs addressed efficacy/effectiveness, 22 focused on administration and 9 each on coverage and safety. For further overlap of SRs in other topic areas, refer to Figure 8.

4. Discussion

This bibliometric analysis provides a comprehensive overview of the landscape of SRs on vaccination topics published in the past 13 years. A total of 2275 SRs were identified. The results indicate a considerable increase in vaccination-related SRs during this time period. Several factors might have contributed to this increase, including the introduction of new vaccines or advances in research funding with support from international organizations or initiatives such as WHO (Immunization Agenda 2030 Partners, 2024) or GAVI (GAVI, n.d.). Moreover, from early 2020 onwards, the COVID-19 pandemic had a strong impact on research orientation and hot topic areas. Several vaccines, such as mRNA vaccines, were developed at an unprecedented pace, leading to an increase in SRs to assess their efficacy/effectiveness, safety and technological innovation (Pardi et al., 2018). Nonetheless, the trend in the number of SRs in vaccination appears to be in line with the growth of general scientific output. Until 2023, more than 30 million articles had been indexed in MEDLINE (National Library of Medicine, n.d.). The expansion of primary research may have favoured the emergence and proliferation of systematic literature reviews.

In recent years, there have been some bibliometric analyses of articles on vaccine-preventable diseases such as COVID-19, HPV, pertussis or Anthrax, revealing publishing activity in this area (Ahmad et al., 2022; Alsulaiman et al., 2025; Bruel et al., 2021; Tan et al., 2024). Similarly to our results, Vaccine was the most productive journal. However, to our knowledge only one previous bibliometric analysis focused on vaccination-related SRs in terms of journals, trends over time, disease/pathogen, topic, population and geographical focus. In 2018, Fernandes et al. analyzed SRs published on vaccination from 2008 to 2016 and found the most addressed disease/pathogen areas being influenza, HPV and pneumococcal disease. These trends have remained the same during the past seven years, in addition to COVID-19. As this study showed, the efficacy/effectiveness and safety remained the predominant vaccination topics.

During the past decades, interest in vaccine research has grown due to the development of new vaccines to prevent serious illness (Steffen et al., 2021). However, the mass proliferation of SRs has raised concerns about their quality and rigorousness (Ioannidis, 2016). To our knowledge, this is the first study analyzing the trends of the methodological quality of vaccination-related SRs. Our analysis shows that most SRs were critically low in methodological quality using AMSTAR 2, questioning the internal validity of these SRs (see chapter 3.7). AMSTAR 2 is one of the most widely used instruments for methodological quality assessment of SRs, yet the assessment of some items is inherently subjective. According to the developers, there are seven items that can critically affect the validity of a SR and its conclusions (item 2: protocol registration, item 4: adequacy of literature search strategy, item 7: justification for excluding individual studies, item 9: risk of bias assessment, item 11: appropriateness of meta-analytical methods, item 13: risk of bias interpretation, and item 15: publication bias). In our analysis, there was more than one flaw in the seven critical items present, underscoring the need for stronger adherence to reporting and methodological standards such as PRISMA and PROSPERO (Schiavo, 2019; Tugwell & Tovey, 2021). Nonetheless, there may be items not always regarded as critical and considered less important (Shea et al., 2017). Depending on this, SRs with critically low quality might still yield useful information and may be considered by NITAGs. Interestingly, results showed that the percentages of high-quality SRs increased in 2018. This may have been favoured by the release of the AMSTAR 2 instrument and the Cochrane Collaboration Handbook for review authors in 2017 (Higgins et al., 2017; Shea et al., 2017). As both releases aimed at setting methodological standards for the conduct of SRs, they might have contributed to an improvement of the methodological quality of SRs published in the following year. Going into detail on the individual AMSTAR 2 items is beyond the scope of this paper. Furthermore, a comparison with other assessment tools, such as ROBIS, which assesses the risk of bias of SRs, may be necessary to confirm the validity of the findings. To date, however, only AMSTAR 2 has been routinely applied within the SYSVAC registry. Future analyses are planned, focusing on the methodological quality of the SRs included in the SYSVAC registry.

The potential for duplication and overlap rises with the increase of SRs. Our results showed numerous SRs targeting similar diseases/pathogens, vaccination topics and population groups. This overlap underscores the redundancy of research outputs with identical settings. It emphasizes the importance of improving resource deployment. A more practical approach for authors may be to register their SR protocols in advance and foster collaborations to combine efforts and minimize waste of limited resources. Policymakers and journal editors can play a crucial role in promoting these practices, ensuring a more efficient and effective approach to SR production. We therefore recommend that researchers should be encouraged, e.g., by journal editors, to check the landscape of existing SRs before planning to conduct a new SR. Moreover, in cases when SRs on a particular topic already exist, journal editors should ask authors of new submissions to clearly explain the rational for performing a new SR. Lastly, policymakers are encouraged to use existing SRs, instead of asking for new ones to support their policy-making. On the other hand, the findings from this bibliometric analysis can be leveraged by NITAGs and authors of new SRs to identify gaps in the vaccination literature and identify areas where further SRs are needed. By examining the overlap of existing SRs, NITAGs and authors can determine which topics have been thoroughly explored and underexplored. The SYSVAC registry can be a resource to identify these emerging gaps. NITAGs and authors can conduct new SRs that avoid redundancy and contribute novel insights to the vaccination field. This targeted approach ensures new SRs to address relevant and impactful research, offering high value to NITAGs and policymakers, who rely on robust and non-redundant evidence to inform their decisions.

The SYSVAC registry and this bibliometric analysis have two limitations. First, there is a potential for language bias due to the inclusion criteria of the SYSVAC registry which is restricted to SRs written in English language (n = 280 non-English articles, see flowchart, Figure 1). This limitation may affect the representation of evidence from low and middle-income countries or non-English-speaking regions, where research may be conducted and disseminated in local languages. Additionally, the exclusion of SRs due to lack of institutional access can lead to an overrepresentation of SRs from well-funded institutions with better access to resources. Nonetheless, systematic searches with no language restrictions are conducted to embed SRs into the registry, ensuring a broad and thorough collection of evidence from multiple databases (see Supplementary Materials for comprehensive search strategies). Additionally, the number of SRs not written in English or not accessible is relatively small, minimizing their likely impact on the SR pool (see Figure 1 for flowchart). A list of these excluded SRs is available upon request. However, some SRs on vaccination topics may not be included in the registry because they are deemed irrelevant to the work of NITAGs, such as the review of the evolution of studies concerning HPV vaccination (Bruel et al., 2021). We also acknowledge that research trends may have shifted since 2024. However, this analysis is limited to SRs published up to 2023 to ensure data quality and consistency, as the incorporation of SRs published in 2024 into the SYSVAC registry has not yet been completed. As of April 2025, 282 SRs published in 2024 were included in the SYSVAC registry WHO (n.d.-a). Second, the presented descriptive analysis on identified overlaps of SR topics is based on the developed and assigned keywords, but not on the specific research question of the SR (see Supplementary Materials, Table S2 for keyword categorization). It is currently unclear to what extent the keywords match the question of interest of the SR such as the intervention, comparator or outcomes. A further assessment by the user is essential. Future analyses combining bibliometrics with qualitative content analysis could bridge the gap and provide a further understanding of the alignment between SR scope and content.

5. Conclusions

The number of vaccination-related SRs increased tremendously from 2011 to 2023. However, the potential for overlap and duplication remains high, emphasizing the importance of improving resource deployment. This bibliometric analysis can provide a basis for understanding the current state and priorities in the vaccination research. The SYSVAC registry can facilitate identify existing SRs and decrease the potential of overlap in newly developed SRs (www.nitag-resource.org/sysvac-systematic-reviews) (accessed on 22 April 2025).