Next Article in Journal
BNT162b2 mRNA COVID-19 Vaccine Does Not Impact the Honeymoon Phase in Type 1 Diabetes: A Case Report
Next Article in Special Issue
Attitudes toward COVID-19 Vaccination on Social Media: A Cross-Platform Analysis
Previous Article in Journal
Willingness to Accept the COVID-19 Vaccine and Related Factors among Indian Adults: A Cross-Sectional Study
Previous Article in Special Issue
Knowledge of University Students in Health Care Settings on Vaccines and Vaccinations Strategies: Impact Evaluation of a Specific Educational Training Course during the COVID-19 Pandemic Period in Italy
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Vaccines
Volume 10
Issue 7
10.3390/vaccines10071097
vaccines-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Systematic Review

First COVID-19 Booster Dose in the General Population: A Systematic Review and Meta-Analysis of Willingness and Its Predictors

by
Petros Galanis
1,*,
Irene Vraka
2,
Aglaia Katsiroumpa
1,
Olga Siskou
3,
Olympia Konstantakopoulou
4,
Theodoros Katsoulas
5,
Theodoros Mariolis-Sapsakos
5 and
Daphne Kaitelidou
4
1
Clinical Epidemiology Laboratory, Faculty of Nursing, National and Kapodistrian University of Athens, 11527 Athens, Greece
2
Department of Radiology, P. & A. Kyriakou Children’s Hospital, 11527 Athens, Greece
3
Department of Tourism Studies, University of Piraeus, 18534 Piraeus, Greece
4
Center for Health Services Management and Evaluation, Faculty of Nursing, National and Kapodistrian University of Athens, 11527 Athens, Greece
5
Faculty of Nursing, National and Kapodistrian University of Athens, 11527 Athens, Greece
*
Author to whom correspondence should be addressed.
Vaccines 2022, 10(7), 1097; https://doi.org/10.3390/vaccines10071097
Submission received: 17 June 2022 / Revised: 5 July 2022 / Accepted: 6 July 2022 / Published: 8 July 2022
(This article belongs to the Special Issue Factors Associated with COVID-19 Vaccination Intentions)
Browse Figures
Review Reports Versions Notes

Abstract

:
The emergence of breakthrough infections and new highly contagious variants of SARS-CoV-2 threaten the immunization in individuals who had completed the primary COVID-19 vaccination. This systematic review and meta-analysis investigated, for the first time, acceptance of the first COVID-19 booster dose and its associated factors among fully vaccinated individuals. We followed the PRISMA guidelines. We searched Scopus, Web of Science, Medline, PubMed, ProQuest, CINAHL and medrxiv from inception to 21 May 2022. We found 14 studies including 104,047 fully vaccinated individuals. The prevalence of individuals who intend to accept a booster was 79.0%, while the prevalence of unsure individuals was 12.6%, and the prevalence of individuals that intend to refuse a booster was 14.3%. The main predictors of willingness were older age, flu vaccination in the previous season, and confidence in COVID-19 vaccination. The most important reasons for decline were adverse reactions and discomfort experienced after previous COVID-19 vaccine doses and concerns for serious adverse reactions to COVID-19 booster doses. Considering the burden of COVID-19, a high acceptance rate of booster doses could be critical in controlling the pandemic. Our findings are innovative and could help policymakers to design and implement specific COVID-19 vaccination programs in order to decrease booster vaccine hesitancy.

1. Introduction

The emergence of breakthrough infections and new highly contagious variants of SARS-CoV-2 threaten the immunization in individuals who had completed the primary COVID-19 vaccination [1,2]. Moreover, breakthrough infections are more common in immunodeficient patients, causing significant clinical outcomes, e.g., hospitalization and mortality [3,4]. Importantly, evidence shows that new SARS-CoV-2 variants of concern may have reduced COVID-19 vaccine susceptibility and have increased infectivity [5,6].
COVID-19 vaccines were administrated by early 2021, were proven sufficiently safe and effective and demonstrated high protection against SARS-CoV-2 infections, hospitalizations and deaths [7,8]. However, efficacy or effectiveness against SARS-CoV-2 infection and symptomatic disease decreased six months after full vaccination [9]. Thus, there has been a discussion about the need for a COVID-19 booster dose [10,11]. Eventually, several countries deployed extra COVID-19 vaccine doses, especially for vulnerable groups, in order to strengthen immune responses and prolong protection against SARS-CoV-2.
Primary real-world data support the effectiveness of a first booster dose since SARS-CoV-2 infection rate, hospitalization rate and COVID-19-related mortality are lower among individuals who receive a first booster shot after the primary vaccination [12,13]. In addition, a first booster dose, when it is administered several months after the second COVID-19 vaccine dose, induces a robust immune response and prolongs protection [14,15]. Moreover, many countries have already recommended a second booster dose for high-risk groups to further support immunization against SARS-CoV-2. Thus, public willingness to accept booster doses could be a viable option to shore up protection against COVID-19 and control the pandemic.
Moreover, adherence to the recommended protective measures during the COVID-19 pandemic (e.g., COVID-19 vaccination) is influenced by several factors, such as knowledge and attitudes, especially among older adults [16,17]. In addition, higher levels of knowledge concerning COVID-19 are associated with COVID-19 vaccine acceptance [18,19]. Adequate knowledge about COVID-19 vaccination and positive beliefs regarding COVID-19 strengthen the confidence of the population in proactive behaviors to prevent COVID-19.
Until now, no systematic review has investigated individuals’ intention to accept COVID-19 booster doses. Therefore, we conducted a systematic review and meta-analysis to investigate the acceptance of a first booster dose and its associated factors among fully vaccinated individuals. Moreover, we estimated the percentage of the fully vaccinated individuals who refused a first booster dose, as well as those who were unsure.

2. Materials and Methods

2.1. Data Sources and Strategy

We applied the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines for this systematic review and meta-analysis [20]. We searched Scopus, Web of Science, Medline, PubMed, ProQuest, CINAHL and a pre-print service (medrxiv) from inception to 21 May 2022. We used the following strategy in all fields: (((vaccin*) AND (COVID-19)) AND (SARS-CoV-2)) AND (booster).

2.2. Selection and Eligibility Criteria

We used the following inclusion criteria: studies reporting the willingness of fully vaccinated individuals to receive or refuse a first COVID-19 booster dose, quantitative studies, studies that included adults, studies with samples from the general population, studies published in English and studies published in peer-reviewed journals. We excluded studies involving specific population groups (e.g., healthcare workers and patients) since the aim of our review was to investigate acceptance of a first COVID-19 booster dose among fully vaccinated individuals in the general population only. We considered that specific population groups should not be mixed with the general population since they are different populations regarding their attitudes toward COVID-19 vaccination. Moreover, we excluded reviews, protocols, case reports, opinion articles, commentaries, editorials and letters to the editor.
We used Zotero software to remove duplications, and we then consecutively reviewed titles, abstracts and full texts. Moreover, we hand-searched the reference lists of all relevant reviews and articles. Two independent authors performed study selection, and the most experienced authors resolved the discrepancies.

2.3. Data Extraction

Two authors independently collected data for the following items: reference, country, data collection time, sample size, percentage of females, age of participants, study design, sampling method, recruitment method, response rate, publication type (journal or pre-print), question to measure public willingness to accept a first COVID-19 booster dose, response scales, percentage of individuals who intended to accept a booster dose, percentage of individuals who intended to refuse a booster dose, percentage of individuals who were unsure and predictors of individuals’ decision to accept a booster dose.

2.4. Quality Assessment

Two independent authors used the Joanna Briggs Institute critical appraisal tool to assess the risk of bias of the included studies [21]. The most experienced authors solved any discrepancies. The Joanna Briggs Institute critical appraisal tool consists of eight questions (e.g., Were the criteria for inclusion in the sample clearly defined? Were the study subjects and the setting described in detail?, etc.). Response options for each question were the following: “yes”, “no”, “unclear” and “not applicable”. Considering the answers to the eight questions, we divided studies into low, medium or high risk of bias.

2.5. Statistical Analysis

There was great variation in the way that authors of different studies measured willingness and refusal of individuals towards a first COVID-19 booster dose. In particular, authors used different questions (e.g., “How likely do you think you are to get a COVID-19 booster vaccine if/when you are offered one?”, “Are you willing to receive the potential additional dose of the COVID-19 vaccine if it would be made available?”, “Do you accept receiving the COVID-19 booster vaccine?”, etc.) and different answers. Possible response options were in (a) yes/no options, (b) yes/no/unsure options and (c) Likert scales (e.g., very willing; willing; fair; unwilling; very unwilling; unsure). For each study, we followed the authors’ criteria regarding the decision of participants to accept a booster dose or not. Then, we divided the positive answers of participants by the total number of participants in order to calculate the percentage of participants who intended to accept a first COVID-19 booster dose. In the same way, we calculated the percentage of participants who intended to refuse a booster dose and the percentage of participants who were unsure. Finally, we used the Freeman–Tukey Double Arcsine method to transform the above three percentages. Moreover, we calculated the 95% confidence intervals (CI) for these percentages [22].
We performed a meta-analysis to calculate the prevalence of each included study, as well as the pooled estimate. We used the inconsistency index I2 to assess statistical heterogeneity among studies, with values higher than 75% indicating high heterogeneity [23]. We adopted a random effect model for all analyses since the statistical heterogeneity was very high [23]. We pre-specified the following sources of heterogeneity: data collection time, sample size, gender distribution, age, study design, sampling method, recruitment method, response rate, publication type (journal or pre-print service), response scales (studies with or without unsure option), quality of the studies and the country in which studies were conducted. Due to limited data for some variables (response rate), limited variability in some variables (study design, recruitment method, studies quality and publication type) and high heterogeneity in the measurement of some variables (age), we decided to perform subgroup analyses for the sampling method, response scales and countries. Moreover, we used data collection time, sample size and gender distribution as independent variables in meta-regression models. We considered data collection time as a continuous variable, assigning the number 1 for studies that were conducted in April 2021, the number 2 for studies that were conducted in May 2021, etc. A leave-one-out sensitivity analysis was used to determine the influence of each study on the overall prevalence. Funnel plots and the Egger’s test (p-value < 0.05) were used to assess potential publication bias [24]. The analysis was performed using OpenMeta[Analyst] statistical software [25].
High heterogeneity in the way that authors investigated the relationship between independent variables and individuals’ willingness to accept a first COVID-19 booster dose did not allow the performance of a meta-analysis. However, in order to quantify the magnitude of independent variables, we measured the percentage of studies finding positive or negative significant relationships (p-value < 0.05). Therefore, we calculated this percentage by dividing the number of studies with a positive significant relationship between the independent variable and individuals’ willingness to accept a booster dose by the total number of studies that examined the independent variable.

3. Results

3.1. Identification and Selection of Studies

Initially, we found 4712 unique records. Applying the inclusion and exclusion criteria, we identified 14 articles (Figure 1).

3.2. Characteristics of the Studies

Details of the studies included in this systematic review are presented in Table 1. We found 14 studies including 104,047 fully vaccinated individuals [26,27,28,29,30,31,32,33,34,35,36,37,38,39]. Nine studies were conducted in Asia (four in China; two in Jordan; one in Vietnam; one in Japan; one in Indonesia), three studies in Europe (one in the United Kingdom; one in Denmark; one in Poland), one study in the USA and one study in Africa (Algeria). Data collection times among studies ranged from April 2021 to March 2022. Sample sizes ranged from 413 to 31,721 individuals, with a median number of 2237 individuals. The percentage of females was higher than the percentage of males in seven studies, while in one study, we found the opposite. In addition, in five studies, the distribution of the two genders was almost equal. Thirteen studies were cross-sectional, and one study was cohort. Data collection was performed through online questionnaires in all studies. Seven studies used a convenience sample, six studies used the snowball sampling method and one study used a stratified random sample. All studies were published in peer-reviewed journals. Seven studies included an “unsure” response option for individuals’ willingness to accept a first COVID-19 booster dose, while seven studies did not include this response option (Table 2).

3.3. Characteristics of the Studies

The risk of bias was low in thirteen studies and moderate in one study. The quality assessment of the studies included in this review is shown in Supplementary Table S1.

3.4. Individuals’ Willingness and Refusal to Accept a First COVID-19 Booster Dose

Fourteen studies reported the number of individuals who intend to accept a first COVID-19 booster dose. The prevalence of individuals who intend to accept a booster was 79.0% (95% CI: 71.8–85.3%) (Figure 2). The heterogeneity between results was very high (I2 = 99.84%, p-value for the Hedges Q statistic < 0.001). Individuals’ willingness to accept a booster dose ranged from 44.6% to 97.9%. A leave-one-out sensitivity analysis showed that no single study had a disproportional effect on the overall willingness, which varied between 77.0% (95% CI: 69.3–83.9%) and 81.4% (95% CI: 74.6–87.3%). Publication bias was probable since the p-value for the Egger’s test was lower than 0.05, and the funnel plot was asymmetrical (Supplementary Figure S1).
Thirteen studies reported the number of individuals who intend to refuse a booster. The prevalence of individuals who intend to refuse a booster was 14.3% (95% CI: 8.4–21.4%) (Figure 3). The heterogeneity between results was very high (I2 = 99.82%, p-value for the Hedges Q statistic < 0.001). Individuals’ refusal to accept a booster dose ranged from 1.0% to 43.7%. A leave-one-out sensitivity analysis showed that no single study had a disproportional effect on the overall refusal, which varied between 12.0% (95% CI: 7.4–17.4%) and 15.8% (95% CI: 9.1–23.9%). Publication bias was probable since the p-value for the Egger’s test was lower than 0.05, and the funnel plot was asymmetrical (Supplementary Figure S2).
Six studies presented the number of individuals reporting that they were unsure of their intention to accept a booster. The prevalence of unsure individuals was 12.6% (95% CI: 6.8–19.9%) (Figure 4). The heterogeneity between results was very high (I2 = 99.80%, p-value for the Hedges Q statistic < 0.001). The prevalence of unsure individuals ranged from 4.3% to 24.7%. A leave-one-out sensitivity analysis showed that no single study had a disproportional effect on the overall refusal, which varied between 10.2% (95% CI: 4.5–17.5%) and 14.2% (95% CI: 7.1–23.1%). Publication bias was probable since the p-value for the Egger’s test was lower than 0.05, and the funnel plot was asymmetrical (Supplementary Figure S3).

3.5. Impact of the “Unsure” Response Option

The prevalence of willingness was lower in studies with an “unsure” response option (69.6%, 95% CI = 63.3–75.9%, I2 = 99.82%) than in studies without an “unsure” response option (83.3%, 95% CI = 76.5–90.1%, I2 = 99.76%). On the other hand, there was no impact of the “unsure” response option on the individuals’ refusal of a COVID-19 booster dose. In particular, the prevalence of refusal in studies with an “unsure” response option was 16.4% (95% CI = 12.5–20.4%, I2 = 99.73%), and in studies without an “unsure” response option, it was 16.7% (95% CI = 9.9–23.5%, I2 = 99.76%).

3.6. Impact of the Sampling Method

The sampling method did not affect the prevalence of individuals’ willingness to accept a booster dose. In particular, the prevalence of willingness in studies with a convenience sample was 77.6% (95% CI = 60.8–90.7%, I2 = 99.49%), and in studies that adopted a snowball sampling method, it was 76.9% (95% CI = 63.3–88.1%, I2 = 99.84%). Similarly, the impact of the sampling method on the prevalence of individuals’ refusal to accept a booster was very low, since the prevalence of refusal in studies with a convenience sample was 14.8% (95% CI = 3.7–31.6%, I2 = 99.86%), and in studies that adopted a snowball sampling method, it was 16.1% (95% CI = 3.5–35.2%, I2 = 99.83%).

3.7. Impact of the Countries

We separated countries according to the Confucian culture circle (China, Vietnam and Japan), and we found that the prevalence of willingness was higher in studies that were conducted in countries that belong to the Confucian culture circle (88.5%, 95% CI = 79.2–95.3%, I2 = 99.78%) than in studies that were conducted in countries that do not belong to the Confucian culture circle (71.8%, 95% CI = 57.5–84.1%, I2 = 99.89%). On the other hand, there was no impact of the countries on the individuals’ refusal of a COVID-19 booster dose. In particular, the prevalence of refusal in studies that were conducted in countries that belong to the Confucian culture circle was 7.8% (95% CI = 1.3–19.2%, I2 = 99.72%), and in studies that were conducted in countries that do not belong to the Confucian culture circle, it was 7.0% (95% CI = 3.3–11.9%, I2 = 99.70%).

3.8. Meta-Regression Analysis

An increased sample size was associated with an increase in individuals’ willingness to accept a booster dose (coefficient = 0.000012, 95% CI = 0.000003 to 0.00021, p = 0.012). Moreover, an increased percentage of females in studies was associated with a decreased willingness (coefficient = −0.930, 95% CI = −1.736 to −0.123, p = 0.024). On the other hand, data collection time did not affect willingness (coefficient = -0.008, 95% CI = −0.055 to 0.040, p = 0.752).
Regarding individuals’ refusal to accept a booster dose, an increased sample size was associated with decreased refusal (coefficient = −0.00001, 95% CI = −0.00002 to −0.000008, p = 0.045). The percentage of females (coefficient = 0.101, 95% CI = −0.995 to 1.197, p = 0.857) and data collection time (coefficient = −0.008, 95% CI = −0.050 to 0.022, p = 0.718) had no impact on the prevalence of refusal.

3.9. Predictors of Individuals’ Willingness to Accept a First COVID-19 Booster Dose

Twelve studies investigated predictors of individuals’ willingness to accept a first COVID-19 booster dose (Table 3). Only one study did not use multivariable analysis to eliminate confounders.
Authors have mainly investigated the effect of socio-demographic characteristics on individuals’ willingness to accept further vaccination(s). In particular, in eight out of twelve studies, older individuals were more likely to accept a first COVID-19 booster dose; in two out of twelve studies, younger individuals were more willing; and two studies found no effect with age. Two studies out of ten found that males intend to accept a booster dose more often than females, but two studies found the opposite, and six studies found no effect of gender. Higher educational level was associated with individuals’ willingness to accept a booster in four out of ten studies, while three studies found the opposite relationship, and three studies found no effect of educational level. In two out of four studies, individuals with higher income were more likely to report an intention to accept a booster dose than individuals with lower income, while two studies did not find a significant relationship. Chronic comorbidities presented a controversial issue, since in three out of six studies, individuals with at least one chronic condition were more likely to accept further vaccination; in two out of six studies, chronic comorbidities showed no association with willingness; and in one study, individuals with comorbidity were less likely to accept vaccination. Ethnicity, marital status and residence were non-significant predictors in two out of two, four out of five and eight out of eight studies, respectively.
Few studies investigated the impact of COVID-19-related variables on individuals’ decision to accept a first COVID-19 booster dose. In particular, history of COVID-19 infection (three out of six studies; no association in three studies), higher risk perception of infection (three out of five studies; no association in two studies), and higher severity perception of COVID-19 (three out of six studies; negative association in one study; no association in two studies) were associated with an increase in individuals’ willingness to accept a first COVID-19 booster dose.
Confidence in COVID-19 vaccines (two out of two studies) and COVID-19 booster doses (four out of four studies) was associated with an increased acceptance of a first COVID-19 booster dose. On the other hand, adverse reactions and discomfort experienced after previous COVID-19 vaccines doses (four out of four studies) and concerns for serious adverse reactions to COVID-19 booster doses (two out of two studies) decreased individuals’ willingness to accept a booster.
A limited number of studies identified that a flu vaccination in the previous season (two out of three studies), confidence in the healthcare system/government/physicians (two out of three studies) and compliance with prevention measures (one out of one study) positively affected individuals’ intention to accept a booster dose.

4. Discussion

To the best of our knowledge, this is the first systematic review that estimates the acceptance and refusal of a first COVID-19 booster dose among fully vaccinated individuals. Moreover, we reviewed the available literature on predictors of individuals’ willingness to accept a booster dose. Applying inclusion and exclusion criteria, we found 14 studies carried out mainly in Asia from April 2021 to March 2022.

4.1. Willingness and Refusal of Individuals to Accept a First COVID-19 Booster Dose

Worldwide, among individuals who are fully vaccinated, 79% would take a first booster if recommended. There is great variability in the willingness rate among studies, from 44.6% to 97.9%. Moreover, we found that 14.3% of fully vaccinated individuals would refuse a booster dose, and 12.6% are unsure whether they would accept a booster dose. Thus, even among fully vaccinated individuals, many of them have no fixed opinion on boosters and may yet be persuadable. A similar review including studies in the general population found lower global primary COVID-19 vaccination willingness (66%) [40]. The willingness rate is even lower among healthcare workers (63.5%) [41] and parents who intend to vaccinate their children (60.1%) [42]. The acceptance rate of the first COVID-19 booster dose is higher than that of primary doses, but there is still a potential for improvement. Moreover, considering that this review included studies with fully vaccinated individuals, the acceptance rate of vaccination is relatively low. Our review shows that, even among those individuals who chose to receive the primary COVID-19 doses, a significant percentage expressed substantial hesitancy or reluctance to accept a booster dose. Concerns about booster safety, effectiveness and side effects, the belief that the primary COVID-19 vaccination provides sufficient immunization and a low severity perception of COVID-19 seem to be the primary reasons for booster vaccine hesitancy [25,26,30,34,35]. New highly contagious variants of SARS-CoV-2 and the potential waning of vaccine protection highlight the urgent need for booster vaccination in order to end the COVID-19 pandemic. Therefore, policymakers should develop and implement effective communication strategies, emphasizing vaccination safety and effectiveness, in order to increase the proportion of individuals receiving the booster doses.

4.2. Subgroup Analyses

Interestingly, we found that the willingness rate was lower in studies with an “unsure” response option than in studies without an “unsure” response option. This finding is confirmed by two similar meta-analyses including individuals from the general population and parents [42,43]. The public willingness rate of primary COVID-19 vaccination in studies with an “unsure” response option was 63.5%, while in studies without an “unsure” response option, it was 82.8% [43]. Similarly, the proportion of parents who intend to vaccinate their children against COVID-19 in studies with an “unsure” response option was 58.3%, while in studies without an “unsure” response option, it was 64.5% [42]. It is reasonable that an “unsure” response option decreases the acceptance rate of booster doses since many individuals are unsure whether they would accept a booster.
Our meta-analysis identified that the increased percentage of females in studies was associated with decreased willingness. This finding is confirmed by a recent meta-analysis that found lower COVID-19 vaccination intentions among females than males [44]. Moreover, females are less likely than males to accept a first COVID-19 booster dose [31,34]. Moreover, more males than females actually receive a COVID-19 vaccine [45]. Females with limited knowledge regarding pregnancy, fertility and breastfeeding could explain their hesitancy towards COVID-19 vaccines [46,47]. Our finding is also in line with previous research on other vaccines. For example, male adolescents had a higher likelihood of being fully vaccinated compared with female adolescents [48]. In addition, females have lower vaccination rates than males in the case of pandemic influenza and influenza vaccinations [49,50,51]. Psychological and hormonal gender differences could explain vaccine hesitancy among females [52,53].
Moreover, we found that the prevalence of willingness was higher in studies that were conducted in countries that belong to the Confucian culture circle (China, Vietnam and Japan) than in studies that were conducted in other countries. The cultural background of individuals seems to influence their decision to accept a booster dose. However, there is a need to conduct studies that directly compare individuals from different cultural backgrounds in order to find valid results.

4.3. Predictors of Individuals’ Willingness to Accept a First COVID-19 Booster Dose

According to our review, positive attitudes and perceptions, including confidence in COVID-19 vaccination, the healthcare system, government and physicians, was associated with a willingness to receive a booster dose. On the other hand, negative attitudes and perceptions, including perceptions of the adverse reactions and low levels of safety and effectiveness of COVID-19 vaccination, were significantly associated with hesitance and reluctance. These findings are confirmed by similar reviews since individuals with positive attitudes towards COVID-19 vaccination are more likely to accept and less likely to refuse a vaccine [40,54,55]. Many people discuss concerns about the safety, efficacy and effectiveness of COVID-19 vaccines and how they are less willing to vaccinate. Public concerns about possible side effects of COVID-19 vaccines play a critical role in the intention to vaccinate. Evidence shows that the acceptance rate of COVID-19 vaccines changes with reported vaccine effectiveness changes [56,57]. Increased COVID-19 vaccine efficacy and prolonged protection time are associated with an increased acceptance rate [58]. Moreover, public attitudes towards COVID-19 vaccination tend to change over time because people want to see more data that have been gathered after vaccination [59,60]. In addition, vulnerable groups (e.g., elderly and patients who are more susceptible to clinical complications) are less willing to be vaccinated because they are more concerned about the side effects [61,62]. Therefore, reliable information regarding the safety and effectiveness of COVID-19 vaccines should be provided to those who have previously experienced side effects. Moreover, it is an unprecedented challenge for public health authorities to achieve a valid post-marketing surveillance in order to determine the long-term side effects of COVID-19 vaccinations [63]. This continuous evidence on COVID-19 vaccine safety may build trust among the general population and public health authorities to increase the vaccine confidence level.
We found that several socio-demographic characteristics affect individuals’ decision to accept a booster dose. In particular, older individuals were more likely to accept a booster. Evidence supports this finding since increased age is associated with increased COVID-19 vaccination intentions and uptake [36,41,50]. The elderly may have a greater sense of responsibility and accountability for themselves and their societies relative to younger individuals. Moreover, older people may think they are more susceptible to COVID-19-related clinical complications since an older age is a predictor of mortality in COVID-19 patients [64,65]. On the other hand, a lower risk perception of COVID-19 infection among younger people could explain their vaccine hesitancy [54].
Our review identified that people with a higher educational level were more likely to accept a booster dose. This finding is confirmed by the literature since an increased educational level is associated with an increased primary COVID-19 vaccine acceptance rate [36,50,51]. Well-educated individuals have a higher risk perception of COVID-19 infection, while less-educated people are vaccine-hesitant because they do not think they will be affected by COVID-19 [66]. Moreover, individuals who have a higher level of education also have access to multiple and valid information sources [67]. An educational level affects the general knowledge and awareness of individuals towards COVID-19 vaccines, while conspiracy theories, fake news and misinformation increase COVID-19 vaccine hesitancy, especially among less-educated people. Thus, there is a need for the implementation of social campaigns that deliver trusted news regarding COVID-19 vaccination.
Additionally, we found that people with higher income are more likely to accept a booster dose than those with lower income. Income has an effect on information achievements since people whose incomes are high are more aware of the negative consequences of COVID-19 and more willing to be vaccinated in order to maintain their health [68]. In contrast, people with low incomes are less likely to vaccinate since they lack the health insurance and financial resources that may be necessary for access to COVID-19 vaccines [69]. Moreover, socioeconomic disadvantages and health inequalities can increase vaccine hesitancy among low-income people [54]. On the other hand, high-income people have a higher risk perception of COVID-19 infection and are aware of the vaccines’ safety and effectiveness, which help them accept the vaccine more easily [66].

4.4. Limitations

Firstly, vaccine acceptance and hesitancy repeatedly change over time as new evidence and vaccination data are observed. Therefore, the findings of this review and meta-analysis may be invalid after a certain period. Longitudinal studies should be conducted in order to clarify individuals’ attitudes towards COVID-19 vaccination and identify changes in preference over time. Secondly, according to our review, only one study used a stratified random sample, while the other studies used convenience and snowball sampling methods. Sampling methods may influence vaccination willingness and refusal, and our findings may have been different if sampling had been different. For example, it is difficult to reach minority and under-represented groups through convenience and snowball sampling methods. Therefore, future studies should adopt random sampling in order to achieve an unbiased representation of the source populations. Additionally, 12 out of 14 studies in our review were conducted in Asian and European countries. This over-presentation of Asian and European countries may introduce bias in our review, and thus, our findings could not be generalized in other continents. Thus, there is an urgent need for further studies in continents other than Asia and Europe. Moreover, almost all studies in our review (13 out of 14) were cross-sectional, and thus, causal inferences could not be achieved. Prospective cohort studies should be conducted in order to reduce bias and achieve more valid results. Similarly, only 12 studies investigated predictors of individuals’ willingness to accept a first COVID-19 booster dose. Even worse, most studies focused only on the effect of socio-demographic characteristics on individuals’ willingness to accept booster doses. Future research should expand our knowledge regarding the predictors of COVID-19 willingness. For example, we could examine the role of psychological factors, social media variables, infection-related variables, etc. Moreover, we did not include specific population groups (e.g., healthcare workers and patients) in our review since our aim was to investigate the acceptance of a first COVID-19 booster dose in the general population only. We consider that patients have a different approach regarding COVID-19 vaccination since it is well-known that comorbidities significantly increase the negative outcomes of COVID-19, and thus, vaccination acceptance among patients is expected to be higher than the general population. In addition, healthcare workers are a high-risk group, and the frame for their decision to accept COVID-19 vaccines is different than that of the general population. For example, COVID-19 vaccination is obligatory for healthcare workers in nursing homes. Therefore, a systematic review that included specific population groups could add invaluable evidence in this field. Another limitation of our study is that we could not perform subgroup analysis for all the variables that we pre-planned due to limited data and variability and high heterogeneity in the measurements. Future studies should provide more data in order to give us the opportunity to make a better assessment of sources of heterogeneity. Finally, COVID-19 booster uptake should be investigated in the future since intentions do not always predict individuals’ actions. Thus, we should estimate the prevalence of the COVID-19 booster uptake in the general population and the factors that affect this behavior.

5. Conclusions

Our findings show that, even among fully vaccinated individuals, many of them have no fixed opinion on booster doses. With a low level of willingness towards receiving booster doses, it may be extremely difficult to manage the COVID-19 pandemic. The situation is becoming even worse due to the waning immunity to the first COVID-19 booster dose and the emergence of highly contagious variants of SARS-CoV-2. Thus, several countries have already recommended a second booster for vulnerable groups [15,70].
Policymakers should emphasize their strategies on COVID-19 vaccine/vaccination-related knowledge since several studies found a positive relationship between this knowledge and vaccine acceptance among adults [18,71,72]. Moreover, COVID-19-related knowledge is associated with the correct practice of preventive measures (e.g., hand washing, facial masks, vaccination, etc.) among the elderly [17,73,74]. In addition, knowledge about COVID-19 is associated with positive vaccine attitudes among healthcare workers since healthcare workers with a higher knowledge level about COVID-19 vaccination are more willing to be vaccinated [75,76]. An important determinant of the acceptance of COVID-19 vaccination among adults with chronic illnesses is knowledge of COVID-19 and information about the COVID-19 vaccine [77,78]. Informative communication with specific population groups and policymakers about COVID-19 vaccination is crucial to decrease vaccine hesitancy.
We should create awareness towards the COVID-19 vaccine in order to increase the acceptance of the COVID-19 vaccine among the general population. In that case, good knowledge about vaccination will help individuals to understand the safety, efficacy and effectiveness of the COVID-19 vaccine. Since several factors affect individuals’ decision to accept a COVID-19 vaccine, a holistic educational approach to improve confidence in the COVID-19 vaccine should be implemented. Moreover, policymakers should develop and implement targeted education for people with a low level of knowledge and a low level of willingness. In that case, transparent procedures, such as vaccination guarantee policies, open vaccine review procedures and reduced vaccine costs, could help to improve the acceptance rate of vaccination. Moreover, mass media and social media messages could promote adherence to protective measures, such as vaccination, hand washing, facial masks, etc.
The decline of individuals to accept future booster doses or even a new COVID-19 vaccine may undermine the public health advantages of a safe and effective vaccine. Therefore, understanding individuals’ willingness to take booster doses and the possible predictors affecting their vaccine attitudes will give policymakers the opportunity to develop and implement effective vaccination programs. Likewise, understanding COVID-19 vaccine hesitancy could improve COVID-19 vaccine uptake and tackle the COVID-19 pandemic. Booster vaccination is a high-priority task, and we should improve attitudes towards COVID-19 booster doses, providing community health education as soon as possible. There is an urgent need for continuous and updated evidence regarding public attitudes towards COVID-19 booster doses since a second booster dose for the general population in autumn 2022 is a possible scenario.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/vaccines10071097/s1, Table S1: Quality of studies included in this systematic review; Figure S1: Funnel plot of individuals’ willingness to accept a first COVID-19 booster dose; Figure S2: Funnel plot of individuals’ refusal to accept a first COVID-19 booster dose; Figure S3: Funnel plot of individuals reporting unsure of their intention to accept a first COVID-19 booster dose.

Author Contributions

Conceptualization, P.G., I.V. and A.K.; methodology, P.G., I.V., A.K., O.S., O.K., T.K. and T.M.-S.; software, P.G. and O.K.; validation, O.S. and O.K.; formal analysis, P.G. and O.K.; investigation, P.G., I.V. and O.S.; resources, T.K. and T.M.-S.; data curation, O.K., A.K. and D.K.; writing—original draft preparation, P.G., I.V. and D.K.; writing—review and editing, P.G., I.V., A.K., O.S., O.K., T.K., T.M.-S. and D.K.; visualization, P.G.; supervision, P.G. and D.K.; project administration, P.G. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Chung, H.-Y.; Jian, M.-J.; Chang, C.-K.; Lin, J.-C.; Yeh, K.-M.; Chen, C.-W.; Hsieh, S.-S.; Hung, K.-S.; Tang, S.-H.; Perng, C.-L.; et al. Emergency SARS-CoV-2 Variants of Concern: Novel Multiplex Real-Time RT-PCR Assay for Rapid Detection and Surveillance. Microbiol. Spectr. 2022, 10, e0251321. [Google Scholar] [CrossRef] [PubMed]
  2. Haque, A.; Pant, A.B. Mitigating COVID-19 in the Face of Emerging Virus Variants, Breakthrough Infections and Vaccine Hesitancy. J. Autoimmun. 2022, 127, 102792. [Google Scholar] [CrossRef] [PubMed]
  3. Gopinath, S.; Ishak, A.; Dhawan, N.; Poudel, S.; Shrestha, P.S.; Singh, P.; Xie, E.; Tahir, P.; Marzaban, S.; Michel, J.; et al. Characteristics of COVID-19 Breakthrough Infections among Vaccinated Individuals and Associated Risk Factors: A Systematic Review. Trop. Med. Infect. Dis. 2022, 7, 81. [Google Scholar] [CrossRef] [PubMed]
  4. Wang, L.; Kaelber, D.C.; Xu, R.; Berger, N.A. COVID-19 Breakthrough Infections, Hospitalizations and Mortality in Fully Vaccinated Patients with Hematologic Malignancies: A Clarion Call for Maintaining Mitigation and Ramping-up Research. Blood Rev. 2022, 54, 100931. [Google Scholar] [CrossRef] [PubMed]
  5. Boehm, E.; Kronig, I.; Neher, R.A.; Eckerle, I.; Vetter, P.; Kaiser, L. Novel SARS-CoV-2 Variants: The Pandemics within the Pandemic. Clin. Microbiol. Infect. 2021, 27, 1109–1117. [Google Scholar] [CrossRef] [PubMed]
  6. Davies, N.G.; Jarvis, C.I.; Edmunds, W.J.; Jewell, N.P.; Diaz-Ordaz, K.; Keogh, R.H. Increased Mortality in Community-Tested Cases of SARS-CoV-2 Lineage B.1.1.7. Nature 2021, 593, 270–274. [Google Scholar] [CrossRef]
  7. Fan, Y.-J.; Chan, K.-H.; Hung, I.F.-N. Safety and Efficacy of COVID-19 Vaccines: A Systematic Review and Meta-Analysis of Different Vaccines at Phase 3. Vaccines 2021, 9, 989. [Google Scholar] [CrossRef]
  8. Pormohammad, A.; Zarei, M.; Ghorbani, S.; Mohammadi, M.; Razizadeh, M.H.; Turner, D.L.; Turner, R.J. Efficacy and Safety of COVID-19 Vaccines: A Systematic Review and Meta-Analysis of Randomized Clinical Trials. Vaccines 2021, 9, 467. [Google Scholar] [CrossRef]
  9. Feikin, D.R.; Higdon, M.M.; Abu-Raddad, L.J.; Andrews, N.; Araos, R.; Goldberg, Y.; Groome, M.J.; Huppert, A.; O’Brien, K.L.; Smith, P.G.; et al. Duration of Effectiveness of Vaccines against SARS-CoV-2 Infection and COVID-19 Disease: Results of a Systematic Review and Meta-Regression. Lancet 2022, 399, 924–944. [Google Scholar] [CrossRef]
  10. Callaway, E. COVID Vaccine Boosters: The Most Important Questions. Nature 2021, 596, 178–180. [Google Scholar] [CrossRef]
  11. Tanne, J.H. COVID-19: Moderna Plans Booster Doses to Counter Variants. BMJ 2021, 372, n232. [Google Scholar] [CrossRef] [PubMed]
  12. Barda, N.; Dagan, N.; Cohen, C.; Hernán, M.A.; Lipsitch, M.; Kohane, I.S.; Reis, B.Y.; Balicer, R.D. Effectiveness of a Third Dose of the BNT162b2 MRNA COVID-19 Vaccine for Preventing Severe Outcomes in Israel: An Observational Study. Lancet 2021, 398, 2093–2100. [Google Scholar] [CrossRef]
  13. Kamar, N.; Abravanel, F.; Marion, O.; Couat, C.; Izopet, J.; Del Bello, A. Three Doses of an MRNA COVID-19 Vaccine in Solid-Organ Transplant Recipients. N. Engl. J. Med. 2021, 385, 661–662. [Google Scholar] [CrossRef] [PubMed]
  14. Falsey, A.R.; Frenck, R.W.; Walsh, E.E.; Kitchin, N.; Absalon, J.; Gurtman, A.; Lockhart, S.; Bailey, R.; Swanson, K.A.; Xu, X.; et al. SARS-CoV-2 Neutralization with BNT162b2 Vaccine Dose 3. N. Engl. J. Med. 2021, 385, 1627–1629. [Google Scholar] [CrossRef]
  15. Jantarabenjakul, W.; Sodsai, P.; Chantasrisawad, N.; Jitsatja, A.; Ninwattana, S.; Thippamom, N.; Ruenjaiman, V.; Tan, C.W.; Pradit, R.; Sophonphan, J.; et al. Dynamics of Neutralizing Antibody and T-Cell Responses to SARS-CoV-2 and Variants of Concern after Primary Immunization with CoronaVac and Booster with BNT162b2 or ChAdOx1 in Health Care Workers. Vaccines 2022, 10, 639. [Google Scholar] [CrossRef]
  16. Clements, J.M. Knowledge and Behaviors Toward COVID-19 Among US Residents During the Early Days of the Pandemic: Cross-Sectional Online Questionnaire. JMIR Public Health Surveill. 2020, 6, e19161. [Google Scholar] [CrossRef]
  17. Gallè, F.; Sabella, E.A.; Roma, P.; Ferracuti, S.; Da Molin, G.; Diella, G.; Montagna, M.T.; Orsi, G.B.; Liguori, G.; Napoli, C. Knowledge and Lifestyle Behaviors Related to COVID-19 Pandemic in People over 65 Years Old from Southern Italy. Int. J. Environ. Res. Public Health 2021, 18, 10872. [Google Scholar] [CrossRef]
  18. Abebe, H.; Shitu, S.; Mose, A. Understanding of COVID-19 Vaccine Knowledge, Attitude, Acceptance, and Determinates of COVID-19 Vaccine Acceptance Among Adult Population in Ethiopia. Infect. Drug Resist. 2021, 14, 2015–2025. [Google Scholar] [CrossRef]
  19. Gallè, F.; Sabella, E.A.; Roma, P.; De Giglio, O.; Caggiano, G.; Tafuri, S.; Da Molin, G.; Ferracuti, S.; Montagna, M.T.; Liguori, G.; et al. Knowledge and Acceptance of COVID-19 Vaccination among Undergraduate Students from Central and Southern Italy. Vaccines 2021, 9, 638. [Google Scholar] [CrossRef]
  20. Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G.; The PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med. 2009, 6, e1000097. [Google Scholar] [CrossRef] [Green Version]
  21. Santos, W.M.d.; Secoli, S.R.; Püschel, V.A.d.A. The Joanna Briggs Institute Approach for Systematic Reviews. Rev. Lat.-Am. Enferm. 2018, 26. [Google Scholar] [CrossRef] [Green Version]
  22. Barendregt, J.J.; Doi, S.A.; Lee, Y.Y.; Norman, R.E.; Vos, T. Meta-Analysis of Prevalence. J. Epidemiol. Community Health 2013, 67, 974–978. [Google Scholar] [CrossRef] [PubMed]
  23. Higgins, J.P.T. Measuring Inconsistency in Meta-Analyses. BMJ 2003, 327, 557–560. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  24. Egger, M.; Smith, G.D.; Schneider, M.; Minder, C. Bias in Meta-Analysis Detected by a Simple, Graphical Test. BMJ 1997, 315, 629–634. [Google Scholar] [CrossRef] [Green Version]
  25. Wallace, B.C.; Schmid, C.H.; Lau, J.; Trikalinos, T.A. Meta-Analyst: Software for Meta-Analysis of Binary, Continuous and Diagnostic Data. BMC Med. Res. Methodol. 2009, 9, 80. [Google Scholar] [CrossRef] [Green Version]
  26. Al-Qerem, W.; Al Bawab, A.Q.; Hammad, A.; Ling, J.; Alasmari, F. Willingness of the Jordanian Population to Receive a COVID-19 Booster Dose: A Cross-Sectional Study. Vaccines 2022, 10, 410. [Google Scholar] [CrossRef] [PubMed]
  27. Chu, D.-T.; Vu Thi, H.; Nguyen Thi, Y.V.; Nguyen, M.-A.; Bui, N.-L.; Hoang, V.T.; Hoang Nam, D.; Do, D.-L.; Than, V.T.; Al-Tawfiq, J.A. Willingness to Receive COVID-19 Vaccine Booster Doses for Adults and Their Children in Vietnam. J. Hum. Behav. Soc. Environ. 2022, 1–13. [Google Scholar] [CrossRef]
  28. Jørgensen, F.J.; Nielsen, L.H.; Petersen, M.B. Willingness to Take the Booster Vaccine in a Nationally Representative Sample of Danes. Vaccines 2022, 10, 425. [Google Scholar] [CrossRef]
  29. Lai, X.; Zhu, H.; Wang, J.; Huang, Y.; Jing, R.; Lyu, Y.; Zhang, H.; Feng, H.; Guo, J.; Fang, H. Public Perceptions and Acceptance of COVID-19 Booster Vaccination in China: A Cross-Sectional Study. Vaccines 2021, 9, 1461. [Google Scholar] [CrossRef]
  30. Lounis, M.; Bencherit, D.; Rais, M.A.; Riad, A. COVID-19 Vaccine Booster Hesitancy (VBH) and Its Drivers in Algeria: National Cross-Sectional Survey-Based Study. Vaccines 2022, 10, 621. [Google Scholar] [CrossRef]
  31. Miao, Y.; Li, Y.; Zhang, W.; Wu, J.; Gu, J.; Wang, M.; Wei, W.; Ye, B.; Miao, C.; Tarimo, C.S.; et al. The Psychological Experience of COVID-19 Vaccination and Its Impact on the Willingness to Receive Booster Vaccines among the Chinese Population: Evidence from a National Cross-Sectional Study. Int. J. Environ. Res. Public Health 2022, 19, 5464. [Google Scholar] [CrossRef] [PubMed]
  32. Paul, E.; Fancourt, D. Predictors of Uncertainty and Unwillingness to Receive the COVID-19 Booster Vaccine: An Observational Study of 22,139 Fully Vaccinated Adults in the UK. Lancet Reg. Health-Eur. 2022, 14, 100317. [Google Scholar] [CrossRef] [PubMed]
  33. Rababa’h, A.M.; Abedalqader, N.N.; Ababneh, M. Jordanians’ Willingness to Receive Heterologous Prime-Boost COVID-19 Vaccination and Vaccine Boosters. Eur. Rev. Med. Pharmacol. Sci. 2021, 25, 7516–7525. [Google Scholar] [CrossRef] [PubMed]
  34. Rzymski, P.; Poniedziałek, B.; Fal, A. Willingness to Receive the Booster COVID-19 Vaccine Dose in Poland. Vaccines 2021, 9, 1286. [Google Scholar] [CrossRef] [PubMed]
  35. Wang, X.; Liu, L.; Pei, M.; Li, X.; Li, N. Willingness of the General Public to Receive A COVID-19 Vaccine Booster—China, April–May 2021. China CDC Wkly. 2022, 4, 66–70. [Google Scholar] [CrossRef]
  36. Wirawan, G.B.S.; Harjana, N.P.A.; Nugrahani, N.W.; Januraga, P.P. Health Beliefs and Socioeconomic Determinants of COVID-19 Booster Vaccine Acceptance: An Indonesian Cross-Sectional Study. Vaccines 2022, 10, 724. [Google Scholar] [CrossRef]
  37. Wu, F.; Yuan, Y.; Deng, Z.; Yin, D.; Shen, Q.; Zeng, J.; Xie, Y.; Xu, M.; Yang, M.; Jiang, S.; et al. Acceptance of COVID-19 Booster Vaccination Based on the Protection Motivation Theory: A Cross-sectional Study in China. J. Med. Virol. 2022, in press. [Google Scholar] [CrossRef]
  38. Yadete, T.; Batra, K.; Netski, D.M.; Antonio, S.; Patros, M.J.; Bester, J.C. Assessing Acceptability of COVID-19 Vaccine Booster Dose among Adult Americans: A Cross-Sectional Study. Vaccines 2021, 9, 1424. [Google Scholar] [CrossRef]
  39. Yoshida, M.; Kobashi, Y.; Kawamura, T.; Shimazu, Y.; Nishikawa, Y.; Omata, F.; Zhao, T.; Yamamoto, C.; Kaneko, Y.; Nakayama, A.; et al. Factors Associated with COVID-19 Vaccine Booster Hesitancy: A Retrospective Cohort Study, Fukushima Vaccination Community Survey. Vaccines 2022, 10, 515. [Google Scholar] [CrossRef]
  40. Nehal, K.R.; Steendam, L.M.; Campos Ponce, M.; van der Hoeven, M.; Smit, G.S.A. Worldwide Vaccination Willingness for COVID-19: A Systematic Review and Meta-Analysis. Vaccines 2021, 9, 1071. [Google Scholar] [CrossRef]
  41. Galanis, P.; Vraka, I.; Fragkou, D.; Bilali, A.; Kaitelidou, D. Intention of Healthcare Workers to Accept COVID-19 Vaccination and Related Factors: A Systematic Review and Meta-Analysis. Asian Pac. J. Trop. Med. 2021, 14, 543. [Google Scholar] [CrossRef]
  42. Galanis, P.; Vraka, I.; Siskou, O.; Konstantakopoulou, O.; Katsiroumpa, A.; Kaitelidou, D. Willingness, Refusal and Influential Factors of Parents to Vaccinate Their Children against the COVID-19: A Systematic Review and Meta-Analysis. Prev. Med. 2022, 157, 106994. [Google Scholar] [CrossRef] [PubMed]
  43. Robinson, E.; Jones, A.; Lesser, I.; Daly, M. International Estimates of Intended Uptake and Refusal of COVID-19 Vaccines: A Rapid Systematic Review and Meta-Analysis of Large Nationally Representative Samples. Vaccine 2021, 39, 2024–2034. [Google Scholar] [CrossRef] [PubMed]
  44. Zintel, S.; Flock, C.; Arbogast, A.L.; Forster, A.; von Wagner, C.; Sieverding, M. Gender Differences in the Intention to Get Vaccinated against COVID-19: A Systematic Review and Meta-Analysis. J. Public Health 2022, 1–25. [Google Scholar] [CrossRef] [PubMed]
  45. Galanis, P.; Vraka, I.; Siskou, O.; Konstantakopoulou, O.; Katsiroumpa, A.; Kaitelidou, D. Predictors of COVID-19 Vaccination Uptake and Reasons for Decline of Vaccination: A Systematic Review. MedRxiv 2021. [Google Scholar] [CrossRef]
  46. Nguyen, L.H.; Joshi, A.D.; Drew, D.A.; Merino, J.; Ma, W.; Lo, C.-H.; Kwon, S.; Wang, K.; Graham, M.S.; Polidori, L.; et al. Racial and Ethnic Differences in COVID-19 Vaccine Hesitancy and Uptake. MedRxiv 2021. [Google Scholar] [CrossRef]
  47. Schrading, W.A.; Trent, S.A.; Paxton, J.H.; Rodriguez, R.M.; Swanson, M.B.; Mohr, N.M.; Talan, D.A.; Project COVERED Emergency Department Network; Bahamon, M.; Carlson, J.N.; et al. Vaccination Rates and Acceptance of SARS-CoV-2 Vaccination among U.S. Emergency Department Health Care Personnel. Acad. Emerg. Med. 2021, 28, 455–458. [Google Scholar] [CrossRef]
  48. Sakou, I.-I.; Tsitsika, A.K.; Papaevangelou, V.; Tzavela, E.C.; Greydanus, D.E.; Tsolia, M.N. Vaccination Coverage among Adolescents and Risk Factors Associated with Incomplete Immunization. Eur. J. Pediatr. 2011, 170, 1419–1426. [Google Scholar] [CrossRef]
  49. Bish, A.; Yardley, L.; Nicoll, A.; Michie, S. Factors Associated with Uptake of Vaccination against Pandemic Influenza: A Systematic Review. Vaccine 2011, 29, 6472–6484. [Google Scholar] [CrossRef]
  50. Jiménez-García, R.; Hernández-Barrera, V.; de Andres, A.L.; Jimenez-Trujillo, I.; Esteban-Hernández, J.; Carrasco-Garrido, P. Gender Influence in Influenza Vaccine Uptake in Spain: Time Trends Analysis (1995–2006). Vaccine 2010, 28, 6169–6175. [Google Scholar] [CrossRef]
  51. Pulcini, C.; Massin, S.; Launay, O.; Verger, P. Factors Associated with Vaccination for Hepatitis B, Pertussis, Seasonal and Pandemic Influenza among French General Practitioners: A 2010 Survey. Vaccine 2013, 31, 3943–3949. [Google Scholar] [CrossRef]
  52. Flanagan, K.L.; Fink, A.L.; Plebanski, M.; Klein, S.L. Sex and Gender Differences in the Outcomes of Vaccination over the Life Course. Annu. Rev. Cell Dev. Biol. 2017, 33, 577–599. [Google Scholar] [CrossRef] [PubMed]
  53. McLean, C.P.; Anderson, E.R. Brave Men and Timid Women? A Review of the Gender Differences in Fear and Anxiety. Clin. Psychol. Rev. 2009, 29, 496–505. [Google Scholar] [CrossRef]
  54. Biswas, M.R.; Alzubaidi, M.S.; Shah, U.; Abd-Alrazaq, A.A.; Shah, Z. A Scoping Review to Find Out Worldwide COVID-19 Vaccine Hesitancy and Its Underlying Determinants. Vaccines 2021, 9, 1243. [Google Scholar] [CrossRef]
  55. Wake, A.D. The Willingness to Receive COVID-19 Vaccine and Its Associated Factors: “Vaccination Refusal Could Prolong the War of This Pandemic”—A Systematic Review. Risk Manag. Healthc. Policy 2021, 14, 2609–2623. [Google Scholar] [CrossRef] [PubMed]
  56. Dong, D.; Xu, R.H.; Wong, E.L.; Hung, C.; Feng, D.; Feng, Z.; Yeoh, E.; Wong, S.Y. Public Preference for COVID-19 Vaccines in China: A Discrete Choice Experiment. Health Expect. 2020, 23, 1543–1578. [Google Scholar] [CrossRef] [PubMed]
  57. Pogue, K.; Jensen, J.L.; Stancil, C.K.; Ferguson, D.G.; Hughes, S.J.; Mello, E.J.; Burgess, R.; Berges, B.K.; Quaye, A.; Poole, B.D. Influences on Attitudes Regarding Potential COVID-19 Vaccination in the United States. Vaccines 2020, 8, 582. [Google Scholar] [CrossRef]
  58. Kreps, S.; Prasad, S.; Brownstein, J.S.; Hswen, Y.; Garibaldi, B.T.; Zhang, B.; Kriner, D.L. Factors Associated with US Adults’ Likelihood of Accepting COVID-19 Vaccination. JAMA Netw. Open 2020, 3, e2025594. [Google Scholar] [CrossRef]
  59. Seale, H.; Heywood, A.E.; Leask, J.; Sheel, M.; Durrheim, D.N.; Bolsewicz, K.; Kaur, R. Examining Australian Public Perceptions and Behaviors towards a Future COVID-19 Vaccine. BMC Infect. Dis. 2021, 21, 120. [Google Scholar] [CrossRef]
  60. Yang, F.; Li, X.; Su, X.; Xiao, T.; Wang, Y.; Hu, P.; Li, H.; Guan, J.; Tian, H.; Wang, P.; et al. A Study on Willingness and Influencing Factors to Receive COVID-19 Vaccination among Qingdao Residents. Hum. Vaccines Immunother. 2021, 17, 408–413. [Google Scholar] [CrossRef]
  61. Palamenghi, L.; Barello, S.; Boccia, S.; Graffigna, G. Mistrust in Biomedical Research and Vaccine Hesitancy: The Forefront Challenge in the Battle against COVID-19 in Italy. Eur. J. Epidemiol. 2020, 35, 785–788. [Google Scholar] [CrossRef]
  62. Sallam, M.; Dababseh, D.; Eid, H.; Al-Mahzoum, K.; Al-Haidar, A.; Taim, D.; Yaseen, A.; Ababneh, N.A.; Bakri, F.G.; Mahafzah, A. High Rates of COVID-19 Vaccine Hesitancy and Its Association with Conspiracy Beliefs: A Study in Jordan and Kuwait among Other Arab Countries. Vaccines 2021, 9, 42. [Google Scholar] [CrossRef] [PubMed]
  63. Dhanda, S.; Osborne, V.; Lynn, E.; Shakir, S. Postmarketing Studies: Can They Provide a Safety Net for COVID-19 Vaccines in the UK? BMJ Evid.-Based Med. 2022, 27, 1–6. [Google Scholar] [CrossRef] [PubMed]
  64. Mehraeen, E.; Karimi, A.; Barzegary, A.; Vahedi, F.; Afsahi, A.M.; Dadras, O.; Moradmand-Badie, B.; Seyed Alinaghi, S.A.; Jahanfar, S. Predictors of Mortality in Patients with COVID-19–a Systematic Review. Eur. J. Integr. Med. 2020, 40, 101226. [Google Scholar] [CrossRef]
  65. Sepandi, M.; Taghdir, M.; Alimohamadi, Y.; Afrashteh, S.; Hosamirudsari, H. Factors Associated with Mortality in COVID-19 Patients: A Systematic Review and Meta-Analysis. Iran. J. Public Health 2020, 49, 1211–1221. [Google Scholar] [CrossRef] [PubMed]
  66. Ehde, D.M.; Roberts, M.K.; Herring, T.E.; Alschuler, K.N. Willingness to Obtain COVID-19 Vaccination in Adults with Multiple Sclerosis in the United States. Mult. Scler. Relat. Disord. 2021, 49, 102788. [Google Scholar] [CrossRef]
  67. Wang, J.; Jing, R.; Lai, X.; Zhang, H.; Lyu, Y.; Knoll, M.D.; Fang, H. Acceptance of COVID-19 Vaccination during the COVID-19 Pandemic in China. Vaccines 2020, 8, 482. [Google Scholar] [CrossRef]
  68. Williams, L.; Flowers, P.; McLeod, J.; Young, D.; Rollins, L.; The CATALYST Project Team. Social Patterning and Stability of Intention to Accept a COVID-19 Vaccine in Scotland: Will Those Most at Risk Accept a Vaccine? Vaccines 2021, 9, 17. [Google Scholar] [CrossRef]
  69. Callaghan, T.; Moghtaderi, A.; Lueck, J.A.; Hotez, P.; Strych, U.; Dor, A.; Fowler, E.F.; Motta, M. Correlates and Disparities of Intention to Vaccinate against COVID-19. Soc. Sci. Med. 2021, 272, 113638. [Google Scholar] [CrossRef]
  70. Patalon, T.; Gazit, S.; Pitzer, V.E.; Prunas, O.; Warren, J.L.; Weinberger, D.M. Odds of Testing Positive for SARS-CoV-2 Following Receipt of 3 vs 2 Doses of the BNT162b2 MRNA Vaccine. JAMA Intern. Med. 2022, 182, 179. [Google Scholar] [CrossRef]
  71. Bari, M.S.; Hossain, M.J.; Ahmmed, F.; Sarker, M.M.R.; Khandokar, L.; Chaithy, A.P.; Aziz, F.; Mitra, S.; Emran, T.B.; Islam, M.S.; et al. Knowledge, Perception, and Willingness towards Immunization among Bangladeshi Population during COVID-19 Vaccine Rolling Period. Vaccines 2021, 9, 1449. [Google Scholar] [CrossRef]
  72. Šljivo, A.; Ćetković, A.; Abdulkhaliq, A.; Kiseljaković, M.; Selimović, A.; Kulo, A. COVID-19 Vaccination Knowledge, Attitudes and Practices among Residents of Bosnia and Herzegovina during the Third Wave of COVID-19 Outbreak. Ann. Ig. 2021. online ahead of print. [Google Scholar] [CrossRef]
  73. Lu, P.; Kong, D.; Shelley, M. Risk Perception, Preventive Behavior, and Medical Care Avoidance among American Older Adults During the COVID-19 Pandemic. J. Aging Health 2021, 33, 577–584. [Google Scholar] [CrossRef] [PubMed]
  74. Rubin, G.J.; Amlot, R.; Page, L.; Wessely, S. Public Perceptions, Anxiety, and Behaviour Change in Relation to the Swine Flu Outbreak: Cross Sectional Telephone Survey. BMJ 2009, 339, b2651. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  75. Li, X.-H.; Chen, L.; Pan, Q.-N.; Liu, J.; Zhang, X.; Yi, J.-J.; Chen, C.-M.; Luo, Q.-H.; Tao, P.-Y.; Pan, X.; et al. Vaccination Status, Acceptance, and Knowledge toward a COVID-19 Vaccine among Healthcare Workers: A Cross-Sectional Survey in China. Hum. Vaccines Immunother. 2021, 17, 4065–4073. [Google Scholar] [CrossRef] [PubMed]
  76. Ciardi, F.; Menon, V.; Jensen, J.L.; Shariff, M.A.; Pillai, A.; Venugopal, U.; Kasubhai, M.; Dimitrov, V.; Kanna, B.; Poole, B.D. Knowledge, Attitudes and Perceptions of COVID-19 Vaccination among Healthcare Workers of an Inner-City Hospital in New York. Vaccines 2021, 9, 516. [Google Scholar] [CrossRef]
  77. Tadele Admasu, F. Knowledge and Proportion of COVID-19 Vaccination and Associated Factors Among Cancer Patients Attending Public Hospitals of Addis Ababa, Ethiopia, 2021: A Multicenter Study. Infect. Drug Resist. 2021, 14, 4865–4876. [Google Scholar] [CrossRef]
  78. Huynh, G.; Nguyen, T.V.; Nguyen, D.D.; Lam, Q.M.; Pham, T.N.; Nguyen, H.T.N. Knowledge About COVID-19, Beliefs and Vaccination Acceptance Against COVID-19 Among High-Risk People in Ho Chi Minh City, Vietnam. Infect. Drug Resist. 2021, 14, 1773–1780. [Google Scholar] [CrossRef]
Vaccines 10 01097 g001 550
Figure 1. Flowchart of the literature search according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis.
Figure 1. Flowchart of the literature search according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis.
Vaccines 10 01097 g001
Vaccines 10 01097 g002 550
Figure 2. Forest plot of individuals’ willingness to accept a first COVID-19 booster dose. The size of black squares denotes the sample size, while the horizontal line through the square denotes the 95% confidence interval for the willingness proportion.
Figure 2. Forest plot of individuals’ willingness to accept a first COVID-19 booster dose. The size of black squares denotes the sample size, while the horizontal line through the square denotes the 95% confidence interval for the willingness proportion.
Vaccines 10 01097 g002
Vaccines 10 01097 g003 550
Figure 3. Forest plot of individuals’ refusal to accept a first COVID-19 booster dose. The size of black squares denotes the sample size, while the horizontal line through the square denotes the 95% confidence interval for the refusal proportion.
Figure 3. Forest plot of individuals’ refusal to accept a first COVID-19 booster dose. The size of black squares denotes the sample size, while the horizontal line through the square denotes the 95% confidence interval for the refusal proportion.
Vaccines 10 01097 g003
Vaccines 10 01097 g004 550
Figure 4. Forest plot of individuals reporting that they were unsure of their intention to accept a first COVID-19 booster dose. The size of black squares denotes the sample size, while the horizontal line through the square denotes the 95% confidence interval for the unsure proportion.
Figure 4. Forest plot of individuals reporting that they were unsure of their intention to accept a first COVID-19 booster dose. The size of black squares denotes the sample size, while the horizontal line through the square denotes the 95% confidence interval for the unsure proportion.
Vaccines 10 01097 g004
Table
Table 1. Overview of the studies included in this systematic review.
Table 1. Overview of the studies included in this systematic review.
Reference Country Data Collection TimeSample Size (N)Females (%)Age, Mean (Standard Deviation)Study DesignSampling MethodRecruitment MethodResponse Rate (%)Published In
Lounis et al. [30]AlgeriaJanuary to March 202278761.618–40 years, 58.2%;
41–60 years, 36.8%;
>60 years, 5%		Cross-sectionalSnowballOnline surveyNAJournal
Rzymski et al. [34]Poland September 2021242750.7<50 years, 62.3%;
≥50 years, 37.7%		Cross-sectionalSnowballOnline surveyNAJournal
Lai et al. [29]China June 2021114550.3≤40 years, 65.7%;
41–59 years, 34.3%		Cross-sectionalSnowballOnline surveyNAJournal
Al-Qerem et al. [26]JordanOctober to December 2021915NR18–29 years, 45.7%;
≥30 years, 54.3%		Cross-sectionalConvenienceOnline surveyNAJournal
Wu et al. [37]ChinaOctober 2021822969.026–45 years, 78.5%;
≥46 years, 21.5%		Cross-sectionalSnowballOnline surveyNAJournal
Miao et al. [31]ChinaAugust 202126,75552.6<40 years, 83.8%;
≥40 years, 16.2%		Cross-sectionalSnowballOnline surveyNAJournal
Chu et al. [27]Vietnam November 202190025.718–44 years, 91.8%;
≥45 years, 8.2%		Cross-sectionalConvenienceOnline surveyNAJournal
Jørgensen et al. [28]Denmark December 202131,721NRNRCross-sectionalStratified random Online survey25Journal
Wang et al. [35]ChinaApril to May 2021204759.318–44 years, 64.7%;
≥45 years, 35.3%		Cross-sectionalSnowballOnline surveyNAJournal
Rababa’h et al. [33]JordanAugust 20214137618–39 years, 74.6%;
≥40 years, 25.4%		Cross-sectionalConvenienceOnline surveyNAJournal
Paul et al. [32]United KingdomNovember to December 202122,1395118–44 years, 44%;
≥45 years, 56%		Cross-sectionalConvenienceOnline surveyNAJournal
Yadete et al. [38]USAJuly 2021145649.7NRCross-sectionalConvenienceOnline surveyNAJournal
Yoshida et al. [39]Japan September to October 2021243958.3 52.6 (19.3)Cohort ConvenienceOnline surveyNRJournal
Wirawan et al. [36]IndonesiaDecember 2021 to January 202226745829 (24–35) aCross-sectionalConvenienceOnline surveyNAJournal
NA, not applicable; NR, not reported. a median (interquartile range).
Table
Table 2. Response scales and results of individuals’ willingness to accept a first COVID-19 booster dose in studies included in systematic review.
Table 2. Response scales and results of individuals’ willingness to accept a first COVID-19 booster dose in studies included in systematic review.
Reference Question/Statement to Measure Patients’ WillingnessResponse Scale aWillingness Results (%)
Lounis et al. [30]Are you willing to receive the potential additional dose of the COVID-19 vaccine if it would be made available?Yes (Y), unsure (U), no (N)Yes: 51.6
Unsure: 23.4
No: 25
Rzymski et al. [34]Are you willing to receive the potential additional dose of the COVID-19 vaccine if it would be made available?Yes (Y), unsure (U), no (N)Yes: 71.0
Unsure: 4.3
No: 24.7
Lai et al. [29]If a COVID-19 booster is recommended as a supplement to the current vaccination schedule, would you accept it?Yes (Y), no (N)Yes: 84.8
No: 15.2
Al-Qerem et al. [26]Are you willing to take the booster dose?Yes (Y), unsure (U), no (N)Yes: 44.6
Unsure: 24.7
No: 30.7
Wu et al. [37]To what extent do you want to take the booster COVD-19 vaccine?Yes, definitely (Y), unsure but tend to be willing (U), unsure but tend to be unwilling (U), definitely no (N)Yes: 76.8
Unsure: 22.2
No: 1.0
Miao et al. [31]Are you willing to take the booster dose?Very willing (Y), willing (Y), fair (U), unwilling (U), very unwilling (U), don’t know (U)Yes: 93.8
Unsure: 6.2
Chu et al. [27]If an extra dose of COVID-19 vaccine is available, would you get it?Definitely yes (Y), probably yes (Y), probably no (N), definitely no (N)Yes: 93.7
No: 6.3
Jørgensen et al. [28]Will you accept the booster vaccine?I have received the booster dose (Y), I have not yet received the invitation to the booster dose, but I wish to be vaccinated with the booster dose (Y), I have received the invitation to the booster dose, and I wish to be vaccinated, but have not yet been vaccinated with the booster dose (Y), I have received an invitation to the booster dose, but a do not wish the booster dose (N), I have not yet received the invitation to the booster dose, and I do not wish to be vaccinated with the booster dose (N), Do not want to answer (N)Yes: 95.5
No: 4.5
Wang et al. [35]Are you willing to receive the potential additional dose of the COVID-19 vaccine if it would be made available?Yes (Y), no (N)Yes: 75.2
No: 24.8
Rababa’h et al. [33]Do you accept receiving the COVID-19 booster vaccine?Yes (Y), unsure (U), no (N)Yes: 54.5
Unsure: NR
No: NR
Paul et al. [32]How likely do you think you are to get a COVID-19 booster vaccine if/when you are offered one?A scale from 1 (very unlikely) to 6 (very likely); 5–6 (Y), 3–4 (U), 1–2 (N)Yes: 92.3
Unsure: 4.1
No: 3.6
Yadete et al. [38]Do you accept receiving the COVID-19 booster vaccine?Yes (Y), no (N)Yes: 79.1
No: 20.9
Yoshida et al. [39]Do you accept receiving the COVID-19 booster vaccine?Yes (Y), no (N)Yes: 97.9
No: 2.1
Wirawan et al. [36]Will you accept the booster vaccine?I have received the booster dose (Y), A scale from 1 (would not accept) to 5 (certainly would accept); 5 (Y), 1–4 (N)Yes: 56.3
No: 43.7
NR: not reported. a (Y), (N) and (U) indicate extracted response options representing yes, no and unsure in this meta-analysis.
Table
Table 3. Studies examining factors related to individuals’ willingness to accept a first COVID-19 booster dose.
Table 3. Studies examining factors related to individuals’ willingness to accept a first COVID-19 booster dose.
Reference Older AgeMales Married Higher Educational LevelEthnicityHigher IncomeHealthcare WorkersResidence Chronic ComorbiditiesHistory of COVID-19 InfectionHospitalization Higher Risk Perception of InfectionHigher Severity Perception of COVID-19
Lounis et al. [30]↑↑NS↓--↓NS↑↑NS--
Rzymski et al. [34]↑↓-NS---NS↑↑---
Lai et al. [29]↓NSNS↓---NSNS↑-NSNS
Al-Qerem et al. [26]NS--NS---NS---NSNS
Wu et al. [37]↑-----↑----↑↓
Miao et al. [31]↑↓↑↑---NS↓----
Chu et al. [27]NSNSNS↑NSNSNSNSNSNS---
Jørgensen et al. [28]↑NS-NS--------↑
Wang et al. [35]↓↑-↓-NS↓NS-----
Paul et al. [32]↑NSNS↑NS↑NSNS↑NS-↑↑
Yoshida et al. [39]↑NS-----------
Wirawan et al. [36]↑NS-↑-↑---NS-↑↑
Positive association a8/122/101/54/100/22/41/50/83/63/60/13/53/6
Negative association b2/122/100/53/100/20/42/50/81/60/60/10/51/6
No association c2/126/104/53/102/22/42/58/82/63/61/12/52/6
ReferenceInfection in FamilyMortality in FamilyInfluenza VaccineConfidence in Healthcare System/Government/PhysiciansConfidence in COVID-19 VaccinesConfidence in COVID-19 Booster DosesCompliance with Prevention MeasuresAdverse Reactions and Discomfort Experienced after Previous DosesConcerns for Serious Adverse Reactions to COVID-19 Booster Doses
Lounis et al. [30]NSNSNS↑↑↑---
Rzymski et al. [34]--↑--↑-↓↓
Lai et al. [29]-----↑--↓
Miao et al. [31]---↑---↓-
Wang et al. [35]--↑-↑----
Paul et al. [32]---NS--↑--
Al-Qerem et al. [26]-------↓-
Wu et al. [37]-------↓-
Wirawan et al. [36]NS----↑---
Positive association a0/20/12/32/32/24/41/10/40/2
Negative association b0/20/10/30/30/20/40/14/42/2
No association c2/21/11/31/30/20/40/10/40/2
ReferenceDesire to Travel AbroadHarm in Immune SystemFurther Vaccination Is UnnecessaryLow Safety of COVID-19 Booster DosesKnowledge LevelInitial Uncertainty and Unwillingness to Accept the First COVID-19 Vaccine
Lounis et al. [30]↑↓----
Rzymski et al. [34]--↓---
Lai et al. [29]---↓--
Al-Qerem et al. [26]----NS-
Paul et al. [32]----NS↓
Positive association a1/10/10/10/10/20/1
Negative association b0/11/11/11/10/21/1
No association c0/10/10/10/12/20/1
a number of studies with a positive significant association (p-value < 0.05) between the predictor and individuals’ willingness to accept a first COVID-19 booster dose/total number of studies that examined the predictor. b number of studies with a negative significant association (p-value < 0.05) between the predictor and individuals’ willingness to accept a first COVID-19 booster dose/total number of studies that examined the predictor. c number of studies without a significant association (p-value ≥ 0.05) between the predictor and individuals’ willingness to accept a first COVID-19 booster dose/total number of studies that examined the predictor. NS: non-significant. ↑ more likely to accept. ↓ less likely to accept. - not investigated.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Galanis, P.; Vraka, I.; Katsiroumpa, A.; Siskou, O.; Konstantakopoulou, O.; Katsoulas, T.; Mariolis-Sapsakos, T.; Kaitelidou, D. First COVID-19 Booster Dose in the General Population: A Systematic Review and Meta-Analysis of Willingness and Its Predictors. Vaccines 2022, 10, 1097. https://doi.org/10.3390/vaccines10071097

AMA Style

Galanis P, Vraka I, Katsiroumpa A, Siskou O, Konstantakopoulou O, Katsoulas T, Mariolis-Sapsakos T, Kaitelidou D. First COVID-19 Booster Dose in the General Population: A Systematic Review and Meta-Analysis of Willingness and Its Predictors. Vaccines. 2022; 10(7):1097. https://doi.org/10.3390/vaccines10071097

Chicago/Turabian Style

Galanis, Petros, Irene Vraka, Aglaia Katsiroumpa, Olga Siskou, Olympia Konstantakopoulou, Theodoros Katsoulas, Theodoros Mariolis-Sapsakos, and Daphne Kaitelidou. 2022. "First COVID-19 Booster Dose in the General Population: A Systematic Review and Meta-Analysis of Willingness and Its Predictors" Vaccines 10, no. 7: 1097. https://doi.org/10.3390/vaccines10071097

APA Style

Galanis, P., Vraka, I., Katsiroumpa, A., Siskou, O., Konstantakopoulou, O., Katsoulas, T., Mariolis-Sapsakos, T., & Kaitelidou, D. (2022). First COVID-19 Booster Dose in the General Population: A Systematic Review and Meta-Analysis of Willingness and Its Predictors. Vaccines, 10(7), 1097. https://doi.org/10.3390/vaccines10071097

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop