Next Article in Journal
World Experience in Immunization against Noncommunicable Diseases: Successes and Vectors for Further Development
Previous Article in Journal
SARS-CoV-2 m-RNA Vaccine Response in Immunocompromised Patients: A Monocentric Study Comparing Cancer, People Living with HIV, Hematopoietic Stem Cell Transplant Patients and Lung Transplant Recipients
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Childhood Vaccine Attitude and Refusal among Turkish Parents

by
Osman Kurt
1,
Osman Küçükkelepçe
1,*,
Erdoğan Öz
1,
Hülya Doğan Tiryaki
2 and
Mehmet Emin Parlak
3
1
Adiyaman Provincial Health Directorate, 02100 Adıyaman, Turkey
2
Gaziantep Provincial Health Directorate, 27310 Gaziantep, Turkey
3
Adıyaman Training and Research Hospital, 02100 Adıyaman, Turkey
*
Author to whom correspondence should be addressed.
Vaccines 2023, 11(8), 1285; https://doi.org/10.3390/vaccines11081285
Submission received: 9 June 2023 / Revised: 3 July 2023 / Accepted: 24 July 2023 / Published: 26 July 2023

Abstract

:
We aimed to understand and resolve anti-vaccine attitudes by examining the factors associated with vaccine attitudes and exploring potential strategies to improve childhood vaccination rates. Between 2014 and 2021, a total of 628 families refused vaccination in Adiyaman. A total of 300 families accepted visits and were visited. During the visits, the families were administered a questionnaire to determine the reasons for vaccine rejection and their opinions on the matter. While providing general information about the vaccine, parents were encouraged to reconsider their decision, and at the end, parents completed the questionnaire. The questionnaire included sociodemographic questions, reasons for vaccine refusal, and a vaccine attitude scale. Among the participants in the study, 9.3% were convinced about the vaccine. The mean vaccine attitude scale score was calculated as 23.6 ± 2.5 (min = 15–max = 29). Significantly higher rates of persuasion were observed among fathers (17.3%) compared to mothers (7.7%) (p = 0.038). Participants who had received some vaccinations had a higher rate of persuasion (11.6%) compared to those who had not received any vaccinations (2.6%) (p = 0.02). Childhood vaccine refusal is a complex issue that has been the subject of numerous studies. Studies on this subject will increase awareness of vaccines.

1. Introduction

Despite the availability of safe and effective vaccines, vaccine refusal remains a significant public health concern. Parental vaccine hesitancy can be a barrier to routine childhood immunization and contribute to a greater risk of vaccine-preventable diseases [1]. Factors such as lower educational level, lower household income, and distrust of the medical profession have been associated with vaccine hesitancy [2]. The COVID-19 pandemic has further complicated the issue, with some households reporting increased vaccine hesitancy [3]. To address this issue, it is important to understand the factors contributing to vaccine hesitancy/refusal and explore potential strategies to improve childhood vaccination rates [4].
In Turkey, vaccine refusal has increased over the past decade. Previously, cases of vaccination refusal were very rare but have increased rapidly since 2015 after the victory of a lawsuit for “obtaining parental consent for vaccination”, as well as frequent arguments against vaccination in the media. The number of families who refused to vaccinate their children was 183 in 2011, which increased to 980 in 2013, 5400 in 2015, 12,000 in 2016, and 23,000 in 2018 [5].
Several studies have been conducted to investigate vaccine attitudes in Turkey, particularly in relation to the COVID-19 vaccine. These studies provide insights into the levels of vaccine hesitancy, beliefs about the origin of the virus, and factors influencing vaccine acceptance among different populations in Turkey. One study by Salali and Uysal aimed to examine the levels of COVID-19 vaccine hesitancy and its association with beliefs on the origin of the novel coronavirus in a cross-cultural study between the UK and Turkey. The study conducted an online survey in Turkey with 3936 participants and found that 31% of the participants were unsure about getting vaccinated with the COVID-19 vaccine [6]. Another study by Bendau et al. explored the predictors of vaccine acceptance, including anxiety, fear, and individual risk perception. The study found that individuals with higher risk perception and more anxiety showed significantly higher vaccine acceptance in Turkey [7]. Karagöz and Yalman conducted a study to evaluate the attitudes, hesitancy, and confidence of healthcare professionals towards COVID-19 vaccines in Turkey. The study used a hospital-based cross-sectional research design and surveyed health workers from three different training and research hospitals in Istanbul. The findings revealed that nearly two-thirds of the participants had a positive attitude towards potential COVID-19 vaccines, which was associated with higher vaccine confidence and less hesitancy [8]. A study by Kaplan et al. focused on healthcare professionals and their attitudes towards the COVID-19 vaccine. The study found that about 84.6% of healthcare professionals in Turkey declared their willingness to accept the COVID-19 vaccine whenever possible. Factors influencing vaccine acceptance among healthcare professionals included advanced age, male gender, working in a primary health care center, living with family, having a child, having a chronic disease, and a higher fear of COVID-19 [9]. Topçu et al. conducted a study with mothers who refused vaccination in two different hospitals [10].
Overall, these studies provide valuable insights into vaccine attitudes in Turkey, highlighting the levels of vaccine hesitancy, the association between attitudes and vaccine acceptance, and the factors influencing vaccine attitudes among different populations, including healthcare professionals. Understanding these attitudes and factors is crucial for designing effective vaccination campaigns and addressing vaccine hesitancy in Turkey. The present study aims to obtain information about vaccine attitudes and evaluate the persuasion rate after people with vaccination hesitations are informed about vaccine attitudes. The present study aimed to determine the vaccination attitude, refusal, and persuasion rates according to the patient’s statements and objectively and definitively by examining the records.

2. Materials and Methods

2.1. Study Design and Location

This descriptive study was conducted in the Adiyaman province, located in the Southeastern Anatolia region of Turkey.

2.2. Study Population

The study population consisted of parents who had refused vaccination for their children by filling out vaccine rejection forms in Adiyaman. Between 2014 and 2021, a total of 628 families refused vaccination. The aim was to reach all of these families without using a sampling method. According to Turkish Statistical Institute data, the total number of births in Adiyaman province between 2014 and 2021 was 99,637. According to the sampling method with a 95% confidence interval, a non-response rate of 20% (it was thought that the non-response rate would be high in vaccine refusal), and a prevalence of vaccine refusal of 50% (the vaccine refusal rate was accepted as such since the vaccine refusal rate is not known precisely in the literature and the refusal rate is gradually increasing), it was seen that at least 306 families should be reached. All families who refused childhood vaccinations and filled out a refusal form were included in the study. Families with vaccine refusals who were visited at home were also included. Those who filled out a vaccine refusal form and were later convinced by the family physician’s advice and those who did not accept the home visit when called by phone were excluded. Initially, 628 families with vaccine refusals were called by telephone, and 615 could be reached by telephone. We told the families we contacted that we wanted to visit their homes, and the homes of 300 families who agreed were visited. All of the families were visited in their homes, and an interview was conducted with the parent at home for 30–60 min, depending on the parent’s compliance (Figure 1).

2.3. Study Instrument

The primary dependent variable in the study is whether or not the respondent is convinced about vaccination and vaccination behavior. The independent variables were age, interviewee, educational status, occupation, number of children, presence of children under one year of age, vaccination behavior, vaccination attitude, and reasons for not vaccinating. The visitors included general practitioners, nurses working in the directorate, and public health specialists among the authors.
Visits were made between 1 September 2022 and 31 March 2023. During the visits, the families were administered a questionnaire to determine the reasons for vaccine rejection and their perceptions on the matter. While providing general information about the vaccine, parents were encouraged to reconsider their decision, and at the end, parents completed the questionnaire. The questionnaire included sociodemographic questions, reasons for vaccine refusal, and a vaccine attitude scale. Families who were convinced to receive vaccinations were promptly supported in getting their vaccinations. Six months after the initial application, these families were contacted by phone to determine that they had vaccinated their children. In order to determine whether they were convinced about the vaccination, not only were their statements taken as a reference, but it was also determined through the information processing system that these families had been vaccinated.
The vaccine attitude scale (VAS), developed by Wallace et al., was used in this study. The scale’s Turkish validity and reliability study was conducted by Ceylan et al. [11,12]. It consists of 11 items on a 3-point Likert-type scale that measures parents’ attitudes towards vaccines. The scale covers four domains: “benefits of the vaccine” (1 and 2 items), “past vaccination behavior” (3 and 4 items), “efficacy and safety” (5 and 6 items), and “awareness of vaccine-preventable diseases” (7 and 8 items). It also includes a “confidence” sub-dimension (items 9, 10, and 11). Each item on the vaccine attitude scale is scored as agree = 1, undecided = 2, and disagree = 3. The total score is calculated by summing the individual item scores. The total score ranges from 11 to 33 points, where 11 indicates a positive attitude towards vaccination and 33 indicates a negative attitude. The scale does not have a specific cutoff point. Parents’ hesitancy towards vaccination increases as the score increases, indicating a negative attitude [12].

2.4. Ethics

The study obtained ethics committee approval from the Non-Interventional Research Ethics Committee of Firat University, with a decision dated 1 September 2022, and numbered 22 October 2022. Institutional permission was also obtained from the Adiyaman Provincial Health Directorate to conduct the research. Written and verbal consent was obtained from all participants, and the study followed the principles outlined in the Declaration of Helsinki.

2.5. Statistical Analysis

The statistical analysis was performed using the SPSS (Statistical Package for Social Sciences) software, version 22 (SPSS Inc., Chicago, IL, USA). Descriptive data were presented as frequencies and percentages for categorical variables and mean ± standard deviation (Mean ± SD) for continuous variables. The chi-square analysis (Pearson’s chi-square) was used to compare categorical variables between groups. The Kolmogorov–Smirnov test was employed to assess the normal distribution of continuous variables. The Mann–Whitney U-test was used to compare paired groups, while the Kruskal–Wallis test was used for comparisons involving more than two variables. The Spearman correlation test was conducted to examine the relationship between continuous variables. Statistical significance was set at p < 0.05.

3. Results

The study results showed that 300 participants, with a mean age of 34.2 ± 5.8 years (range: 21–53), were included in the analysis. Interviews were conducted with fathers in 17.3% of the cases and mothers in 82.7%. Regarding the participants’ education level, 8% had graduated primary/secondary school, 51.3% had graduated high school, and 40.7% had a university degree. In terms of occupation, 60% of the participants were homemakers. Regarding the number of children, 14% had one child, 35.7% had two children, and 50.3% had three or more children. Among the participants, 17% had children under the age of one.
Regarding the participants’ vaccination attitudes, 12.3% considered vaccines necessary, and 88% had vaccinated at least one of their children previously. Furthermore, 41.7% reported experiencing a problem related to vaccines. Regarding compliance with the Ministry’s recommended vaccines, 74.7% of the participants stated that their children had received some vaccinations, while 25.3% reported having not received any of the recommended vaccines (Table 1).
When participants were asked about the potential harms of vaccinating their children, the following responses were obtained: 39% believed that their children would “get sick more quickly”, 32% thought they would “contract deadly infectious diseases”, 29.3% expressed concerns about their children “getting sick often”, 23% believed their children would experience “a more severe prognosis of illness”, 14% stated that their children might become disabled after contracting a contagious disease, and 20.3% expressed concerns about other potential damages (Figure 2).
Among the participants included in the study, 89% stated that vaccines had side effects, 6% stated that they did not, and 5% were unsure. The most commonly reported side effects were pain (63.7%), fever (58.4%), infection (53.9%), rash (53.6%), infertility (26.6%), and other side effects (21.7%). Additionally, 1.5% of the participants were unsure if vaccines had any effects (Figure 3).
Among the study participants, 29.7% identified themselves as midwives/nurses/doctors; 76.3% were neighbors/friends; and 79.7% mentioned the media/press as their sources of information.
When asked about the reasons for not getting vaccinated, 99% said they did not believe the vaccines were safe or had concerns about their side effects. Additionally, 71.7% believed that vaccination was not necessary, 61% expressed doubts about the effectiveness of vaccines in protecting against diseases, 50.3% were concerned about the harmful chemicals in vaccines, 44.3% mentioned hearing about a child experiencing an adverse reaction after vaccination from another person, 36.7% attributed their decision to negative information they read or heard in the media about vaccines, 23.7% had negative experiences with healthcare personnel or institutions related to vaccination, and 8% cited religious reasons for not getting vaccinated (Table 2).
Among the participants in the study, 9.3% were convinced about receiving vaccines for their children. The mean vaccine attitude scale score was calculated as 23.6 ± 2.5 (min = 15 − max = 29).
Significantly higher rates of persuasion were observed among fathers (17.3%) compared to mothers (7.7%) (p = 0.038). Moreover, fathers had a significantly higher mean vaccine attitude scale score than mothers (p = 0.007).
Regarding education level, 25% of those with a primary/secondary education, 7.1% with a high school education, and 9% with a university education were convinced. A significant difference was observed among those with a primary/secondary education compared to other groups (p = 0.02).
Occupational groups also showed a significant difference in vaccine attitude scale scores (p < 0.001), primarily driven by lower scores among homemakers compared to other groups.
Regarding the number of children, 19% of those with one child, 4.7% with two children, and 9.9% with three or more children were convinced. The difference between those with one child and those with two children was significant (p = 0.024).
Participants with children under the age of one had a higher rate of persuasion (17.6%) compared to those without children under one (7.6%) (p = 0.034).
Regarding the perception of vaccine necessity, 21.6% of those who considered vaccination necessary, 7.6% of those who did not, and 7.1% of those who were undecided were convinced. A significant difference in persuasion was observed between those who considered vaccination necessary and the other two groups (p = 0.031).
Participants who had experienced problems with vaccines in the past had a lower rate of persuasion (4.5%) compared to those who had no problems (13.6%) (p = 0.015).
Regarding the belief in vaccine side effects, 7.1% of those who believed in side effects, 44.4% of those who did not, and 6.7% of those who were unsure were convinced. A significant difference was observed between these groups, primarily driven by the difference between those who believed they had no side effects and the other two groups (p < 0.001).
There was also a significant difference in vaccine attitude scale scores based on belief in side effects, with higher scores among those who believed in side effects than those who did not (p = 0.044).
Participants who had received some vaccinations had a higher rate of persuasion (11.6%) compared to those who had not received any vaccinations (2.6%) (p = 0.02) (Table 3).
The VAS score (p < 0.001) and age (p = 0.008) of those who were persuaded were found to be significantly lower than those who were not convinced (Table 4).
A significant positive correlation was observed between VAS and age, and it was determined that the VAS score increased as the age increased (r = 0.222; p < 0.001) (Figure 4).

4. Discussion

According to various studies, parents who express vaccine hesitancy or rejection due to the side effects of vaccines report a range of concerns. Some parents believe that vaccines can cause severe illness or pose a risk to their child’s health [13]. Others express concerns about the safety and potential side effects of vaccines, including seizures, myocarditis, stroke, and multisystem inflammatory syndrome in children (MIS-C) [14,15]. Additionally, some parents report concerns about the usefulness of vaccines and a lack of knowledge about recommended vaccinations [16,17]. In the present study, when parents who were hesitant about the vaccine were asked about the harms of the vaccine, 53% of the parents said that they did not know the harms of the vaccine, while the others gave different answers. After families are informed about the necessity of vaccines, they should be informed about their potential harm.
Based on the references provided, parents who express vaccine hesitancy or rejection due to the side effects of vaccines report a range of concerns. Some parents express concerns about the serious side effects of both routine childhood and influenza vaccines [18]. Others express concerns about the side effects of COVID-19 vaccines, including unknown effects and potential serious side effects [13]. In the present study, when parents who refused vaccination were asked about the vaccine’s side effects, the parents were most frequently concerned about side effects such as pain, fever, local redness, and infertility, respectively. The most frequently reported side effects are common and relatively simple, such as fever and pain. Health professionals should inform families that these side effects can be treated with simple medication if necessary and that this is not a situation to be feared.
Based on the references provided, the reasons for parents with childhood vaccine hesitancy or refusal not to get vaccinated include:
-
Safety concerns: Some parents express concerns about the safety of vaccines and their potential side effects [14].
-
Lack of need: Some parents may not see the need for vaccines or believe that their child is not at risk for vaccine-preventable diseases [19].
-
Negative media information: Exposure to negative information about vaccines through social media or other sources can contribute to vaccine hesitancy [20].
-
Fear of adverse events: Some parents may be hesitant to vaccinate their child due to the fear of adverse events following immunization (AEFIs) [21].
-
Time constraints: Parents may have difficulty finding time to take their child for vaccinations due to work schedules or other commitments [13].
-
Special physical conditions: Some parents may believe that their child’s physical condition is not suitable for vaccination [13].
-
Beliefs about risk: Some parents may believe that their child is not at risk for vaccine-preventable diseases or that the risk of severe illness from the disease is low [13].
-
Medical mistrust: Some parents may have a general mistrust of medical establishments or healthcare providers [22].
-
Cultural or religious beliefs: Some parents may have cultural or religious beliefs that conflict with vaccination [23].
In the present study, when parents who were hesitant about the vaccine were asked about the reasons, the most common reason was the vaccine’s side effects and the fact that the vaccine was not considered necessary, and the least common reason was religious reasons.
In 2019, a two-center study conducted in Turkey showed that vaccine refusal rates decreased as the education level of the mother and father increased. The same study found an inverse correlation between income level and vaccine refusal rate [10]. In our study, the persuasion rate of families with primary school graduates was significantly higher than the others. In another study conducted in Turkey in 2022, no correlation was found between educational status, income, and vaccination attitude/hesitancy [20]. The literature has drawn different conclusions on how education and income affect vaccination behavior and refusal [24].
In studies conducted in Turkey and worldwide, different determinations were made about vaccination attitudes according to gender. A study conducted in 2022 with nursing students in Turkey showed that males had a more negative vaccine attitude [25]. In a study conducted in 2022 in Turkey, no difference was observed in vaccination attitudes according to gender [24]. In our study, fathers were more convinced to vaccinate their children than mothers. If more studies on this subject are conducted and it is determined that fathers can be convinced more, information and persuasion programs that will appeal more to fathers about vaccine refusal can be implemented.
Comparing our study’s findings with those of other studies reveals some similarities and differences in the relationship between the age of parents and vaccine acceptance. In a study conducted by Soysal and Akdur, the frequency of factors related to vaccine hesitancy and refusal among parents of children under five was examined [23]. They found vaccine hesitancy was higher among parents over 42 years of age. This finding contrasts with our study, which suggests that as parents’ ages decrease, the rate of being convinced about the vaccine increases. It is important to note that this study focused on parents of children under five, which may explain the difference in findings. A study by Lu et al. explored parental attitudes toward vaccination against COVID-19 in China. They found that parents’ willingness to accept vaccinations for their children increased with the children’s age. It indicates that parental attitudes towards vaccination may vary depending on the age of their children [26]. In the study by Loncarevic et al., parental knowledge and attitudes toward the MMR vaccine in Serbia were examined. While their study did not specifically focus on parents’ age, they found that getting information on vaccination from a pediatrician was associated with an increased probability of vaccinating a child. This suggests that healthcare providers, including pediatricians, play a crucial role in shaping parental attitudes toward vaccination [27]. Comparing the findings from our study with other studies in the literature reveals some similarities and differences. While some studies did not directly examine parents’ ages, they provided insights into parental vaccine hesitancy and acceptance. The relationship between parents’ age and vaccine acceptance may vary depending on the specific context and population studied. Factors such as the type of vaccine, the age of the child, and the role of healthcare providers can influence parental attitudes toward vaccination.
The literature provides some insights into the vaccination attitudes of homemakers, although the specific focus on homemakers may be limited. In a study by Abedin et al., which examined the willingness to vaccinate against COVID-19 among Bangladeshi adults, it was reported that homemakers had a high level of resistance to being vaccinated against COVID-19 [28]. This finding aligns with our study, which suggests that homemakers have higher vaccination hesitancy and refusal. It indicates that homemakers may exhibit more reluctance toward vaccination than other groups. A systematic review by Tamara et al. investigated the refusal of the COVID-19 vaccination and its associated factors. The review reported that females, including homemakers, lacked literacy regarding COVID-19 vaccination compared to males [29]. This finding suggests that homemakers may have less knowledge and information about COVID-19 vaccination, which could contribute to their vaccination hesitancy. Another study by Bukhari et al. (2020) examined the effectiveness of health education messages in improving tetanus health literacy among women of childbearing age. While the study did not specifically focus on homemakers, it provides insights into the impact of health education messages on women’s health literacy. The findings indicated that the health education message was more effective for students and employees than for homemakers [30]. This suggests that targeted health education interventions may be necessary to improve vaccination attitudes and knowledge among homemakers. In parallel to our research, the available literature provides some insights into the vaccination attitudes of homemakers. While some studies focused on homemakers, others indirectly addressed their attitudes through related topics. We suggest that homemakers may exhibit higher vaccination hesitancy and resistance, potentially due to limited access to information, engagement with lifestyle influencers, and lower health literacy. However, further research explicitly targeting the vaccination attitudes of homemakers is needed to gain a more comprehensive understanding of this population’s perspectives.
The finding from our study, which suggests that parents with children under one year of age are more likely to be persuaded for childhood vaccination, aligns with some findings in the literature. A systematic review by Schmid et al. identified barriers to influenza vaccination uptake or intention. While the study focused on influenza vaccination, it provides insights into parental decision-making for childhood vaccination. The review found that parental decisions for high-risk children were under-represented in the literature, indicating a gap in understanding this specific population [31]. This suggests that limited research may examine the vaccination decisions of parents with children under one year of age. Further research is needed to better understand the factors influencing parental decision-making for childhood vaccination, particularly for children under one year of age.
The finding from our study, which suggests that people who think that vaccination is necessary but are hesitant about it are more likely to be convinced than people who think that vaccination is not necessary, is supported by some findings in the literature. A study by Brewer et al. discusses the concept of “reluctant but persuadable” individuals who hold ambivalent or unfavorable beliefs about a vaccine but can be convinced. The study highlights the challenges and limited effectiveness of persuasive communications in changing the beliefs and attitudes of these individuals [32]. This aligns with the finding that hesitant individuals who recognize the necessity of vaccination may be more open to persuasion. In a randomized controlled trial by Freeman et al., different types of written information about COVID-19 vaccinations were tested to increase vaccine acceptance. The study found that providing information about the pandemic’s collective benefit, personal benefit, seriousness, and safety concerns increased vaccine acceptance. This suggests that addressing the reasons for hesitancy and providing relevant information can effectively convince hesitant individuals [33]. The literature supports the idea that individuals who acknowledge the necessity of vaccination but are hesitant may be more open to persuasion.
The finding from our study, which suggests that those who had previously experienced problems during vaccination had lower rates of persuasion, is supported by some findings in the literature. A study by Baldwin et al. focused on promoting adolescent HPV vaccination among parents attending safety-net clinics. The study found that many parents remained undecided or ambivalent about the vaccine, even with a provider’s recommendation [34]. This indicates that previous experiences or concerns may contribute to vaccine hesitancy and lower persuasion rates. In another study by Argyris et al., the authors used machine learning to compare pro- and anti-vaccine discourses on social media. The study aimed to understand the attempts made by advocates to influence the public’s acceptance or rejection of immunization [35]. This suggests that previous negative experiences or concerns about vaccines may shape individuals’ attitudes and beliefs, leading to lower persuasion rates. Zhou et al. conducted a multicenter survey among Chinese medical students to examine HPV vaccine hesitancy. The study found that convenience was the primary factor in vaccine hesitancy, and many students relied on a single source for vaccine information [36]. This suggests that previous negative experiences or concerns may contribute to vaccine hesitancy and lower persuasion rates among medical students. The literature supports the finding that previous negative experiences during vaccination can lead to lower persuasion rates, paralleling our study. Understanding these factors and addressing them through targeted interventions and communication strategies can help improve vaccine acceptance and address concerns related to previous negative experiences.
The finding from our study, which suggests that parents who were partially vaccinated were more likely to be convinced than those who did not vaccinate at all, is supported by some findings in the literature. Ellithorpe et al. conducted a study in the United States to compare vaccine acceptance behavior across different groups, including those who were fully accepting of the recommended vaccination schedule, those who were accepted but on a delayed schedule, those who were only partially vaccinated, and those who did not vaccinate at all. The study found apparent differences between the vaccination behavior groups, with financial and insurance-related barriers hindering complete vaccination [37]. This suggests that partially vaccinated individuals may be more open to accepting further vaccination compared to those who did not vaccinate at all, paralleling our study. An et al. conducted a study in China and found that parental vaccine hesitancy led to a refusal to use COVID-19 and influenza vaccines. The study also found that parents were partially hesitant to vaccinate their children with COVID-19 [34]. This suggests that individuals who have had some level of vaccination may be more open to accepting further vaccination than those who have not been vaccinated, which is also similar to our study. Raof conducted a study in Iraq and found that a higher percentage of children were fully vaccinated among parents with lower vaccine hesitancy scores than among parents with higher scores [38]. This indicates that parents who were partially hesitant about vaccination were more likely to have their children fully vaccinated than more hesitant parents. While the literature provides some support for the finding that parents who were partially vaccinated are more likely to be convinced than those who did not vaccinate at all.
Based on the literature provided, the post-training vaccine acceptance rates vary across different studies and populations. Hallas et al. investigated targeting vaccine-hesitant prenatal women and mothers of newborns and found that 82% of prenatal vaccine-hesitant women had full prenatal vaccination coverage after receiving the intervention [39]. Sitaresmi et al. (2020) investigated HPV vaccination in Indonesia and reported improved parental awareness, knowledge, perception, and acceptability of HPV vaccination after the intervention [40]. Nesiama et al. (2022), who conducted a survey among personnel associated with Wright–Patterson Air Force Base, found that 64% of participants remained hesitant following a series of seminars [41]. Comparing these findings with the 9.3% post-training vaccine acceptance rate reported in our research, it falls within the range of vaccine acceptance rates reported in the literature. However, it is essential to note that the literature includes studies on various vaccines, populations, and interventions, which may contribute to the variability in acceptance rates. Additionally, the specific details of the training provided in our research and the characteristics of the study population also influence the observed acceptance rate.
Based on the references provided, there is a relationship between parents with childhood vaccine hesitancy/refusal, their persuasion to get vaccinated, and the vaccine attitude scale. Studies have identified a spectrum of parent attitudes or “positions” on childhood vaccination, with estimates of the proportion of each group based on population studies [42]. Some studies have found no evidence of an association between parental vaccine hesitancy and adolescent vaccine uptake [43]. Parents who were hesitant about childhood vaccinations had lower positive attitudes toward the COVID-19 vaccine than parents who were not hesitant [20]. Perceived financial well-being had significant and negative associations with parents’ attitudes toward COVID-19 vaccines and child vulnerability [44]. Mistrust of the medical profession and higher disgust sensitivity predict parental vaccine hesitancy [45]. Overall, understanding the relationship between parents with childhood vaccine hesitancy/refusal, their persuasion to get vaccinated, and the vaccine attitude scale is crucial in developing strategies to improve childhood vaccination rates and promote public health. In our study, parallel to the literature, a correlation was shown between the vaccination attitude scale and persuasion.
Since our study was conducted in a narrow geographical area, sociocultural factors may have come to the fore. As the study area is relatively rural, results may differ in urban areas. Since the home visits were conducted during working hours, only one of the parents could be interviewed, and the interviewed parent was generally the mother since the working population in Turkish society is primarily male. The study’s results may not be generalizable since not all families who refused vaccination could be interviewed, and only vaccine refusals in one province were examined.

5. Conclusions

Our study aims to determine the vaccine attitudes of families who refuse vaccination as well as plan for the modification of decisions regarding vaccine refusal among the families interviewed. The intention is to provide guidance for future vaccine persuasion efforts by identifying more persuadable groups. In our study, the persuasion rates of families informed by face-to-face interviews with the vaccine attitude scale were followed by six-month records. The rates were determined according to these records. Our research has shown that face-to-face meetings and education are useful options for convincing people who are hesitant about vaccination.

Author Contributions

Conceptualization: O.K. (Osman Kurtand) and O.K. (Osman Küçükkelepçe); methodology, O.K. (Osman Kurtand), O.K. (Osman Küçükkelepçe) and H.D.T.; software, E.Ö. and H.D.T.; validation, M.E.P., O.K. (Osman Kurtand) and O.K. (Osman Küçükkelepçe); formal analysis, O.K. (Osman Kurt); investigation, M.E.P. and H.D.T.; data curation, O.K. (Osman Küçükkelepçe) and E.Ö.; writing—original draft preparation, M.E.P., E.Ö. and O.K. (Osman Küçükkelepçe); writing—review and editing, O.K. (Osman Küçükkelepçe) and H.D.T.; visualization, O.K.(Osman Küçükkelepçe); supervision, O.K. (Osman Kurt) and E.Ö. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study obtained ethics committee approval from the Non-Interventional Research Ethics Committee of Firat University, with a decision dated 1 September 2022 and numbered 22 October 2022. Institutional permission was also obtained from the Adiyaman Provincial Health Directorate to conduct the research. Written and verbal consent was obtained from all participants, and the study followed the principles outlined in the Declaration of Helsinki.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study. Written informed consent has been obtained from the patients to publish this paper.

Data Availability Statement

Data will be available upon request from the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Nguyen, K.H.; Srivastav, A.; Lindley, M.C.; Fisher, A.; Kim, D.; Greby, S.M.; Lee, J.; Singleton, J.A. Parental vaccine hesitancy and association with childhood diphtheria, tetanus toxoid, and acellular pertussis; measles, mumps, and rubella; rotavirus; and combined 7-series vaccination. Am. J. Prev. Med. 2022, 62, 367–376. [Google Scholar] [CrossRef] [PubMed]
  2. Gokce, A.; Ozdemir, A.T.; Boz, G.; Aslan, M.; Ozer, A. The knowledge and behaviours of the healthcare staff regarding vaccine rejection. Eur. J. Public Health 2020, 30, ckaa166.1447. [Google Scholar] [CrossRef]
  3. He, K.; Mack, W.J.; Neely, M.; Lewis, L.; Anand, V. Parental perspectives on immunizations: Impact of the COVID-19 pandemic on childhood vaccine hesitancy. J. Community Health 2022, 1, 39–52. [Google Scholar] [CrossRef] [PubMed]
  4. Lu, X.; Wang, J.; Hu, L.; Li, B.; Lu, Y. Association between adult vaccine hesitancy and parental acceptance of childhood COVID-19 vaccines: A web-based survey in a northwestern region in China. Vaccines 2021, 9, 1088. [Google Scholar] [CrossRef]
  5. Gür, E. Aşı kararsızlığı. In Asılar, 1st ed.; Türkiye Klinikleri: Ankara, Turkey, 2021; pp. 28–33. [Google Scholar]
  6. Salali, G.; Uysal, M. COVID-19 Vaccine Hesitancy Is Associated with Beliefs on the Origin of The Novel Coronavirus In The UK and Turkey. Psychol. Med. 2020, 15, 3750–3752. [Google Scholar] [CrossRef]
  7. Bendau, A.; Plag, J.; Petzold, M.; Ströhle, A. COVID-19 Vaccine Hesitancy and Related Fears and Anxiety. Int. Immunopharmacol. 2021, 97, 107724. [Google Scholar] [CrossRef]
  8. Karagöz, Y.; Yalman, F. The Effect of Health Professionals’ Attitudes Towards COVID-19 Vaccines on Hesitance Situations: The Mediator Role of Vaccine Confidence. Afet Ve Risk Dergisi. 2023, 1, 119–127. [Google Scholar] [CrossRef]
  9. Kaplan, A.; Sahin, M.; Parildar, H.; Güvenç, I. The Willingness to Accept the COVID-19 Vaccine and Affecting Factors Among Healthcare Professionals: A Cross-sectional Study in Turkey. Int. J. Clin. Pract. 2021, 75, e14226. [Google Scholar] [CrossRef]
  10. Topçu, S.; Almış, H.; Başkan, S.; Turgut, M.; Orhon, F.Ş.; Ulukol, B. Evaluation of childhood vaccine refusal and hesitancy intentions in Turkey. Indian J. Pediatr. 2019, 86, 38–43. [Google Scholar] [CrossRef]
  11. Wallace, A.S.; Wannemuehler, K.; Bonsu, G.; Wardle, M.; Nyaku, M.; Amponsah-Achiano, K.; Dadzie, J.F.; Sarpong, F.O.; Orenstein, W.A.; Rosenberg, E.S.; et al. Development of a valid and reliable scale to assess parents’ beliefs and attitudes about childhood vaccines and their association with vaccination uptake and delay in Ghana. Vaccine 2019, 37, 848–856. [Google Scholar] [CrossRef]
  12. Ceylan, S.S.; Erdoğan, Ç.; Turan, T.; Ergin, A.; Akçay, G. Aşı Tutumları Ölçeğinin Türkçe Formunun Geçerlilik ve Güvenirliliği. Türkiye Klin. Pediatri Derg. 2021, 30, 31–33. [Google Scholar]
  13. Xu, Y.; Xu, D.; Luo, L.; Ma, F.; Wang, P.; Li, H.; Wei, L.; Diao, J.; Liu, Y.; Zhang, W.; et al. A cross-sectional survey on COVID-19 vaccine hesitancy among parents from Shandong vs. Zhejiang. Front. Public Health 2021, 9, 779720. [Google Scholar] [CrossRef]
  14. Pandit, T.; Pandit, R.; Goyal, L. Uncommon Side Effects of COVID-19 Vaccination in the Pediatric Population. Cureus 2022, 14, e30276. [Google Scholar] [CrossRef]
  15. Pippert, C.H.; Lin, J.; Lazer, D.; Perlis, R.; Simonson, M.D.; Ognyanova, K.; Druckman, J.; Tandon, U.; Baum, M.A.; Santillana, M. The COVID States Project# 61: Parental Concerns About COVID-19 Vaccines. osfpreprints 2021. [Google Scholar] [CrossRef]
  16. Napolitano, F.; Miraglia del Giudice, G.; Angelillo, S.; Fattore, I.; Licata, F.; Pelullo, C.P.; Di Giuseppe, G. Hesitancy towards childhood vaccinations among parents of children with underlying chronic medical conditions in Italy. Vaccines 2022, 10, 1254. [Google Scholar] [CrossRef]
  17. Kijjambu, A.; Mulogo, E.M. Determinants of Hesitancy to Childhood Immunizations in a Peri-Urban Settlement; A Case Study of Nansana Municipality, Uganda. TIJPH 2013, 3, 213–226. [Google Scholar] [CrossRef]
  18. Kempe, A.; Saville, A.W.; Albertin, C.; Zimet, G.; Breck, A.; Helmkamp, L.; Vangala, S.; Dickinson, L.M.; Rand, C.; Szilagyi, P.G. Parental hesitancy about routine childhood and influenza vaccinations: A national survey. Pediatrics 2020, 146, e20200961. [Google Scholar] [CrossRef]
  19. Al-Ghasham, Y.A.; Al-Feraidi, A.D. Perception regarding routine childhood vaccinations: A cross-sectional study in Al-Qassim married population. IJMDC 2023, 7, 908–920. [Google Scholar] [CrossRef]
  20. Durmaz, N.; Suman, M.; Ersoy, M.; Örün, E. Parents’ Attitudes toward Childhood Vaccines and COVID-19 Vaccines in a Turkish Pediatric Outpatient Population. Vaccines 2022, 10, 1958. [Google Scholar] [CrossRef]
  21. Erchick, D.J.; Gupta, M.; Blunt, M.; Bansal, A.; Sauer, M.; Gerste, A.; Holroyd, T.A.; Wahl, B.; Santosham, M.; Limaye, R.J. Understanding determinants of vaccine hesitancy and acceptance in India: A qualitative study of government officials and civil society stakeholders. PLoS ONE 2022, 17, e0269606. [Google Scholar] [CrossRef]
  22. McKee, C.; Bohannon, K. Exploring the reasons behind parental refusal of vaccines. J. Pediatr. Pharmacol. Ther. 2016, 21, 104–109. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  23. Soysal, G.; Akdur, R. Investigating Vaccine Hesitancy and Refusal Among Parents of Children Under Five: A Community-based Study. J. Curr. Pediatr. 2022, 20, 339–348. [Google Scholar] [CrossRef]
  24. Skitarelić, N.; Vidaić, M.; Skitarelić, N. Parents’ versus Grandparents’ Attitudes about Childhood Vaccination. Children 2022, 9, 345. [Google Scholar] [CrossRef] [PubMed]
  25. Şen, M.; AKIN, B.; Özaydin, T. The Attitudes of Undergraduate Nursing Students to Childhood Vaccines. J. Contemp. Med. 2022, 3, 424–430. [Google Scholar] [CrossRef]
  26. Lu, L.; Gu, W.; Xie, H.; Wang, X.; Cao, L.; Shan, M.; Wu, P.; Tian, Y.; Zhou, K. Parental Attitudes Towards Vaccination Against COVID-19 In China During Pandemic. IDR 2022, 15, 4541–4546. [Google Scholar] [CrossRef] [PubMed]
  27. Loncarevic, G.; Jovanovic, A.; Kanazir, M.; Tepavcevic, D.; Maric, G.; Pekmezovic, T. Are Pediatricians Responsible for Maintaining High Mmr Vaccination Coverage? Nationwide Survey on Parental Knowledge and Attitudes Towards Mmr Vaccine in Serbia. PLoS ONE 2023, 2, e0281495. [Google Scholar] [CrossRef] [PubMed]
  28. Abedin, M.; Islam, M.; Rahman, F.; Reza, H.; Hossain, M.; Hossain, M.; Arefin, A.; Hossain, A. Willingness to Vaccinate Against COVID-19 Among Bangladeshi Adults: Understanding the Strategies to Optimize Vaccination Coverage. PLoS ONE 2021, 4, e0250495. [Google Scholar] [CrossRef] [PubMed]
  29. Tamara, F.; Fajar, J.; Soegiarto, G.; Wulandari, L.; Kusuma, A.; Pasaribu, E.; Putra, R.P.; Rizky, M.; Anshor, T.; Harapan, H.; et al. The Refusal of COVID-19 Vaccination and Its Associated Factors: A Systematic Review. F1000Research 2023, 12, 54. [Google Scholar] [CrossRef]
  30. Bukhari, M.; Ms, K.; Mughal, A.; Tariq, A.; Yaseen, M.; Fayyaz, T. Effectiveness of Health Education Message in Improving Tetanus Health Literacy Among Women of Childbearing Age: A Quasi-experimental Study. JRMC 2020, 1, 27–31. [Google Scholar] [CrossRef]
  31. Schmid, P.; Rauber, D.; Betsch, C.; Lidolt, G.; Denker, M. Barriers of Influenza Vaccination Intention and Behavior—A Systematic Review of Influenza Vaccine Hesitancy, 2005–2016. PLoS ONE 2017, 1, e0170550. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  32. Brewer, N.; Chapman, G.; Rothman, A.; Leask, J.; Kempe, A. Increasing Vaccination: Putting Psychological Science into Action. Psychol. Sci. Public Interest 2017, 3, 149–207. [Google Scholar] [CrossRef] [Green Version]
  33. Freeman, D.; Loe, B.; Yu, L.; Freeman, J.; Chadwick, A.; Vaccari, C.; Shanyinde, M.; Harris, V.; Waite, F.; Lambe, S.; et al. Effects of Different Types of Written Vaccination Information on COVID-19 Vaccine Hesitancy In the Uk (Oceans-iii): A Single-blind, Parallel-group, Randomised Controlled Trial. Lancet Public Health 2021, 6, e416–e427. [Google Scholar] [CrossRef]
  34. Baldwin, A.; Denman, D.; Sala, M.; Marks, E.; Shay, L.; Fuller, S.; Persaud, D.; Lee, S.C.; Skinner, C.S.; Tiro, J.; et al. Translating Self-persuasion Into an Adolescent Hpv Vaccine Promotion Intervention for Parents Attending Safety-net Clinics. Patient Educ. Couns. 2017, 4, 736–741. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  35. Argyris, Y.; Monu, K.; Tan, P.; Aarts, C.; Jiang, F.; Wiseley, K. Using Machine Learning to Compare Provaccine and Antivaccine Discourse Among the Public on Social Media: Algorithm Development Study. JMIR Public Health Surveill. 2021, 6, e23105. [Google Scholar] [CrossRef]
  36. Zhou, L.; Wang, J.; Cheng, P.; Li, Y.; Liu, G.; Zhang, X. Hpv Vaccine Hesitancy Among Medical Students in China: A Multicenter Survey. Front. Public Health 2022, 10, 774767. [Google Scholar] [CrossRef]
  37. Ellithorpe, M.; Adams, R.; Aladé, F. Parents’ Behaviors and Experiences Associated with Four Vaccination Behavior Groups for Childhood Vaccine Hesitancy. Matern. Child Health J. 2022, 2, 280–288. [Google Scholar] [CrossRef]
  38. Raof, A. Parental Attitude and Beliefs Towards Child Vaccination: Identifying Vaccine Hesitant Groups in A Family Health Center, Erbil City, Iraq. ME-JFM 2018, 6, 17–26. [Google Scholar] [CrossRef] [Green Version]
  39. Hallas, D.; Altman, S.; Mandel, E.; Fletcher, J. Vaccine Hesitancy in Prenatal Women and Mothers of Newborns. Nurse Pract. 2023, 3, 36–47. [Google Scholar] [CrossRef] [PubMed]
  40. Sitaresmi, M.N.; Rozanti, N.M.; Simangunsong, L.B.; Wahab, A. Improvement of Parent’s Awareness, Knowledge, Perception, and Acceptability of HPV Vaccination After an Educational Intervention. BMC Public Health 2020, 20, 1–18. [Google Scholar] [CrossRef]
  41. Nesiama, E.; Vopni, R.; Fuentes, N.; Prabhu, F.; Bennett, K. Assessing and Addressing COVID-19 Vaccine Hesitancy in A West Texas Free Clinic Through Motivational Interview-guided Intervention. Chronicles 2022, 45, 42–47. [Google Scholar] [CrossRef]
  42. Leask, J.; Kinnersley, P.; Jackson, C.; Cheater, F.; Bedford, H.; Rowles, G. Communicating with parents about vaccination: A framework for health professionals. BMC Pediatr. 2012, 12, 154. [Google Scholar] [CrossRef]
  43. Pomares, T.D.; Buttenheim, A.M.; Amin, A.B.; Joyce, C.M.; Porter, R.M.; Bednarczyk, R.A.; Omer, S.B. Association of cognitive biases with human papillomavirus vaccine hesitancy: A cross-sectional study. Hum. Vaccines Immunother. 2020, 16, 1018–1023. [Google Scholar] [CrossRef]
  44. Nia, H.S.; Allen, K.A.; Arslan, G.; Kaur, H.; She, L.; Fomani, F.K.; Gorgulu, O.; Froelicher, E.S. The predictive role of parental attitudes toward COVID-19 vaccines and child vulnerability: A multi-country study on the relationship between parental vaccine hesitancy and financial well-being. Front. Public Health 2023, 11, 1085197. [Google Scholar] [CrossRef]
  45. Reuben, R.; Aitken, D.; Freedman, J.L.; Einstein, G. Mistrust of the medical profession and higher disgust sensitivity predict parental vaccine hesitancy. PLoS ONE 2020, 15, e0237755. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Study Flow Diagram.
Figure 1. Study Flow Diagram.
Vaccines 11 01285 g001
Figure 2. Opinions of the participants about the harms of the vaccine.
Figure 2. Opinions of the participants about the harms of the vaccine.
Vaccines 11 01285 g002
Figure 3. Opinions of the participants about the side effects of the vaccine.
Figure 3. Opinions of the participants about the side effects of the vaccine.
Vaccines 11 01285 g003
Figure 4. Correlation of VAS with age.
Figure 4. Correlation of VAS with age.
Vaccines 11 01285 g004
Table 1. All characteristics of the participants in the study.
Table 1. All characteristics of the participants in the study.
Number%
Age, Mean ± SD34.2 ± 5.8
Interviewed personFather5217.3
Mother24882.7
Educational statusPrimary/secondary school248.0
High School15451.3
University12240.7
ProfessionHomemaker18060.0
Teacher3612.0
Government employee4314.3
Healthcare profession144.7
Artisan165.3
Other113.7
Number of Children14214.0
210735.7
3 and above15150.3
Presence of children under one-year-oldYes5117.0
No24983.0
Opinion on necessity of the vaccineYes3712.3
No24983.0
Indecisive144.7
Having a child vaccinated previously.Yes26488.0
No3612.0
Having problems with vaccination beforeYes11041.7
No15458.3
Status of getting the vaccines recommended by the MinistryHas gotten some vaccines22474.7
Has not gotten any vaccines7625.3
Table 2. Reasons for not getting vaccinated by the study participants *.
Table 2. Reasons for not getting vaccinated by the study participants *.
Number%
I do not think vaccines are safe/I am worried about the side effects29799.0
I think it is not necessary21571.7
I do not think vaccines are effective in protecting from diseases18361.0
Chemical substances in vaccines are harmful to human health.15150.3
Someone else said that their child had a bad reaction after vaccination.13344.3
Negative things about the vaccine I read or heard from the media11036.7
Bad experiences with healthcare professionals or healthcare provider7123.7
Religious reasons248.0
* some participants have more than one reason for not being vaccinated.
Table 3. Comparison of the persuasion status and vaccine attitude scale scores of the participants according to various parameters.
Table 3. Comparison of the persuasion status and vaccine attitude scale scores of the participants according to various parameters.
Those Who Are Convincedp *VASp **
Number%Mean ± SD
Interviewed personFather917.30.03824.3 ± 3.50.007
Mother197.723.5 ± 2.1
Educational statusPrimary/secondary school625.0 a0.0223.5 ± 2.50.258
High School117.1 b23.5 ± 2.1
University119.0 b23.8 ± 2.8
ProfessionHomemaker1910.60.74523.0 ± 2.0 a<0.001
Teacher38.325.0 ± 2.6 b
Government employee49.324.2 ± 2.5 b
Healthcare profession214.324.2 ± 4.5 b
Artisan00.024.4 ± 2.1 b
Other00.026.1 ± 1.8 b
Number of Children1819.0 a0.02423.5 ± 1.70.480
254.7 b23.9 ± 2.7
3 and above159.9 ab23.5 ± 2.4
Presence of children under one-year-oldYes917.60.03424.0 ± 3.10.280
No197.623.6 ± 2.3
Perception on necessity of the vaccineYes821.6 a0.03123.4 ± 3.00.128
No197.6 b23.6 ± 2.3
Indecisive17.1 b24.5 ± 2.8
Having a child vaccinated previously.Yes269.80.55123.7 ± 2.50.202
No25.623.2 ± 1.7
Having problems with vaccination beforeYes54.50.01523.7 ± 2.40.937
No2113.623.7 ± 2.7
Is there any side effect?Yes197.1 a<0.00123.8 ± 2.3 a0.044
No844.4 b21.8 ± 3.4 b
I do not know16.7 a22.9 ± 2.2 ab
Status of getting the vaccines recommended by the MinistryHas gotten some vaccines2611.60.0223.7 ± 2.50.320
Has not gotten any vaccines22.623.5 ± 2.4
* Chi-square analysis, ** Mann–Whitney U test in pairwise comparisons, Kruskal–Wallis analysis in more than two comparisons. Row percentage is used. a,b It is the parameter caused to statistical differences.
Table 4. Comparison of VAS and age according to persuasion status.
Table 4. Comparison of VAS and age according to persuasion status.
Those Who Are ConvincedThose Who Are Not Convincedp *
Mean ± SDMean ± SD
VAS20.7 ± 2.223.9 ± 2.3<0.001
Age31.4 ± 5.334.5 ± 5.80.008
* Mann-Whitney U test was applied.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Kurt, O.; Küçükkelepçe, O.; Öz, E.; Doğan Tiryaki, H.; Parlak, M.E. Childhood Vaccine Attitude and Refusal among Turkish Parents. Vaccines 2023, 11, 1285. https://doi.org/10.3390/vaccines11081285

AMA Style

Kurt O, Küçükkelepçe O, Öz E, Doğan Tiryaki H, Parlak ME. Childhood Vaccine Attitude and Refusal among Turkish Parents. Vaccines. 2023; 11(8):1285. https://doi.org/10.3390/vaccines11081285

Chicago/Turabian Style

Kurt, Osman, Osman Küçükkelepçe, Erdoğan Öz, Hülya Doğan Tiryaki, and Mehmet Emin Parlak. 2023. "Childhood Vaccine Attitude and Refusal among Turkish Parents" Vaccines 11, no. 8: 1285. https://doi.org/10.3390/vaccines11081285

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop