Factors Affecting Attitudes towards COVID-19 Vaccination: An Online Survey in Slovenia

While the problem of vaccine hesitancy is not new, it has become more pronounced with the new COVID-19 vaccines and represents an obstacle to resolving the crisis. Even people who would usually trust vaccines and experts now prefer to wait for more information. A cross-sectional online survey was conducted in Slovenia in December 2020 to find out the attitudes of the population regarding COVID-19 vaccination and the factors that affect these attitudes. Based on 12,042 fully completed questionnaires, we find that higher intention to get vaccinated is associated with men, older respondents, physicians and medical students, respondents who got the influenza vaccination, those who knew someone who had gotten hospitalised or died from COVID-19 and those who have more trust in experts, institutions and vaccines. Nurses and technicians were less likely to get vaccinated. In answers to an open question, sceptics were split into those doubting the quality due to the rapid development of the vaccine and those that reported personal experiences with side effects of prior vaccinations. Although the Slovenian population is diverse in its attitudes towards vaccination, the results are comparable to those found in other countries. However, there are potential limitations to the generalizability of the findings that should be addressed in future studies.


Introduction
While vaccination is widely recognised as an effective way to reduce or eliminate the burden of infectious diseases by health authorities and the medical community [1], its benefit also depends on the willingness of individuals to be vaccinated [2]. This is the case with the COVID-19 vaccines that might be key to stopping the COVID-19 epidemic.
Vaccine hesitancy, i.e., the unwillingness of a proportion of the population to get vaccinated, is closely related to public trust in the health system [3]. To develop an effective communication strategy to advance vaccine readiness, it is crucial to understand the various socio-demographic factors that affect the decision [4]. It is particularly important to find out the positions of those on the vaccination priority list such as different groups of healthcare professionals (HCP).
An online survey about the attitudes towards COVID-19 vaccination in Slovenia was conducted in December 2020, just before the arrival of the first doses of the vaccine. In addition to finding out how many would take the vaccine, the study aimed to estimate the factors that impact the decision. These results can provide healthcare policymakers in Slovenia and other countries reliable information to allow them to improve immunisation plans and create better-targeted vaccination-related communication campaigns.

Methods
A cross-sectional study was designed to study public opinion towards COVID-19 vaccination in Slovenia. The target population of the survey were Slovenian residents older than 15. To assure high standards of data quality, the questionnaire was pretested on 17 people before being fielded. Test subjects used the comment function to note unclear wordings, enabling us to improve questions and answers to more understandable and straight-forward. In total, 12 comments were made and were addressed.
The questionnaire was developed based on previous studies [2,[5][6][7] and had 11 closedended questions regarding the opinion on vaccination in general, attitudes towards different news sources and personal perception of lockdown orders as well as one open-ended question to entail everything a person might want to add. At the end, participants were asked seven demographic questions (gender, age group, education level, work status, profession, region and type of dwelling). Professions were classified as either non-healthcare professionals (non-HCP) or the healthcare professionals (HCP) identified as physicians, other HCP (nurses and technicians), medical students and HC students (health care students). An English translation of the questionnaire is added in Supplement 1.
The survey was conducted with the OneClick Survey (v 20.12.03, RRID:SCR_019283) tool and different media outlets were used to disseminate the survey link. To improve the representativeness of the nonprobability online sample, a wide array of media and other organisations (n = 167) were contacted and asked to disseminate the survey link. The survey invitation is added as Supplement 2.
Analysis of quantitative data was done with SPSS (release 27.0.0.0, RRID:SCR_019096), except for ordinal regression and mediation analysis that was done in STATA (release 15.0, RRID:SCR_012763). The sample was described using frequency distributions and bivariate cross-tabulations and correlations were computed for pairs of variables. To reduce the dimensionality of data for scale questions, a principal component analysis (PCA) was carried out to compute dimensions to be used in further analysis. Based on the literature review, we developed a theoretical regression model with the intention to get vaccinated as the dependent variable that we were able to operationalise with our survey data. We ran ordinal regression to estimate the effect of a set of independent variables with the intention to get vaccinated; the model included gender, age, education, previous vaccination against influenza, HCP and PCA components.
In addition, we performed an analysis of textual data using Orange Data Mining (release 3.27.1, RRID:SCR_019811). The answers for a subsample of the respondents who completed the free-form answer were preprocessed with lowercase transform, tokenization by words, lemmatization with UDPipe 2 lemmatizer [8], stop word removal and removal of digits. TF-IDF transform was used for vectorization of documents. Clustering was performed with cosine distance and hierarchical clustering with Ward linkage. Large subgroups were further explored with another layer of clustering with the same parameters. We analysed the clusters with Chi2, Student's t-test and ANOVA for determining the difference between groups (either a selected subgroup versus all or subgroups between one another). To determine the content of each group's answers, we used a mixture of close and distant reading.
An in-depth description of the methods according to the CHERRIES checklist for reporting results of online surveys [9] can be found in Supplement 3.

Survey Response
Of the 169 contacted organisations, at least 20 have shared the survey link (possibly more but they did not confirm by e-mail). We estimate the sum potential reach of these organisations is more than 400,000 users (assuming a zero overlap in followers) which is about 48% of all social media users according to the data of the Statistical office of Slovenia. However, organic posts usually do not reach all users and according to some estimates the average reach is only between 5-6% [10] of the followers, i.e., between 20,000 and 24,000. However, in our case the reach was definitely higher since there were 45,633 unique clicks on the survey link in the eleven days that the survey ran, December 17-27 in 2020. Of those who clicked, 27,356 did not start to respond after landing on the survey page, while 18,277 started to respond but 3765 dropped out and so only 12,042 completed the survey. We estimate the response rate is between 3% (assuming the survey link reached more than 400,000 people) and 26% (assuming the reach was the same as the number of clicks).

Demographic Characteristics
The majority of respondents were female (63%), had high education (64%) and most of them were relatively young (42% were below 35, 40% from 35 to below 54 and 18% were 55 or older). Since many among the contacted individuals or organisations were healthrelated, 2068 respondents (17%) were HCP. The demographic data of HCP in our sample specifically compared with national registry can be found in Supplement 4. Demographic data are presented in Table 1.

Intention to Get Vaccinated
Among all respondents, 3964 (33%) indicated that they definitively intended to participate in vaccination and an additional 3119 (26%) replied that they would probably agree to vaccination; in total, 59% intended to vaccinate. As shown in Figure 1, the intention was similar for non-HCP (57%) and some groups of HCP (51% HC students, 50% other HCP), while physicians (84%) and medical students (82%) were significantly more inclined towards vaccination (p < 0.001, c = 0.158).
Respondents who knew someone who was hospitalized or died due to the virus (44% of all respondents) were more likely to accept vaccination (66%) as compared those who did not know anyone with that experience (53%) (p < 0.001, Chi Square = 202.946, Contingency Coeff. = 0.129). In the past, only 18% of respondents had been regularly vaccinated against influenza. A strong association was observed between prior vaccination against influenza and a positive attitude towards vaccination against SARS-CoV-2 (43% vs. 78%, p < 0.001, Chi Square = 1542.675, Contingency Coeff. = 0.337) ( Figure 3). Respondents who knew someone who was hospitalized or died due to the virus (44% of all respondents) were more likely to accept vaccination (66%) as compared those who did not know anyone with that experience (53%) (p < 0.001, Chi Square = 202.946, Contingency Coeff. = 0.129).

Principal Component Analysis
The questionnaire included 11 questions about trusting information from different sources and seven items about agreement with different COVID-19 vaccine statements. We extracted three components for which the eigenvalue was above 1. The threshold was decided based on the change in slope observed in the scree plot of eigenvalues (i.e., after the third component it gets flatter). Together, these three components explain 60% of the variance.
To interpret the meaning of variables, we looked at their weights ( Table 2). The first component had the highest positive weight (0.86) for trust in the National Institute of Public Health (NIPH) and the safety and efficiency of the vaccine, followed by trust in the Ministry of Health (0.79), the World Health Organisation (WHO) and opinion of experts (0.78), news on television and radio (0.77), professional articles and research results (0.74) and daily newspapers (0.72). At the same time, weights for the first component had a high negative value for believing that vaccination is an attempt to control the population (−0.79) and having negative experiences with prior vaccinations (−0.61). We call this component "trust in official sources". Nevertheless, it should be noted that this component has non-negligible weights for some items that are not completely aligned with this name (i.e., being scared of getting infected). The second component had a high positive value for trusting information from friends and acquaintances that are non-HCP (0.74) and alternative explanations on social media (0.66). To a lesser extent, it also involved trust in information from those friends and acquaintances that are HCP (0.53). We call this component "trust in alternative sources". The third component had a strong negative weight for trust in the government (−0.63) and, to some extent, a strong desire to wait for more information about the safety of the vaccine (0.47). We call this component "distrust in government".
We computed mean component values for different HCP (S5: Table 1). Physicians and medical students have a positive mean value for trust in official sources and negative for trust in alternative sources, while other HCP, HC students and those respondents that are non-HCP have a negative mean value for trusting official sources and positive for alternative sources. The differences for the third component were less pronounced. Figure 4 shows the scatterplot of values for the first and second component. non-HCP have a negative mean value for trusting official sources and positive for alternative sources. The differences for the third component were less pronounced. Figure 4 shows the scatterplot of values for the first and second component.

Ordinal Regression
We ran an ordinal regression to estimate the relationships between the intent to vaccinate and gender, age, education, experience with COVID-19, profession and the three dimensions extracted with PCA. The results are presented in Table 3.

Ordinal Regression
We ran an ordinal regression to estimate the relationships between the intent to vaccinate and gender, age, education, experience with COVID-19, profession and the three dimensions extracted with PCA. The results are presented in Table 3. The regression coefficients are significant for age, gender, education, being a physician and the three PCA components. Respondents who were older, male, physicians and those who trust official sources were more ready to get vaccinated, while more educated respondents, those who trust alternative sources and distrust the government were more vaccine hesitant. It should be noted, however, that for education the result is in a different direction than expected, so we performed a sensitivity analysis that showed (unlike other variables) education is sensitive to different settings; thus, we interpret the evidence as inconclusive. Knowing someone who had been hospitalized or had died due to COVID-19, being other HCP, medical student or HC student did not have a significant effect on the intention to get vaccinated.
Although prior vaccinations were also strongly associated with COVID-19 vaccination intention, we did not include it in the above ordinal regression model because it has a strong relationship with all other variables in the model (S5: Table 2).
Moreover, mediation analysis was used to correct for the effect of prior influenza vaccinations, deriving both the direct effects of relationships as well as the indirect ones through their effect on prior vaccinations. The summary results are shown in Table 4. The mediation effect is the highest for physicians (100%), which means that being a physician only affects the intention to get vaccinated indirectly through prior influenza vaccinations. In other words, if you are a physician, you are more prone to get vaccinated whether it is for influenza or COVID-19. The second highest effect, but much lower, is for education (33%), which means that only 67% of the negative effect from higher education on COVID-19 vaccination is direct, while one third is indirect, through influenza vaccination. Thus, the regression coefficient in Table 2 can be reduced by 33% to −0.135 (as shown in the last column of Table 4). The effect is considerable for age (18%) and gender (12%), while it is rather small (5-6%) for the three components.
In addition, since plots in Figure 4 indicate there is an interaction between the PCA components and healthcare profession, we conducted an additional analysis to moderate it. Results show that being a physician or other HCP increases the trust in official resources and lowers distrust in government, while for trust in alternative resources there are no significant effects.

Analysis of Textual Data
We analysed responses from the 2320 respondents (12%) who answered the openended question. Compared to others, they had lower trust in the WHO ( We used hierarchical clustering to discover groups of similar responses. After preprocessing, we were left with 7625 unique tokens (words), which we transformed into a TF-IDF weighted bag of words. We computed the distance matrix with cosine distance and used Ward linkage for clustering groups. While the dendrogram did not reveal a good number of clusters ( Figure 5), we decided on 10 clusters and further explored larger ones. We used hierarchical clustering to discover groups of similar responses. After preprocessing, we were left with 7625 unique tokens (words), which we transformed into a TF-IDF weighted bag of words. We computed the distance matrix with cosine distance and used Ward linkage for clustering groups. While the dendrogram did not reveal a good number of clusters ( Figure 5), we decided on 10 clusters and further explored larger ones. The first three clusters are most different from the rest as they join the larger cluster at later stages. The larger bottom subcluster forms two groups, the first containing clusters C4 and C5 and the second containing the remaining clusters. We used word enrichment to determine which words best define each cluster (S5: Table 3). The first three clusters are most different from the rest as they join the larger cluster at later stages. The larger bottom subcluster forms two groups, the first containing clusters C4 and C5 and the second containing the remaining clusters. We used word enrichment to determine which words best define each cluster (S5: Table 3).
After determining the initial ten clusters, we further explored the two largest clusters, namely C5 and C10. C5 mostly contains notes on vaccination, such as questions about vaccine safety for risk groups and worries about side effects (Table 5). Cluster C10 contains comments about government measures as well as a sizable number of sceptics (Table 6). Table 6. Detailed analysis of cluster C10. It contains roughly five subclusters, but it is fairly diverse. It contains both respondents that wish to get vaccinated quickly (C10.3) and those that believe the vaccine is a Big Pharma scam (C10.5).

Subcluster Description No. of Answers
C10.1 "Don't get vaccinated." 7 C10.2 "Experts should be in higher agreement." 172 C10. 3 "We need a vaccine as soon as possible." 15 C10. 4 "Vaccine is a genocide." 11 C10. 5 Conspiracy theories 917 In summary, clusters C3 and C8 represent the sceptics. These respondents were also in higher agreement that the vaccine is an attempt to control the population, that the virus is the same as influenza and were against vaccinations in general. These two groups reported trusting alternative explanations on social media to a higher degree than the official sources (Q6.7).
Cluster C6 represents pro-vaccine respondents, which also had a higher agreement that the vaccine is safe and efficient. Respondents in C6 and C1 plan to get vaccinated as soon as possible (Q9). The former cluster has higher trust in official sources, while the latter in experts.
To validate the findings and determine the context of significant words, we also read a couple of responses in each cluster. An interesting finding was that many sceptics used scientific language to convey their doubt in the vaccine. For example, a large number of people claimed that the virus had not yet been isolated according to the Koch postulate. The second group of sceptics pointed to the role of the human immune system and how it should have been the primary line of defence. The third group of sceptics listed their personal bad experiences with vaccines as the reason why they were against vaccination. Among the milder sceptics, there were many comments regarding length of immunity after the vaccine, responsibility for the side effects of the vaccine and safety of the vaccine for people with allergies.
Even the pro-vaccine respondents with academic degrees stressed the importance of educating the public and presenting the findings. For instance, one of the physicians wrote: "Medical science should take people with bad vaccine experience more seriously. They should design measures to explain why there was an adverse reaction to the vaccine, how it can be prevented further, and how the people who indeed have bad reactions are the ones who need the rest of the population to have a high vaccination rate."

Discussion
In terms of sample size, this survey is the biggest national study on attitude towards vaccination carried out in any European state to date [4]. Among all respondents, 59% answered that they will definitely or probably get vaccinated, which is about the same as in the USA in November 2020 (60%) [11], lower than China (91%), Australia (85%) and most Western and Northern European countries [4,[12][13][14] but higher than most Eastern and South-Eastern European countries [4]. However, it should be noted that most of the surveys had been conducted in spring 2020 and the attitudes have probably changed since then.
A positive attitude towards vaccination is more prevalent among men and increases with age, which was also confirmed in previous vaccination studies [2,12,15]. Conversely, we were not able to confirm the finding that a high education level is associated with the intention to get vaccinated [15,16]. There are no studies exploring the reasons for gender and age differences in SARS-CoV-2 vaccine hesitancy, but we assume it can be explained by the fact that COVID-19 is more deadly for men and the elderly. However, there is some research for other diseases. For instance, a pneumococcal vaccine survey has shown that older people have more trust in vaccine and have more positive attitude towards vaccination based on former good experiences [17].
Studies in the USA and China have shown a strong association between prior vaccination against influenza and the intention to get vaccinated against SARS-CoV-2 [11,12], a finding which was confirmed by our study as well. An added value of our analysis was to include influenza vaccinations as a mediator variable, while the effects of other variables are lower than it appears on first sight. Another US study indicated that those who personally knew someone who had been hospitalized or had died as a result of having COVID-19 had a higher intention to get vaccinated compared to those without that experience [11], which was also confirmed by our crosstabulation analysis. A study about hypothetical Ebola vaccine had the same results as the USA study regarding COVID-19, the authors of the study go a step further and suggest that it is possible that knowing someone who got sick and witnessing their treatment and disease progress scares the individuals enough to make them more amendable to vaccination [18,19]. However, the effect of this variable is not significant in the ordinal regression model, which means we cannot generalise this association.
In contrast with other studies, the population in our study included a significant proportion of HCP and, thus, allowed us to analyse their attitudes towards vaccination. The attitudes and behaviour of HCP's related to COVID-19 vaccination are of great importance as people are more likely to get vaccinated if recommended by their HCP [20]. In France, their intention to get vaccinated was higher among HCP than non-HCP [21]; while a survey in the USA showed that being employed as HCP was negatively associated with the intent to get a vaccine as compared with those who were never employed in HCP [20]. A survey in Los Angeles found that with nurses and personnel, regardless of patient contact roles, there are higher odds for delay or refusal of a coronavirus vaccine [22].
Discrepancies in findings of previous studies are probably due to the heterogeneity of HCP. Our study found that physicians and medical students have a higher trust in official sources and are more likely to believe in the safety and efficiency of the vaccine, while other HCP and HC students are more likely to believe alternative sources, even more than non-HCP. Nevertheless, the results of ordinal regression show that, with the exception of physicians (indirectly), HCP does not have an effect on vaccine readiness.
There are many reasons explored by other authors on vaccine hesitation among HCP. A review in 2015 found that HCP vaccination rates vary between different infectious agents and between different specialities [23]. A qualitative study on nurses found that they prefer conventional health beliefs rather than evidence-based medicine [24]. This is in line with our finding that many of the other HCP trust alternative sources mor than official ones [24].
A general trust in science and experts, together with accepting that the vaccine as safe and efficient, is the predictor of intention to get vaccinated against COVID-19 [25,26], while alternative explanations for the pandemic, distrust in science and government, as well as an unstable political situation, play a major role in vaccine rejection [4,22,27]. An American survey has indicated that the trust not only in government but also in science is significantly higher among those who endorse vaccination [6]. Similarly, a European survey found that the leading reasons for people choosing not to get vaccinated are not believing in the efficacy of vaccine, believing to have influenza is better than vaccination, having influenza-like illness despite vaccination, believing that influenza is not fatal and experiencing vaccine-related adverse events [28]. Moreover, a UK survey found that low confidence in the health system to handle the pandemic was associated with greater mistrust of vaccine safety and more worries about the unforeseen vaccine effects, whilst low confidence in government to handle the pandemic was associated with lower scores on worries about unforeseen effects and preference for natural immunity [29]. Furthermore, a Polish survey points out that lack of trust in public authorities could increase support towards conspiracy theories and lead to insufficient vaccination rate [30].
In our study, we included several items about trust and other attitudes from these previous studies and confirmed that they are affecting vaccination intention. With PCA, we extracted three dimensions: trust in official sources, trust in alternative sources and distrust in government. The intention to get vaccinated is positively associated with the first one and negatively with the other two.
The analysis of open-ended responses provided us with a deeper insight into how respondents think about vaccinations and offers practical suggestions on how to communicate with sceptics. For instance, some of them like to use scientific language to present their statements, despite incorrect or irrelevant theories. Another group of sceptics likes to point out the role of the human immune system and yet another brings up personal bad experiences with vaccines such as side effects. Both were also mentioned in other studies [29,31,32]. Thus, it is important to take them seriously and address these issues when communicating with the public.
The study has some potential limitations. Although the sample was much larger than in any of the previous national studies, it should be pointed out that we used nonprobability sampling (as in most of the previous research, with a few exceptions) so the findings should not be generalized to the whole population. Nevertheless, we were able to validate almost all presented findings on the probability panel sample that used the same questionnaire on a sample of 512 respondents and was proven to be representative of the Slovenian population [33], but did not include a large enough number of HCP to be used for the main study.
Having a cross-sectional study design, we are limited to a specific time frame and are not able to estimate how attitudes change in time. Another drawback is being limited to only one country in Europe; however, previous studies did not show any significant differences between countries, so we can assume that the findings in a national context have wider application.
Furthermore, our questionnaire did not include all the relevant factors mentioned in previous studies. For instance, we could have also included questions about socioeconomic status [6,11,15], awareness of herd immunity [34] and fear of side effects [31,32]. The latter were highlighted in our survey's open-ended response results.

Conclusions
Our study confirmed findings in previous literature that respondents who intend to get vaccinated are most likely to be male and older. Those who knew people that were severely affected by the illness are likewise more inclined to take a shot. The intention is also positively associated with more trust in official sources of information, such as experts and public health institutions, while trust in alternative sources and mistrust of government makes for a lower chance that a person will agree to vaccination.
Being the largest public opinion survey in a single country to date, we were also able to gather the data for HCP and have demonstrated that physicians and medical students are more in favour of vaccination when compared to other HCP.
The above-mentioned effects can be broadly explained by prior influenza vaccinations that act as a mediating variable. This is a finding specific to our study as previous research has not applied mediation analysis to their models.
Moreover, open-ended question responses revealed that vaccine hesitancy is related to the rapid vaccine development and scepticism about its efficiency. The main focus in future efforts to vaccinate the global population should address all of the above-mentioned factors.
Further research should include questions about the fear of side effects and other relevant aspects that were revealed to be important for understanding the attitudes of the COVID-19 vaccination sceptics.
Supplementary Materials: The following are available online at https://www.mdpi.com/2076-393 X/9/3/247/s1. Document S1: Questionnaire translation to English language; Document S2: Letter to promote the survey, translated to English language; Document S3: CHERRIES (Checklist for Reporting Results of Internet E-Surveys) compliant methods description; spreadsheet S4: Detailed comparison of demographics in our data sample and national statistical registry; S5 Table 1: Computed mean component values for different HCP and non-HCP, calculations relating to Figure 4; S5 Table 2: Results of binary regression analysis for previous flu vaccinations; S5 Table 3: Results of word enrichment for each cluster. Clusters were compared one against all. Only words with FDR less than 0.1 are reported or the top three words by p-value if there are many significant words.  Institutional Review Board Statement: Ethical review and approval were waived for this study as it did not include any ethical issues and the questionnaire did not collect any personally identifiable data.
Informed Consent Statement: Patient consent was waived due to the nature of research. The introduction of the survey emphasizes that participation is voluntary, and the participant could leave the survey at any time during the process. The participants were also informed that the data will only be used for research and pedagogical purposes.