Hesitancy towards COVID-19 Vaccination among Healthcare Workers: A Multi-Centric Survey in France

Vaccination programs against COVID-19 are being scaled up. We aimed to assess the effects of vaccine characteristics on vaccine hesitancy among healthcare workers in a multi-center survey conducted within French healthcare facilities from 1 December 2020 to 26 March 2021. We invited any healthcare workers naïve of COVID-19 vaccination to complete an online self-questionnaire. They reported on their socio-demographic characteristics, as well as their perception and beliefs towards vaccination. We measured their willingness to get vaccinated in eight scenarios for candidates’ vaccines presented sequentially (1 to 4-point scale). Candidates’ vaccines varied for efficacy (25%, 50%, 100%), length of immunization (1 year or lifetime), frequency (<1/100, <1/10,000), and severity (none, moderate, severe) of adverse events. We analyzed 4349 healthcare workers’ responses with interpretable questionnaires. The crude willingness to get vaccinated was 53.2% and increased over time. We clustered the trajectories of responses using an unsupervised classification algorithm (k-means) and identified four groups of healthcare workers: those willing to get vaccinated in any scenario (18%), those not willing to get vaccinated at all (22%), and those hesitating but more likely to accept (32%) or reject (28%) the vaccination depending on the scenario. In these last two subgroups, vaccine acceptance was growing with age, educational background and was higher among men with condition. Compared to an ideal vaccine candidate, a 50% reduced efficacy resulted in an average drop in acceptance by 0.8 (SD ± 0.8, −23.5%), while it was ranging from 1.4 (SD ± 1.0, −38.4%) to 2.1 (SD ± 1.0, −58.4%) in case of severe but rare adverse event. The acceptance of a mandatory immunization program was 29.6% overall and was positively correlated to the willingness to get vaccinated, ranging from 2.4% to 60.0%. Even if healthcare workers represent a heterogeneous population, most (80%) could accept the vaccination against COVID-19. Their willingness to get the vaccine increased over time and as immunization programs became available. Among hesitant professionals, the fear of adverse events was the main concern. Targeted information campaigns reassuring about adverse events may increase vaccine coverage, in a population with a strong opinion about mandatory immunization programs.


Introduction
The massive scaling-up of immunization programs is essential to tackle the COVID-19 pandemic and progressively reduce the burden of bundles of preventive measures [1].

Results
Over the study period, more than 45,000 workers from 21 healthcare facilities were reached, of whom 8773 opened the online survey: 711 were vaccinated and were therefore excluded from clustering analyses, 4349 had exploitable responses, and were thereby included in the analyses (Figure 1). The sample was mainly composed of women (74.4%), aged 25 to 50 (71.3%), with a broad range of educational levels, and a third having a master's degree or higher. Half of the participants were acting as frontline caregivers, including 31.8% nurses and nurse assistants and 14.4% physicians. The majority (86.7%) were working in public hospitals, and the remaining in private facilities or nursing homes.
Vaccines 2021, 9, x FOR PEER REVIEW 3 of 13 (first scenario). We considered the sequential measurement of willingness as a longitudinal array, corresponding for each respondent, to a trajectory of vaccine acceptance. For complete cases, we clustered individual trajectories to identify subgroups of attitudes towards vaccine acceptance using the measurement of temporal Euclidean distance (kmeans for longitudinal data) [12]. Based on the Elbow method, we determined that four clusters would explain ~80% of the variance: (1) those willing to be vaccinated at any cost; (2) those hesitating but likely to get vaccinated; (3) those hesitating but not likely to get vaccinated; and (4) those not willing to get vaccinated at all. We used frequencies (percentages), and median (interquartile range) to describe these four clusters. Among hesitant healthcare workers, we estimated the conditions which negatively impacted vaccine acceptance: efficacy (100% to 50%), adverse event severity (none to moderate, to severe), or length of immunization (lifetime to 1 year). We finally compared potential leverages to increase willingness to get vaccinated by comparing healthcare workers' beliefs about COVID-19 vaccination. We used the chi-squared test and the Fischer exact test to compare frequencies between clusters. We also used the Student t-test and the Mann-Whitney test to compare distributions. We set the level of significance to 5% bilateral (p-value < 0.050). All analyses were performed on the R software version 4.0.3 (The R Project for Statistical Computing, Vienna, Austria). We used the 'kml' and 'ggplot2′ packages.

Results
Over the study period, more than 45,000 workers from 21 healthcare facilities were reached, of whom 8,773 opened the online survey: 711 were vaccinated and were therefore excluded from clustering analyses, 4,349 had exploitable responses, and were thereby included in the analyses (Figure 1). The sample was mainly composed of women (74.4%), aged 25 to 50 (71.3%), with a broad range of educational levels, and a third having a master's degree or higher. Half of the participants were acting as frontline caregivers, including 31.8% nurses and nurse assistants and 14.4% physicians. The majority (86.7%) were working in public hospitals, and the remaining in private facilities or nursing homes.   The crude willingness to get vaccinated against COVID-19 (including already vaccinated respondents) was 53.2% overall (4,558 responses), including 15.6% vaccinated, 21.8% definitely, yes, and 16.8% yes, likely. Others were neutral (11.9%), not likely (19.8%), or not willing to get vaccinated at all (14.1%). After the start of the immunization program, the proportion of respondents of the poll vaccinated against COVID-19 reached 59.1% after 10 March 2021. As illustrated in Figure 2, the willingness increased over the study period (r 2 = 0.010, p-value < 0.001).

Figure 2.
Time course of raw-willingness score to be vaccinated against COVID-19. White numbers represent the number of participants enrolled each week (poll time points). Participants were categorized in 5 groups according to their self-reported crude willingness to be vaccinated against COVID-19 (1 to 7 scale): definitely, yes (7), yes, likely (5-6), neutral (4), not likely (2-3), definitely, not (1). Black bars show the proportion of vaccinated respondents. Period 1: no vaccine available. Period 2: BNT162b2 mRNA vaccine available for healthcare workers > 50 years and AZD1222 adenovirus vaccine for those <50 years. Period 3: BNT162b2 mRNA vaccine for any healthcare worker, and AZD1222 adenovirus vaccine for those >55 years only (after a temporary ban for safety concerns).

Scenario and Vaccine Acceptance
We presented eight candidate vaccine scenarios to participants, always in the same order, varying for efficacy, duration of immunity, and adverse events. Complete responses for candidate vaccines were available for 3732 (85.8%) of respondents, which were clustered. Figure 3A shows the evolution (trajectories) of vaccine acceptance over the scenario (scale of 1 to 4, from 1, "Definitely, not" to 4, "Definitely, yes"). The average acceptance decreased from a maximum of 3.4 (SD ± 0.9) with an ideal vaccine candidate (scenario 1) to a minimum of 1.7 (SD ± 0.9) in the worst-case scenario (scenario 7), with the lowest efficacy and the worst side effects.
The k-means algorithm clustered participants into four subgroups for vaccine acceptance ( Figure 3B): those with almost constant negative answers (group "Never"), those with almost constant positive answers (group "Always"), and the two remaining hesitant groups despite a trend to rather positive ("Hesitant but willing" group) or negative ("Hesitant but not willing" group) willingness.

Figure 2.
Time course of raw-willingness score to be vaccinated against COVID-19. White numbers represent the number of participants enrolled each week (poll time points). Participants were categorized in 5 groups according to their self-reported crude willingness to be vaccinated against COVID-19 (1 to 7 scale): definitely, yes (7), yes, likely (5-6), neutral (4), not likely (2-3), definitely, not (1). Black bars show the proportion of vaccinated respondents. Period 1: no vaccine available. Period 2: BNT162b2 mRNA vaccine available for healthcare workers >50 years and AZD1222 adenovirus vaccine for those <50 years. Period 3: BNT162b2 mRNA vaccine for any healthcare worker, and AZD1222 adenovirus vaccine for those >55 years only (after a temporary ban for safety concerns).

Scenario and Vaccine Acceptance
We presented eight candidate vaccine scenarios to participants, always in the same order, varying for efficacy, duration of immunity, and adverse events. Complete responses for candidate vaccines were available for 3732 (85.8%) of respondents, which were clustered. Figure 3A shows the evolution (trajectories) of vaccine acceptance over the scenario (scale of 1 to 4, from 1, "Definitely, not" to 4, "Definitely, yes"). The average acceptance decreased from a maximum of 3.4 (SD ± 0.9) with an ideal vaccine candidate (scenario 1) to a minimum of 1.7 (SD ± 0.9) in the worst-case scenario (scenario 7), with the lowest efficacy and the worst side effects.
The k-means algorithm clustered participants into four subgroups for vaccine acceptance ( Figure 3B): those with almost constant negative answers (group "Never"), those with almost constant positive answers (group "Always"), and the two remaining hesitant groups despite a trend to rather positive ("Hesitant but willing" group) or negative ("Hesitant but not willing" group) willingness.  . Trajectory of responses to vaccine scenarios and clustering. Eight vaccine candidate scenarios were submitted to participant in the same order from #1 (S1, ideal vaccine candidate conferring a lifelong immunity with 100% efficacy and no adverse events) to #8 (S8). Vaccine efficacy ranges from 25% to 100%. Immunization duration was 1-year to lifelong. Side effects were absent, moderate (M) or rare but severe (RS). Participant scored (1 to 4 scale) each scenario according to their willingness to be vaccinated with the proposed candidate from 1 ("Not at all") to 4 ("Definitely"). (A) Average trajectory of responses across the scenarios; error bars represent standarddeviations, dashed horizontal line the limit between scores in favor (above) or in disfavor (below) of vaccine acceptance. (B) Average trajectories of vaccine acceptance for the four clusters obtained using a k-mean algorithm for longitudinal data (see methods): "Never" group in red, "Always" group in green, "Hesitating but willing" group in yellow and "Hesitating and not willing" group in blue.

Cluster Characteristics
The characteristics of each clusters' staff members are described in Table 1. Workers not likely or not willing at all to get the vaccine were the youngest, more frequently women, less educated, and working in administrative functions with a relatively lower income. They also had a low coverage for mandatory vaccines-hepatitis B, combined (diphtheria, tetanus, poliomyelitis), and whooping cough-and yearly recommended influenza vaccination (17.6% to 42.3% versus 75.3% to 91.1% in others, p < 0.001). Exposure to COVID-19 cases and rate of infection was similar across clusters, though the frequency of risk factors for severe COVID-19 infection was higher in those willing to get the vaccine. Trajectory of responses to vaccine scenarios and clustering. Eight vaccine candidate scenarios were submitted to participant in the same order from #1 (S1, ideal vaccine candidate conferring a lifelong immunity with 100% efficacy and no adverse events) to #8 (S8). Vaccine efficacy ranges from 25% to 100%. Immunization duration was 1-year to lifelong. Side effects were absent, moderate (M) or rare but severe (RS). Participant scored (1 to 4 scale) each scenario according to their willingness to be vaccinated with the proposed candidate from 1 ("Not at all") to 4 ("Definitely"). (A) Average trajectory of responses across the scenarios; error bars represent standard-deviations, dashed horizontal line the limit between scores in favor (above) or in disfavor (below) of vaccine acceptance. (B) Average trajectories of vaccine acceptance for the four clusters obtained using a k-mean algorithm for longitudinal data (see methods): "Never" group in red, "Always" group in green, "Hesitating but willing" group in yellow and "Hesitating and not willing" group in blue.

Cluster Characteristics
The characteristics of each clusters' staff members are described in Table 1. Workers not likely or not willing at all to get the vaccine were the youngest, more frequently women, less educated, and working in administrative functions with a relatively lower income. They also had a low coverage for mandatory vaccines-hepatitis B, combined (diphtheria, tetanus, poliomyelitis), and whooping cough-and yearly recommended influenza vaccination (17.6% to 42.3% versus 75.3% to 91.1% in others, p < 0.001). Exposure to COVID-19 cases and rate of infection was similar across clusters, though the frequency of risk factors for severe COVID-19 infection was higher in those willing to get the vaccine.  ‡ Relative income was assessed using a semi-quantitative Likert scale ranging from 1 ("I am strongly disadvantaged compared to other staff members") to 10 ("I am strongly advantaged compared to other staff members"; Ϫ Any condition among the following: age ≥65 years, cardiovascular disease, complicated diabetes, chronic respiratory disease, obesity, late-stage cirrhosis or active cancer, immunodepression, chronic renal failure <30 mL/min/1.73 m 2 , sickle cell disease, pregnancy at 3rd trimester.
As described in Table 2, the fear of adverse events (severity and frequency) was higher in healthcare workers not willing to get vaccinated. Most (51.8% to 58.7%) believed that pharmaceutical companies had a beneficial interest in increasing population vaccination coverage (p-value < 0.001). Up to a quarter of them (13.9% to 27.7%) supported that alternative medicine and homeopathy were efficient against COVID-19 (p-value < 0.001). Empathy, assessed through five questions, was similar across clusters (Supplementary Figure S1)  cludes residents, midwives, physiotherapists, students; ‡ Relative income was assessed using a t scale ranging from 1 ("I am strongly disadvantaged compared to other staff members") to 10 ("I compared to other staff members"; Ϫ Any condition among the following: age ≥65 years, cardioicated diabetes, chronic respiratory disease, obesity, late-stage cirrhosis or active cancer, immunol failure <30 mL/min/1.73 m 2 , sickle cell disease, pregnancy at 3rd trimester. As described in Table 2, the fear of adverse events (severity and frequency) was higher in healthcare workers not willing to get vaccinated. Most (51.8% to 58.7%) believed that pharmaceutical companies had a beneficial interest in increasing population vaccination coverage (p-value < 0.001). Up to a quarter of them (13.9% to 27.7%) supported that alternative medicine and homeopathy were efficient against COVID-19 (p-value < 0.001). Empathy, assessed through five questions, was similar across clusters (Supplementary Figure S1).
Any condition among the following: age ≥65 years, cardiovascular disease, complicated diabetes, chronic respiratory disease, obesity, late-stage cirrhosis or active cancer, immunodepression, chronic renal failure <30 mL/min/1.73 m 2 , sickle cell disease, pregnancy at 3rd trimester. Table 2, the fear of adverse events (severity and frequency) was higher in healthcare workers not willing to get vaccinated. Most (51.8% to 58.7%) believed that pharmaceutical companies had a beneficial interest in increasing population vaccination coverage (p-value < 0.001). Up to a quarter of them (13.9% to 27.7%) supported that alternative medicine and homeopathy were efficient against COVID-19 (p-value < 0.001). Empathy, assessed through five questions, was similar across clusters (Supplementary Figure S1).  Figure 4 displays the raw and relative differences in vaccine acceptance by scenario. Among hesitant healthcare workers, the scenario analysis revealed that any decrease in vaccine efficacy or length of immunization resulted in a decrease in vaccine acceptance. However, the strongest and most negative effect was observed for the situation in which adverse events severity was increasing, even if rare (Supplementary Figure S2).

Efficacy, Adverse Events and Length of Immunization Variations on Vaccine Acceptance among Hesitant Healthcare Workers
Indeed, the combination of decreased efficacy, one-year immunization, and severe adverse events (scenario 7) lead to a 2.1 drop (SD ± 0.8) in vaccine acceptance from the ideal (−58.4%). Moreover, in such conditions, the decrease in efficacy from 100% to 50% (scenario 6) only resulted in a 0.3 drop (SD ± 0.6, −17.9%) in vaccine acceptance. Finally, when switching from an ideal to a vaccine candidate with severe adverse events (scenario 8) the vaccine acceptance dropped by 1.4 (SD ± 1.0, −38.4%), whereas moderate but frequent adverse events (scenario 4) only lead to a 0.7 drop (SD ± 0.8, −18.9%).
Consistently, willingness to get vaccinated was inversely correlated with the fear of COVID-19 vaccine composition and its potential adverse effects on the body ( Figure 5A-C). Those with the least vaccine acceptance also had the lowest confidence in vaccine efficacy ( Figure 5D-F).

Mandatory Vaccination
Less than a third of healthcare workers were willing to accept mandatory vaccination programs, ranging from 2.4% among those unwilling to get vaccinated to 60% among those willing.  Figure 4 displays the raw and relative differences in vaccine acceptance by scenario. Among hesitant healthcare workers, the scenario analysis revealed that any decrease in vaccine efficacy or length of immunization resulted in a decrease in vaccine acceptance. However, the strongest and most negative effect was observed for the situation in which adverse events severity was increasing, even if rare (Supplementary Figure S2).  candidate (abscissa) and subsequent scenarios (ordinate) among hesitating 2240 participants (A). In (B), the relative percentage of variation in mean difference is given.

Efficacy, Adverse Events and Length of Immunization Variations on Vaccine Acceptance among Hesitant Healthcare Workers
Indeed, the combination of decreased efficacy, one-year immunization, and severe adverse events (scenario 7) lead to a 2.1 drop (SD ± 0.8) in vaccine acceptance from the ideal (−58.4%). Moreover, in such conditions, the decrease in efficacy from 100% to 50% (scenario 6) only resulted in a 0.3 drop (SD ± 0.6, −17.9%) in vaccine acceptance. Finally, when switching from an ideal to a vaccine candidate with severe adverse events (scenario 8) the vaccine acceptance dropped by 1.4 (SD ± 1.0, −38.4%), whereas moderate but frequent adverse events (scenario 4) only lead to a 0.7 drop (SD ± 0.8, −18.9%).
Consistently, willingness to get vaccinated was inversely correlated with the fear of COVID-19 vaccine composition and its potential adverse effects on the body ( Figure 5A-C). Those with the least vaccine acceptance also had the lowest confidence in vaccine efficacy ( Figure 5D-F).

Discussion
The willingness to get vaccinated against COVID-19 among healthcare workers increased over time and with the availability of vaccines. After clustering, less than a fifth of health professionals appeared to be firmly reluctant towards vaccination, and most are hesitant. Overall, vaccine willingness grew with age, educational background and was higher among men with underlying conditions. The main determinant of vaccine hesitancy seemed to be the fear of adverse events.
Unsupervised classification methods allowed us to investigate healthcare workers as a heterogeneous population and to identify that a majority were hesitant in their willingness to get vaccinated against COVID-19. Compared to multiple regression models, unsupervised clustering measures average perceptions within subgroups having homogeneous profiles of responses to a set of questions though having different responses to individual questions. Therefore, instead of using a subjective self-reported measurement of vaccine acceptance for identifying hesitant healthcare workers, we are using their global attitude towards vaccination. Such approaches are helpful to use perceptions and attitudes of each cluster towards vaccination in cluster targeted-communication campaigns for increasing vaccine acceptance within hesitant staff members.
As such, safety seems to be a genuine concern for healthcare workers, beyond efficacy and length of immunization. This is highlighted by the recent global alert for the AZD1222 adenovirus vaccine. If a choice were to be given, healthcare workers might favor vaccines with the least side effects, including mRNA vaccines [11,13]. Consistently, a recent cohort study conducted among healthcare workers in England showed that 89% of the personnel had been vaccinated with the BNT162b2 mRNA vaccine two months after initiating the immunization program. In that study, the characteristics of unvaccinated personnel were similar to those describing our clusters with the lowest vaccine acceptance [14]. Within our cohort, workers having the least willingness to get vaccinated were also the ones having the lowest confidence in institutions and pharmaceutical companies and an incorrect perception of the efficacy of alternative medicines against COVID-19.
The availability of vaccines is likely to be a driver of acceptance in the context of the deadly epidemic persisting. Indeed, the proportion of vaccinated respondents sharply increased after the generalized access to vaccination started in late January. After the 10 March 2021, 59% of respondents to the poll were already vaccinated against COVID-19, versus 46% in the national surveillance of French healthcare workers [15]. Since mid-2020, 15 studies investigated the willingness, and 3 reported the scaling-up of COVID-19 vaccination among healthcare professionals [14,. Taken together, we can estimate the overall willingness to get vaccinated at 65% (95CI 56% to 73%) among healthcare workers, with a prediction interval ranging from 18% to 94%, and varying by region of the globe (Supplementary Figure S3).
Additionally, global acceptance of mandatory immunization programs was low in our study and correlated with higher COVID-19 vaccine acceptance. This suggests that a mandatory scaling-up of vaccination could be counter-productive and may lead to rejection, particularly in populations with little inclination for vaccination. Coverage for hepatitis B was high among professionals not willing to get vaccinated at all, but in France, this vaccine is mandatory for newly hired hospital staff members. On the other hand, these personnel had a 17.6% coverage for flu. However, from a pragmatic point of view, it seems unrealistic to assess the eligibility for work of caregivers based on their immunization. Communication is therefore essential and should be adapted to healthcare workers' profiles to turn hesitancy into acceptance. As suggested by our study, vaccine acceptance could also increase with its availability and accessibility.
If the number of positive safety reports were to be increased in the coming months, one could expect that anxiety and fear towards adverse event would decrease. Other components such as efficacy, and length of immunization, would become more relevant, especially in light of the emergence of variants of concerns and recombining viruses [39,40].
Most (60%) professionals were hesitant towards vaccination. Despite higher flu vaccine acceptance in the United Kingdom, the recent reporting of 89% coverage for the first dose of the BNT162b2 mRNA vaccine in England brings hope that hesitant staff will get vaccinated at some point [14]. To that end, training peers using motivational interviewing techniques could also be a promising technique [41,42].
There are many limitations to this study. First, the sample is relatively small and sampling bias with strong opinions about the vaccine. Despite reminders, less than 15% of the targeted population responded to the survey, and the observed coverage for vaccination can result from a biased selection towards a population likely to accept the vaccine. Additionally, as the self-questionnaire was online and anonymous, we cannot exclude the possibility of duplicated responses. Females represented the majority of our sample, but administrative data describing the workforce of the participating sites showed a similar sex ratio. Nearly two-third received yearly vaccination against influenza, which is higher than usually reported for healthcare workers in France [10]. Most of the respondents were caregivers from public healthcare institutions, while the French National Institute of Statistics and Economies Studies reports that up to 30% of healthcare workers are administrative personnel, and half of the caregivers are from the private sector. Additionally, three-quarters of the staff members surveyed reported having been vaccinated at the last poll time point. This is higher than the actual national coverage, estimated at 46% overall [13]. Access to the vaccine was limited to health care workers over 50 years of age during most of the survey period. Therefore, their willingness to be vaccinated may be poorly estimated. Among unvaccinated personnel, we observed a relative decrease in willingness to get vaccinated after the temporary ban of the AZD1222 adenovirus vaccine (10 March 2021) by the European Medical Agency. Interestingly, the features of case scenario #6 were somehow close to the AZD1222 adenovirus vaccine and were associated in responses to a sharp decrease in vaccine acceptance. However, at the same time, vaccine availability increased, and more than half of the respondents had been vaccinated, inducing a biased measurement for vaccine hesitancy. Second, we did not randomize the scenario assignment, there was no individual follow-up over time, and it was thereby not possible to draw a causal inference. Nevertheless, the sequential ordering of the scenario allowed us to use longitudinal analysis and to identify clusters of respondents based on their trajectories of responses. Finally, vaccine availability will increase over time and possibly lead to an increase in vaccine acceptance. In France, the current trend for the scaling of vaccination is a 0.5% daily linear increase. However, an asymptote could be reached in the near future. In this perspective, continued monitoring of vaccine coverage and acceptance is crucial.

Conclusions
Healthcare workers are a heterogeneous population, but most (80%) could accept the vaccination against COVID-19. Their willingness to get the vaccine increased over time and with access to immunization programs. Among hesitant professionals, the fear of adverse events is the main concern. Targeted information campaigns reassuring about adverse events may increase vaccine coverage in a population with a strong opinion about mandatory immunization programs.
Supplementary Materials: The following are available online at https://www.mdpi.com/article/10 .3390/vaccines9060547/s1, Figure S1: Self-perception of own empathy within each cluster, Figure S2: Effect of efficacy, adverse events, and length of immunization on vaccine acceptance among hesitating healthcare workers, Figure S3: Global willingness to get vaccinated towards COVID-19 among healthcare workers. The study questionnaire.