Impact of Local and Demographic Factors on Early COVID-19 Vaccine Hesitancy among Health Care Workers in New York City Public Hospitals

Despite the development of several effective vaccines, SARS-CoV-2 continues to spread, causing serious illness among the unvaccinated. Healthcare professionals are trusted sources of information about vaccination, and therefore understanding the attitudes and beliefs of healthcare professionals regarding the vaccines is of utmost importance. We conducted a survey-based study to understand the factors affecting COVID-19 vaccine attitudes among health care professionals in NYC Health and Hospitals, at a time when the vaccine was new, and received 3759 responses. Machine learning and chi-square analyses were applied to determine the factors most predictive of vaccine hesitancy. Demographic factors, education, role at the hospital, perceptions of the pandemic itself, and location of work and residence were all found to significantly contribute to vaccine attitudes. Location of residence was examined for both borough and neighborhood, and was found to have a significant impact on vaccine receptivity. Interestingly, this borough-level data did not correspond to the number or severity of cases in the respective boroughs, indicating that local social or other influences likely have a substantial impact. Local and demographic factors should be strongly considered when preparing pro-vaccine messages or campaigns.


Introduction
New York City was the epicenter of the COVID-19 pandemic in the US in early 2020. As the primary provider of health services for the city's vulnerable patient population, the city's public health system (NYC Health and Hospitals) bore a disproportionate burden of the pandemic [1]. The healthcare workers of the system responded heroically to the unprecedented challenge to ensure the best care for their patients, in spite of facing potential exposure to the virus through their jobs. While there exists a significant gap in data collection, it has been shown that the majority of cases and deaths related to COVID-19 among healthcare workers (HCW) have been in people of color who work in areas where risk of workplace exposure is high [2].
Vaccines have historically had a significant positive impact on public health, reducing disease transmission and/or decreasing the incidence of serious diseases such as polio, smallpox, diphtheria, hepatitis, and others [1]. In December 2020, the U.S. Food and Drug Administration (FDA) issued Emergency Use Authorizations (EUA) for the first

Random Forest Analysis
Machine learning was applied to a table that included the anonymized responses for all questions (columns) across all participants (rows). To evaluate responses to the statement "I will get the vaccine within the next two months", all "Yes" or "NA" answers were removed so that only the "No" and "undecided" answers remained. Empty values in columns with incomplete responses were imputed using the average value across the column. This is required since random forest requires data to be present for all cells in the table. Random forest analysis (Random Forest version 4.6-14 library in R) was used to generate and analyze 10,000 trees with 10 variables tried at each split. The value imputation further reduced the noise since many columns only had a handful of responses, which were generally represented by the same answer. This effectively removed these columns from consideration. After this process, there were 91 columns that remained in the analysis, compared to the original 115 columns. The importance of each feature was ranked according to the mean decrease in Gini value, which was computed by the model generated by the random forest algorithm. These data were then used to generate a

Statistical Analysis
The hospital system employed 35,000 employees and the sample size was calculated with a confidence level of 95% and a 2% margin of error to give a calculated sample size of 2247 participants. Chi-square (χ2) analysis was used to analyze dependency of variables between different groups when the variables were discreet. When the variables were continuous, ANOVA was used to determine whether any statistically significant differences were detectable. Neighborhoods within New York City where the respondents lived were identified using ZIP (Postal) codes [15]. The magnitude of the effect for each vaccine-hesitant group was calculated by modifying the method of Sullivan and Feinn [16]. The median of each group was calculated, then we used the formula item−median interquartile range . This shows the proportion of the interquartile range from the median for each item, making effect size apparent and allowing a comparison across groups.

Study Population
Of the 38,000 individuals who were sent a link to the survey, we received 3759 responses, a rate of approximately 10%. We therefore began by summarizing the wide range of racial/ethnic and educational backgrounds represented by the survey participants, as shown in Table 1.
This section may be divided by subheadings. It should provide a concise and precise description of the experimental results, their interpretation, as well as the experimental conclusions that can be drawn.
Since we sent the survey to the entire healthcare system, a wide variety of different roles within the hospital were included in the responses. To better understand the distribution of roles, we summarized the employment roles of respondents (Table 2). We observed that nurses were the largest group represented, with 17.6% of respondents, followed by physicians, with 17.1%. Administrative support staff were the next largest group of respondents, and many other roles were represented.

Vaccine Status
We administered this survey after preliminary approval of the COVID-19 vaccines, when the vaccines were available primarily to healthcare personnel. Of those who participated in the survey, 71% had already received their first vaccination dose. Of those who had not been vaccinated, 33% planned on being vaccinated, while 21.4% were not and 45.6% were unsure. For the purposes of this paper, "vaccinated", "hesitant", "hesitancy" or "refusal" refers only to the COVID-19 vaccine. When the respondents' attitudes toward vaccines in general is relevant, it will be indicated.

Demographics
Age, race, and gender were all significantly associated with vaccine hesitancy and vaccine refusal (p < 0.001 for all groups) ( Table 3). The youngest age group (18)(19)(20)(21)(22)(23)(24) was the least likely to be vaccinated, and the most likely to be hesitant or refuse vaccination. The over-65 age group was the most likely to be vaccinated. Hesitancy among the other age ranges was not substantially different in magnitude. We discovered that race was also significantly associated with both vaccine hesitancy and refusal. Asian and White participants had the highest level of vaccination, with Black, Native American, and "Other" participants at the lower end. Gender was significantly associated with vaccination status, with men being more likely than women to be vaccinated (87% vs. 77%). Non-binary and "prefer not to answer" participants were the most likely to be unvaccinated, although their numbers were much smaller.

Education and Hospital Role
Level of education was significantly associated with vaccine hesitancy and refusal (p-value < 0.001). Generally, the more education a person had obtained, the more likely they were to be vaccinated, with the exception of those with "some high school" for which the numbers were too small to be meaningful. Interestingly, among those who were not vaccinated, respondents with a high school diploma or GED were more likely to be hesitant (24% for each) than refusers, while those with some college, an associate's degree, or a bachelor's degree all had higher rates of refusal than the high school or GED individuals, even though their overall vaccination rates were higher (Table 4). Hesitant effect size measures proportion of interquartile range from the median for each item. Negative numbers are less likely to be vaccine hesitant.
The respondent's role in the hospital was significantly associated with the vaccination status of the person. Medical staff (including physicians) were highest at 95%, while the hospital police were the lowest, at 35% vaccinated. Clinical support staff were next highest, at 86% vaccinated. Nurses were 75% vaccinated, 16% hesitant, and 8% refusers. Interestingly, considering the nature of their jobs, community outreach tracers were only 64% vaccinated and 32% hesitant (Table 4).

Reasons for Hesitancy towards the COVID-19 Vaccine
The survey enabled respondents who indicated that they would not receive the vaccine to be asked additional questions in an effort to determine why they made that decision. For this question, the respondents could select as many choices as applied to them. The two most commonly cited reasons were concerns with the speed and thoroughness of testing of the vaccines, and the fear of side effects (Figure 1). The next most common reason was a lack of trust in the people advocating for the vaccines, followed by general anti-vaccine attitudes. cine to be asked additional questions in an effort to determine why they made that decision. For this question, the respondents could select as many choices as applied to them. The two most commonly cited reasons were concerns with the speed and thoroughness of testing of the vaccines, and the fear of side effects (Figure 1). The next most common reason was a lack of trust in the people advocating for the vaccines, followed by general anti-vaccine attitudes.

Figure 1. Reasons for vaccine attitudes and behaviors. (A)
. Unvaccinated respondents who indicated that they would not be vaccinated in the next three months (refusers) were asked why (n = 231). (B). Those HCW who did not receive the first dose of the vaccine but indicated they were unsure if they would or not (hesitant) were asked what would help them to make a decision (n = 490).
Since the question asking about intent to be vaccinated had an option for "unsure," those who were unsure (hesitant) were further queried about what would help them make a decision about vaccination. Here, the answers were also grouped, with the most common responses indicating that more time or testing on a larger number of people would be most effective. Of the responses where action could make a difference, having the vaccine receive full FDA approval was the most commonly selected option (30.81%), followed by having a conversation with a doctor (19.59%). Since this study was comprised of a population where the vaccine was available and in fact encouraged their employment, vaccine access was not a major issue, with only 0.4% indicating that a "closer location" and only 1.6% saying "increased access" would help them decide to be vaccinated ( Figure 1).

Machine Learning Analysis
In order to determine which answers were most associated with a respondent being COVID-19 "vaccine hesitant" we applied a random forest machine learning algorithm, which generates a computational model to predict classification of data based on a discreet variable (e.g., vaccine hesitancy) and other variables [17]. This approach was specifically selected since so many different factors/variables were significantly associated using more traditional statistical methods. Our approach used the mean decrease in Gini value, a quantitative metric of uncertainty, for each question or feature. Ranking the list of factors/features by this value enabled us to construct a list of the best predictors for vaccine hesitancy ( Table 5). All other questions had Gini values that were extremely low. Overall, Since the question asking about intent to be vaccinated had an option for "unsure," those who were unsure (hesitant) were further queried about what would help them make a decision about vaccination. Here, the answers were also grouped, with the most common responses indicating that more time or testing on a larger number of people would be most effective. Of the responses where action could make a difference, having the vaccine receive full FDA approval was the most commonly selected option (30.81%), followed by having a conversation with a doctor (19.59%). Since this study was comprised of a population where the vaccine was available and in fact encouraged their employment, vaccine access was not a major issue, with only 0.4% indicating that a "closer location" and only 1.6% saying "increased access" would help them decide to be vaccinated ( Figure 1).

Machine Learning Analysis
In order to determine which answers were most associated with a respondent being COVID-19 "vaccine hesitant" we applied a random forest machine learning algorithm, which generates a computational model to predict classification of data based on a discreet variable (e.g., vaccine hesitancy) and other variables [17]. This approach was specifically selected since so many different factors/variables were significantly associated using more traditional statistical methods. Our approach used the mean decrease in Gini value, a quantitative metric of uncertainty, for each question or feature. Ranking the list of factors/features by this value enabled us to construct a list of the best predictors for vaccine hesitancy ( Table 5). All other questions had Gini values that were extremely low. Overall, the model was able to accurately classify people answering "Unsure" about whether they would be vaccinated 100% of the time, while the model was unable to accurately classify (0%) those answering "No" when asked if they would be vaccinated. Overall, the accuracy rate of this method was 64.8% and a receiver-operator characteristic curve showed an overall performance of 65.2%. This suggests that those answering "No" had no consistent associated answers, while the survey identified several factors/features that associate with those answering "Unsure" (Table 5). Supporting the demographic analysis above, the machine learning approach identified work role, age, and education as being highly important in determining whether a respondent would be vaccine-hesitant. Attitudes about the severity of COVID-19 and vaccinations in general were also highly associated with this role, with "How has your experience with COVID-19 changed your overall opinion on vaccinations" the top predictors and "Based on your overall experience, how serious is COVID-19" at number 5. Interestingly, the location where a respondent works was highly associated with COVID-19 vaccine hesitancy.

Location of Respondents
New York City is diverse, with different neighborhoods having different characteristics. The hospitals in the system were differentially affected by the pandemic, with the facilities in Queens and Bronx bearing the brunt of cases. With this background, we aimed to determine if the location of work is a significant factor in terms of vaccine hesitancy (Table 5).
We found that the vaccination status of respondents was dependent on the borough in which they worked (p < 0.001). Queens was the borough with the highest level of vaccination, at 84%, while Staten Island was by far the lowest, at 62%. However, the number of respondents who worked in Staten Island was considerably lower than the other boroughs (only 45 responses), so this finding may be biased. Intent to be vaccinated significantly (p < 0.01) depended on work location. Queens had the lowest proportion of hesitant respondents, at 11%, and had a low proportion of refusers, at only 4%. Staten Island had a high proportion of vaccine hesitant respondents (26%) and refusers (13%).
We evaluated whether where the respondents lived had an effect on vaccination status, and found a significant association (p < 0.001). In this case, those living in Westchester were most likely to be vaccinated, at 88%, and those living in the Bronx were least likely, at 70%. Among the unvaccinated, those with the highest ratio of hesitant to refusers lived Upstate (4.1:1), whereas in Staten Island, the number of vaccine refusers was nearly as high as hesitant individuals (ratio of 1.22:1).
Since the impact of COVID-19 was different between boroughs, we used ANOVA to determine whether there was any significant difference in how the seriousness of the pandemic was perceived between boroughs. We observed no significant overall difference in how respondents rated the seriousness of COVID-19 based on where they live (p = 0.17).
Within each borough, there are many neighborhoods with substantial differences in population attributes, including vaccination rates. We used ZIP codes to determine the specific neighborhoods where the respondents lived within each borough, and these were compared for differences in vaccination rates. HCW in the Bronx demonstrated a dependence on vaccination rate by neighborhood (p < 0.0021). The most vaccinated location in this borough was Kingsbridge, with 87.75% vaccinated, and the lowest was High Bridge, with 48.33% vaccinated (Figure 2).
Within each borough, there are many neighborhoods with substantial differences in population attributes, including vaccination rates. We used ZIP codes to determine the specific neighborhoods where the respondents lived within each borough, and these were compared for differences in vaccination rates. HCW in the Bronx demonstrated a dependence on vaccination rate by neighborhood (p < 0.0021). The most vaccinated location in this borough was Kingsbridge, with 87.75% vaccinated, and the lowest was High Bridge, with 48.33% vaccinated (Figure 2).  Similarly, Brooklyn neighborhoods were significantly different in vaccination rates of the health care professionals who lived there (p = 0.00028). Brooklyn neighborhoods seemed to cluster into three groups. Greenpoint and Northwest Brooklyn had high levels of vaccination (over 80%). East New York/New Lots and Bushwick/Williamsburg had low levels, with less than 60% vaccinated. The rest of the neighborhood health care workers had between 60 and 80% rates ( Figure 2).
Manhattan, interestingly, showed a difference by neighborhood (p = 0.017) but the variance between neighborhoods was less pronounced. No neighborhood in Manhattan had hospital workers vaccinated at less than 65%. The neighborhood with the lowest rate was Washington Heights, with 66% vaccinated and that with the highest was the Upper East Side, with 85.59%.
Queens did not demonstrate a significant difference between neighborhoods. Staten Island did show a significant difference between neighborhoods (p = 0.022). Staten Island also had the neighborhood with the lowest vaccination rate, Port Richmond, at only 25%. However, this result is based on only 12 respondents in that neighborhood.

Discussion
Vaccine hesitancy is a significant and complex challenge to public health. It is a spectrum including attitudes and behaviors ranging from complete refusal to complete acceptance of vaccination [18,19]. It has led to a delay in getting vaccinated or modification of schedule of vaccination (partial vaccination) [20]. Historically, vaccine hesitancy has been impacted by cultural, social, and political influences in addition to vaccine related factors [21,22].
We performed a survey-based study to understand the factors affecting COVID-19 vaccine attitudes among healthcare professionals in New York City, at a time when the vaccine was new. The results of this study show that a large variety of factors are associated with vaccine attitudes between HCW. These include demographic factors, education, work role, attitudes about the pandemic, and the location of where the respondent lives and works. Notably, we observed that in the early period of the vaccine rollout, the rates of vaccine hesitancy/refusal among HCW were comparable to the general population in New York City [23]. This observation was remarkable, as HCW have been on the frontlines caring for the COVID-19 patients and are trusted sources of vaccine information, with an ability to influence the "choice to vaccinate" for their patients and social contacts [24].

Demographics and COVID-19 Vaccine Hesitancy
Demographic factors, including age, race, and gender, were all significantly associated with hesitancy toward the COVID-19 vaccine. The Household Pulse Survey also demonstrated that COVID-19 vaccine hesitancy was higher among women when compared to men, younger age groups when compared to respondents over 65 years and participants with college degrees compared to those without a degree [25]. Our results are comparable to the data from the Kaiser Family Foundation/The Undefeated survey, which demonstrated that among HCWs there is a substantial cohort that is vaccine hesitant, especially among Black HCW [26]. The role at the hospital was confirmed to be an important factor in COVID-19 vaccine hesitancy by both the χ2 analysis and the machine learning analysis. Notably, we observed a significantly lower vaccine acceptance among maintenance/environmental staff, hospital police and pharmacies, while Medical staff including physicians, residents, nurse practitioners, and physician assistants were highest. Remarkably, those who were directly addressing COVID-19 (community outreach tracers) had some of the lowest levels of vaccination and the highest hesitancy. It was notable that nurses, who have the greatest contact with patients, had a high rate of vaccine hesitancy in spite of being a trusted source of medical information for the patient. A cross-sectional study of nurses in Hong Kong showed that less than two-thirds intended to get the COVID-19 vaccine when available [27]. We believe that a significant reason for HCW in the child-bearing age group being hesitant to COVID-19 vaccination in our survey is because the survey was administered in the initial phase of the vaccine roll out, when there was a paucity of comparative data on vaccine studies in pregnant women and its correlation with birth outcomes [28,29]. Social scientists have commented that people who reject vaccines harbor a transformation of their core beliefs about what they owe one another and are not scientifically less literate or knowledge-able than those who accept vaccination [30]. During the 1950 polio campaign, Americans accepted vaccination as a civic duty. However, in the following years, the "choice" of an individual's health became their personal responsibility, in spite of a multitude of factors, especially "social determinants of health" that are beyond the individual's control and impact their choices. These have contributed to a shift in people's mind, from the idea of the common good.

Location of Work and Home
The geographic pattern of vaccine hesitancy is also significant. Nationally, it has been observed that hesitancy rates are lowest on the North East and West coast and highest in the South, Mid-West/Central US and Alaska with higher rates of hesitancy in rural communities when compared to urban ones [25,31]. The COVID-19 pandemic has disproportionately affected communities of color in NYC. Neighborhood social disadvantage contributed to higher number of infections in these communities [32]. The lowest vaccination rates by borough, as per NYC Department of Health and Mental Hygeine data 31 May 2021 were Flatlands/Midwood, Brooklyn (33% received at least one dose), Edgemere/Far Rockaway, Queens (31% received at least one dose), Central Harlem/Washington heights in Upper Manhattan (37% received at least one dose), Hunts Point, Bronx (34% received at least one dose) with Tottenville, Staten Island (39% had at least one dose). Where a HCW lived and worked was associated with their vaccination status and hesitant attitudes. These varied by both borough and by neighborhood within boroughs. Infection rates, death, and hospitalization rates of specific boroughs did not correspond to these vaccine attitudes in the boroughs. For example, Staten Island had significantly more infections per 100,000 than any other borough (p < 0.0001, test of proportions) [33], yet Staten Island-based medical personnel had the lowest rates of vaccination and pro-vaccine attitudes.

First-Hand Experience with Serious COVID-19 Cases
The best predictor of vaccine hesitancy according to the random forest analysis was how the COVID-19 pandemic had affected the respondents' views towards vaccines in general, a finding that is concerning, as it suggests that people who are hesitant towards the COVID-19 vaccines may transfer that hesitancy to other vaccines, exacerbating the already significant vaccine hesitancy problem. Another attitude that was predictive of vaccine hesitancy was how important the respondent felt the pandemic was. Truthful information and experience with people who have suffered difficult consequences of the pandemic may be helpful in promoting vaccine acceptance among these people.
When asked why they had chosen to not receive a vaccine, "It was developed too quickly", and "I don't want to be experimented on" were the most prevalent answers. In other studies, we have found that attitudes towards vaccines in general are highly influential in making vaccine decisions [2,34], and it is likely that those with a general level of distrust in vaccines will indicate greater concern about the speed of development or the lack of testing.

Changing Attitudes over Time
Multiple studies have demonstrated a reduction in vaccine hesitancy over time [21,31]. The greatest declines in vaccine hesitancy were observed among participants in the 18-24-year age group, women and non-Hispanic Black persons as per the Household Pulse Survey results. Additionally, participants with a lower income, a younger age, and without insurance or on Medicaid were more likely to remain vaccine hesitant over time. The higher vaccine hesitancy rates among Black respondents improved over time when compared to rates among the white population. Unfortunately, the percentage of vaccine refusers have remained constant in spite of time in multiple studies [35,36]. Hence, interventions to improve vaccination uptake are likely to succeed if focused on the vaccine hesitant group rather than on the vaccine refusers [21].

Strategies for Intervention
Targeted messaging directed toward the hesitant population delivered by their trusted individuals will be effective in changing minds. Engaging vaccinated individuals from hesitant/minority communities, and developing communication aids that are culturally, linguistically, and literacy appropriate will help promote vaccination in vulnerable communities [37]. The neighborhood-level differences in vaccination (Figure 2) support the need for highly targeted interventions to improve vaccine hesitancy. Multiple health behavior models, including threat and coping appraisal, cues to action, self-efficacy, perceived benefits/barriers, perceived behavior control, and attitudes and contextual factors are known to influence routine vaccination. Applications of these models to fight the COVID-19 pandemic would be a major tool to combat vaccine hesitancy [38]. The geographical variance in vaccine acceptance suggests that while exposure to the disease is important, attitudes are moderated by other region-specific factors. Social attitudes that have not been captured by the survey may be highly important in determining attitudes towards infection severity and vaccination. Perhaps vaccination messaging that addresses the unique features of these locations would be effective in promoting COVID-19 vaccination or similar vaccination campaigns in the future.
Among the determinants of vaccine hesitancy, communication and media environment are of paramount importance. Negative communications have long term effects on public perception. For example, there is controversy over the association of the measles vaccine and autism, hepatitis B vaccine and Multiple Sclerosis [39]. News about serious adverse events (thrombosis) linked to the Janssen COVID-19 vaccine reduced confidence among the unvaccinated/hesitant especially the Hispanic women [36].
"Unspoken vaccine hesitancy" is a state when HCWs do not voice their concerns about vaccination among their peers or in the workplace due to institutional and societal pressures to vaccinate, especially in the setting of tension and polarization across communities about vaccination [6]. Additionally, studies have demonstrated that HCWs experience significant anxiety, depression and psychosocial distress related to working in the COVID-19 era [40]. Addressing vaccine hesitancy in these situations is difficult. Innovative multidisciplinary approaches to openly discuss vaccine-related concerns among these hesitant HCWs will build confidence and go a long way towards resolving their fear and anxiety.
Vaccine mandates in healthcare institutions are an attempt to protect the patients based on the ethical principle to do no harm and to create a safe environment for them. Health + Hospitals enforced the state-recommended vaccine mandate to have all HCW receive at least one dose of the COVID-19 vaccine by 27 September 2021. This initiative reduced vaccine hesitancy further and forced the exit of vaccine refusers from the workforce. There have been multiple interventions internationally, like green pass implementation, domestic vaccine passports etc. [12], to improve vaccination uptake. In the background of waning immunity following COVID-19 [41,42] vaccination and the need to boost doses to combat arising new SARS-CoV-2 variants of concern, there remains a continued role for mandates due to persistent vaccine hesitancy. While vaccine mandates affect an individual's choice (right to refuse vaccination), it is critical to remember that every HCW has an ethical and moral responsibility to do no harm and to place their patient's health goals and interests above their own [43].

Limitations
This survey was performed early in the vaccine rollout process. Our other work from approximately the same time indicates that the political influence on vaccine acceptance in the United States was not a major factor at that time [34], although it certainly is now [44]. This study also does not consider more forceful measures, such as mandates, that have increased the vaccination percentages in New York City, as well as in other locations [45]. We hope that this work will indicate the directions and areas of focus for vaccine messaging and communication in the future, as well as obstacles and issues that need to be overcome if a similar situation were to arise again.