Association between Risk Communication Format and Perceived Risk of Adverse Events after COVID-19 Vaccination among US Adults

The format used to communicate probability—verbal versus numerical descriptors—can impact risk perceptions and behaviors. This issue is salient for the Coronavirus disease 2019 (COVID-19), where concerns about vaccine-related risks may reduce uptake and verbal descriptors have been widely used by public health, news organizations and on social media, to convey risk. Because the effect of risk-communication format on perceived COVID-19 vaccine-related risks remains unknown, we conducted an online randomized survey among 939 US adults. Participants were given risk information, using verbal or numerical descriptors and were asked to report their perceived risk of experiencing headache, fever, fatigue or myocarditis from COVID-19 vaccine. Associations between risk communication format and perceived risk were assessed using multivariable regression. Compared to numerical estimates, verbal descriptors were associated with higher perceived risk of headache (β = 5.0 percentage points, 95% CI = 2.0–8.1), fever (β = 27 percentage points, 95% CI = 23–30), fatigue (β = 4.9 percentage points, 95% = CI 1.8–8.0) and myocarditis (β = 4.6 percentage points, 95% CI = 2.1–7.2), as well as greater variability in risk perceptions. Social media influence was associated with differences in risk perceptions for myocarditis, but not side effects. Verbal descriptors may lead to greater, more inaccurate and variable vaccine-related risk perceptions compared to numerical descriptors.


Introduction
Three years ago, the Center for Disease Control confirmed the first case of Coronavirus disease 2019  in the United States and scientists believe the virus will not be irradicated for years to come. Vaccines remain as one of the safest strategies to prevent severe disease [1]. However, fear of COVID-19 vaccine-related side effects and adverse events may impede vaccine uptake. Recent studies have shown that many adults rank a fear of COVID vaccine side effects highly in the context of major global health threats [2]. Currently, these risks are described using verbal terms (e.g., common, rare) by major public health as well as media organizations [1]. While well-intentioned, this approach may be problematic because the format for communicating risk-using verbal versus numerical descriptors of probability-can impact risk perceptions and decision making [3,4]. These dynamics are particularly relevant in the age of social media, given its rapid and significant influence on the public's beliefs, attitudes and behaviors. COVID-19 vaccine information has been widely conveyed via social media platforms [5], where high rates of disinformation may lead to vaccine hesitancy [6].
Verbal descriptors can be associated with inaccurate and inflated perceptions of risk, which in turn affect medical treatment decisions such as choosing to initiate a new drug or enter a clinical trial [3,7,8]. The impact of risk communication methods on risk perceptions can vary by context and condition. Currently, little is known about how risk communication formats affect the perceived risk of both minor and serious adverse events after COVID-19 vaccination.
We addressed this evidence gap by testing how risk communication format-numerical versus verbal descriptors of risk-for three side effects (headache, fever and fatigue) and adverse event (myocarditis) affect individuals' perceptions of the risk of these events occurring after vaccination against COVID-19. The aim of this study was to assess the impact of risk communication format on risk perceptions. We hypothesized that risk communication format would affect risk perceptions, with verbal descriptors leading to more variable and inflated risk perceptions. We also hypothesized that participants who rate social media as an important source of information would have greater perceptions of vaccine-related risks regardless of the risk communication format.

Methods
We conducted a web-based randomized survey in January 2022 among adults in the United States who had not received all main series and booster vaccinations. Participants were recruited as a convenience sample using Amazon's Mechanical Turk (MTurk) crowdsourcing platform, which has been extensively utilized to study healthcare decision making and risk perceptions in US adults [4,9,10]. Eligible participants were ≥18 years old, lived in the United States and had an MTurk approval rating of >98%. We excluded participants who indicated their age > 90. Participants who stated they had not received all main series and booster vaccinations were randomized using 1:1 simple randomization to receive information about headache, fatigue, fever and myocarditis after COVID-19 vaccination using either verbal descriptors or numerical point estimates. Headache, fatigue and fever were described as "common" or 41%, 47%, 8%, respectively, while myocarditis was described as "rare" or "0.001205%" based on data obtained from the US Centers for Disease Control and Prevention website in October 2021 [1]. After receiving information in verbal versus numerical form, participants then rated their perceived risk on a scale of 0-100% in response to the following question: "What is your risk of getting [side/adverse effect] after the [next indicated dose]?" The survey also captured information about participant demographics and political identity (7-point scale from 0 = Conservative to 7 = Liberal). Participants were asked their intention toward getting the next indicated vaccine dose (definitely, not sure, definitely not); those that reported being unsure were also asked whether they needed more time to decide. Based on these responses, participants were defined as either vaccine accepting (definitely would get next vaccine dose) vaccine rejecting (definitely would not), vaccine deferring (not sure, need more time to decide), or otherwise vaccine hesitant (not sure, do not need more time to decide).
The survey also asked participants to report the extent to which they were impacted by different sources of information about COVID vaccines (7-point scale): social media, loved ones, doctor or other healthcare professional, religious community, work or school community, news media, public health organizations and others in their lives. Participants reporting an impact from social media that was greater than or equal to all other sources were defined as high social media influence.

Statistical Analysis
Findings were described using means and standard deviations (SD) for headache, fever and fatigue. Given the highly skewed distribution of responses for myocarditis, we described the data using medians and interquartile ranges (IQRs). For each of the four adverse events, a linear regression model with robust standard errors was used to assess the impact of verbal versus numerical risk communication format on individuals' perceived risk, accounting for sociodemographic characteristics, political identity, vaccine hesitancy and social media influence. All hypothesis tests were two-sided and an alpha of 0.05 was used for statistical significance. Analyses were performed in R version 4.1.0 packages gtsummary, tidyverse and ggplot2 [11].
This study was approved by the University of Washington Institutional Review Board (STUDY00014136). We followed the American Association for Public Opinion Research Reporting guidelines for survey studies (Table S1).

Results
Overall, 939 participants were included in the analysis; 575 (61%) of whom were fully vaccinated and not boosted, 61 (6.5%) of whom were partially vaccinated and 303 (32%) of whom were unvaccinated. A total of 495 individuals were randomized to numerical descriptors of probability and 444 were randomized to verbal descriptors. Mean age was 39 years and 43% identified as female. Participant characteristics were well balanced between the two study arms (Table 1).

Headache, Fever and Fatigue
For all three "common" side effects, mean risk perceptions were higher and more variable in the verbal descriptor than numerical descriptor arm. These differences were largest for fever, with mean estimated risk in the verbal descriptor of 41.4% (SD = 28.9%) versus 15.4% (SD = 15.4%) in the numerical descriptor arm. Differences between arms were smaller for headache (verbal descriptor mean = 45.4%; SD = 28.9% vs. numerical descriptor mean = 40.9%; SD = 17.2%) and fatigue (verbal descriptor mean = 49.6%; SD = 30.4% vs. numerical descriptor mean = 45.3%; SD = 17.5%). Verbal descriptors also led to greater differences between participants' risk estimates and published risk estimates (Figure 1), corresponding to greater inaccuracy in risk perceptions. For each side effect, standard deviations were also larger in the verbal versus numerical descriptor arm, indicating increased variability in participants risk perceptions when presented with verbal descriptors of risk ( Figure 2).
We did not observe an association between high social media influence and differences in side effect risk perceptions (Table 2). Self-reported political identity was not linearly associated with perceived risk of headache, fever, or fatigue (Table 2). Individuals in the vaccine-rejecting or vaccine-deferring groups were more likely to possess increased perceptions of the risk of headache and fatigue, but not fever compared to individuals in the vaccine-accepting group. In contrast, we did not find increased risk perceptions amongst those categorized as vaccine-hesitant (i.e., those who stated they need more time to decide). Male gender identity was associated with lower mean risk perceptions for all three side effects ( Table 2). In multivariable analysis, verbal descriptors were associated with higher perceived risk for all three side-effects: headache (β = 5.0 percentage points, 95% CI = 2.0-8.1), fever (β = 27 percentage points, 95% CI = 23-30) and fatigue (β = 4.9 percentage points, 95% CI = 1.8-8.0) ( Table 2).
We did not observe an association between high social media influence and differences in side effect risk perceptions (Table 2). Self-reported political identity was not linearly associated with perceived risk of headache, fever, or fatigue (Table 2). Individuals in the vaccine-rejecting or vaccine-deferring groups were more likely to possess increased perceptions of the risk of headache and fatigue, but not fever compared to individuals in the vaccine-accepting group. In contrast, we did not find increased risk perceptions amongst those categorized as vaccine-hesitant (i.e., those who stated they need more time to decide). Male gender identity was associated with lower mean risk perceptions for all three side effects ( Table 2).   Figure 2. Histograms of participants perceived risk for each complication demonstrating increased variability in risk estimates with verbal probability descriptors. Red bars represent numerical probability descriptors and green bars represent verbal probability descriptors.

Myocarditis
Participants exposed to a verbal descriptor ("rare") had higher median perceived myocarditis risks and greater variability in perceived risk than those exposed to numerical probabilities (median = 2.0%, IQR 0.1% to 15% vs. median = 0.001%, IQR 0.001% to 1.0%). In multivariable analysis (Table 2), verbal descriptors were associated with a higher perceived myocarditis risk (β = 4.6 percentage points, 95% CI = 2.1-7.2). Participants most influenced by social media also had higher perceptions of myocarditis risk (Table 2). More liberal political identity was associated with a lower perceived risk of myocarditis after vaccination.
Compared to those who were vaccine-accepting, respondents who were vaccinedeferring or vaccine-hesitant expressed lower risk perceptions, with risk perception confidence intervals that included more negative value (though confidence intervals for vaccinehesitant individuals included small positive values). Vaccine-rejecting individuals did not have a clear difference in risk perceptions compared to those who were vaccine-accepting. Male gender identity was also associated with lower mean risk perceptions of myocarditis ( Table 2).

Discussion
In this randomized survey, verbal descriptors of the COVID-19 vaccine side effect risks led to less accurate, more variable and heightened risk perceptions compared to numerical risk descriptors for several vaccine-related side effects and adverse events.
The main implication of these findings is that going forward, public health, medical and news media organizations may benefit from reconsidering their approach to engaging the public about vaccine-related risks. In particular, our results underscore the potential value of moving away from conveying risks solely using verbal descriptors and providing numerical estimates of risk.
In some circumstances, such changes may require concerted efforts to avoid verbal descriptors. Verbal probability descriptors are attractive and widely used in health and public health communication because they are concise, easy to communicate conversationally and conveyable to individuals with variable numeracy [3,7,12]. However, our findings add to a body of knowledge outside the COVID-19 context and demonstrate that despite good intentions, there are potential pitfalls in using these descriptors [3,4,7,8].
Verbal descriptors permit ambiguity, require interpretation and rely on individuals' existing subjective sense of risk [3,7,8,12,13]. For all three side effects assessed in this study, participants in the verbal descriptor arm estimated risks consistently between 40 and 50%, likely reflecting internal reference points and connections between the word "common" and that risk range. Because the published risks of developing fatigue and headache also happened to be in this range, there were only small differences between participants' estimated risk perceptions and published risk estimates. The larger difference of 26% for fever may suggest discrepancies that can arise between participants' internal reference points and published risk values [8].
Our data regarding myocarditis are particularly striking: median risk perceptions for verbal descriptors were over 1000 times higher than for numerical descriptors. Prior work has shown that inflated risk perceptions caused by verbal descriptors can decrease patient's willingness to participate in clinical trials [7] or start medications [8]. It is therefore likely that misperceptions of this magnitude may also affect patient's willingness to undergo vaccination against COVID-19.
Even when the average risk perceptions were similar between numerical and verbal descriptors, our data demonstrate that verbal descriptors had much higher variability in risk perceptions at the individual level. For instance, the mean risk perceptions for headaches were similar for verbal and numerical descriptors (46% vs. 41%) but the spread of risk estimates was much higher for verbal descriptors (standard deviation of 29% vs. 17%). While verbal descriptors might appear to perform adequately at the population level, these data imply that many individuals still will have over-or, just as importantly, underappreciated their chance of developing side effects and adverse events [3,7,8].
Without a benchmark for descriptors such as "common" (i.e., common compared to what?), individuals must supply their own comparator and estimate risk in the context of their own beliefs, biases and experiences [12]. For example, someone who received the first dose of a COVID-19 vaccine and developed a fever may interpret "common" very differently than someone who did not. Prior studies have shown that in addition to personal factors, the nature of the data itself matters as well; for instance, side effects of different severity described with the same verbal probability term are often interpreted to have different likelihoods of occurring [13][14][15].
Of course, numerical descriptors are not a panacea. Guidelines for how to present such information-for example as percentages, natural frequencies, or graphical displays-are not always clear for all situations [12]. Numerical values may also convey a degree of certainty in information that is not supported by existing evidence. These dynamics may have combined with individuals' past experiences or beliefs to explain the variability observed in the numerical descriptor arm between the estimates provided to participants and their self-reported risk perceptions. Future work should seek to capture and then address these potential shortcomings, which can be performed in several ways. One is to present data in the form of ranges instead of precise point estimates. Another is to provide contextual cues and graphical displays to place numerical data in the context of events that are likely more familiar to patients [12].
These steps are particularly worthwhile given the ubiquity of social media [16] and findings from our study. In particular, risk perceptions about myocarditis were greater among individuals reporting high social media influence. Though more work is needed to elucidate specific mechanisms behind this relationship, our findings nonetheless highlight the potential impact of social media on risk perceptions and the importance for various stakeholders to recognize this potential dynamic in determining formats for communicating risk probabilities [6,17].
Study limitations include the use of an online sample that may not represent all populations or capture changes in perceptions over time. However, while there may be differences between our sample and populations in either the US or other countries, the MTurk population has been used extensively for research in the social and health sciences [9,10,18,19]. Our findings are also highly salient to the broader pandemic situation given the prevalence and consistent use of verbal descriptors by public health and news organizations. Moreover, our findings are nonetheless valid given the randomized design and the goal of using randomization was to estimate the marginal effect of risk communication format between groups, rather than describing perceptions in a given population. Future work should elucidate changes in perceptions over time in different populations such as non-US populations and those with different levels of numeracy, graphical literacy and risk tolerance [12]. An additional limitation of this study is the overall lack of a validated survey instrument for the measurement of risk perceptions based on risk descriptors. However, the methodology employed in our study has been used extensively in the literature to assess risk perceptions in participants exposed to different risk communication formats [3,4,7,8,19]. We did not measure participants' prior knowledge about vaccine adverse side effects or prior medical knowledge; however, the randomized design of this experiment should mitigate the impact of any bias introduced by these covariates on the marginal effect of risk communication strategies.

Conclusions
In conclusion, describing COVID-19 vaccine-related risks using verbal descriptors may lead to more inaccurate, variable and heightened perceptions of risk among US adults. Individuals reporting a high influence of social media may have heightened perceptions of some vaccine-related risks independent of risk communication format. Medical, public health and news media organizations can benefit from considering numerical risk estimates as an alternative to verbal descriptors in public communications to promote appropriate risk perceptions.