What Motivates the Vaccination Rift Effect? Psycho-Linguistic Features of Responses to Calls to Get Vaccinated Differ by Source and Recipient Vaccination Status

Although vaccination provides substantial protection against COVID, many people reject the vaccine despite the opportunity to receive it. Recent research on potential causes of such vaccine hesitancy showed that those unvaccinated rejected calls to get vaccinated when they stemmed from a vaccinated source (i.e., a vaccination rift). To mend this vaccination rift, it is key to understand the underlying motivations and psychological processes. To this end, we used the voluntary free-text responses comprised of 49,259 words from the original Austrian large-scale data-set (N = 1170) to conduct in-depth psycho-linguistic analyses. These findings indicate that vaccinated message sources elicited longer responses using more words per sentence and simpler language writing more about things rather than themselves or addressing others directly. Contrary to common assumptions, expressed emotions or indicators of cognitive processing did not differ between message source conditions, but vaccinated sources led to more achievement-related expressions. Participant vaccination did not moderate the observed effects but had differential main effects on psycho-linguistic response parameters. We conclude that public vaccination campaigns need to take the vaccination status of the message source and other societal rifts into account to bolster recipients’ achievement.


Introduction
Evidence indicates that COVID-19 vaccines safely and effectively protect individuals from severe illness [1]. Nevertheless, many people in Western countries forego opportunities to receive the vaccine [2][3][4], partially because immunization program communications inadequately address such vaccine hesitancy [5]. Recent research indicates that group-level processes can help understand and potentially mend vaccine hesitancy. According to this view, vaccination status serves as a group membership and the resulting group boundaries cause rifts when communicating calls to get vaccinated between those vaccinated and those unvaccinated (i.e., a vaccination rift effect [6]). The vaccination rift was demonstrated in self-reports and behavior, but this past research is mute to the cognitions, emotions, and motives underlying this effect. We, therefore, use psycho-linguistic analyses to probe the processes underlying the vaccination rift.
Researchers' warned early on during the COVID-19 pandemic that vaccine uptake may be insufficient unless informational campaigns target populations at risk for vaccine hesitancy [7]; these warnings, unfortunately, proved true [2]. The key determinant of decisions to get vaccinated is trust in the responsible health authorities [8][9][10], and confidence in vaccines and the system that delivers them [11]. Common vaccine communication strategies, such as using expert communicators, address the interpersonal, individual, and organizational levels to build trust [2]. However, as divisions in societies increase, trust decreases [12]. The resulting group lines thus represent mounting barriers to vaccination that communication [30], and societal divides [31]. Analyzing the words people use rather than their responses to a Likert-type scale has the advantage that participants can give free responses. Linguistic analyses, therefore, provide the opportunity to explore psychological processes beyond existing theory. We, accordingly, explored psycho-linguistic processes underlying the vaccination rift effect in the content of participants' free-text responses using automated quantitative linguistic analyses [25]. We, moreover, analyzed the influence of recipients' own vaccination status on these processes, as well as a potential effect of matching the vaccination status of the source with the participant. We, thereby, seek to contribute to understanding vaccine hesitancy, why vaccination has become such a divisive topic, and how to communicate to mend the resulting societal divides.
Although no research to date has provided elaborate psycho-linguistic process analyses of vaccination rifts, past research on group criticism gives some indication regarding potential candidate processes. Past research has observed that participants invest more of their own time [32] and money [20,21,33] to refute outgroup criticism. This would suggest that the vaccination rift should lead to longer and more elaborate written responses (i.e., more words per response as well as longer sentences). A related question concerns who responds to vaccination rifts addresses. Responses to group criticism have been categorized as directed at the commenter, bolstering one's own group, or derogating other groups [32]. Evidence in these studies was strongest for personal response towards the commenter. In the context of the vaccination rift, it is also possible that responses address the common public, as vaccination is an issue of societal importance. The psycho-linguistic approach allows quantifying these references by analyzing the frequency of pronouns used. In line with this view, pronoun use can signify one's identity and has been associated with intergroup and intragroup processes [25,[34][35][36].
Beyond the focus and quantity and syntax of communication, specific words can give an indication of different psychological processes. In the group criticism literature, one assumption is that responses to group criticism "are not guided by cold, rational examinations of the evidence" but "emotional, 'hot' processes, infused with threat, uncertainty and doubt" (p. 276, [18]). This suggests that vaccination rifts may reduce cognitive processes but increase emotional responding and "lashing out" [37]. Tentative evidence for this assertion comes from research on group criticism and costly punishment that indicated that the main motivation for responding to group criticism is retribution and not changing the commenter's future behavior [21]. Alternatively, vaccination rifts may change the focus from making an informed decision to winning an argument. Past research on responses to social threats are in line with this view [38]. We used detailed psycho-linguistic analyses to test if and how the vaccination rift influences communication patterns and motivations.
We, thereby, aim to make three key contributions: First, the experimental study design allows identification of causal effects of message source vaccination status on psychological responses to calls to get vaccinated. Second, our large, age-representative sample of vaccinated and unvaccinated Austrian participants provides insights into the psychological processes of these societal groups. Importantly, the standardized study setting makes it unlikely that common biases, such as prejudice towards those unvaccinated [39], impact our findings. Finally, participants can respond to the free-text items in whichever way they chose. Our quantitative analysis of their responses, therefore, allows identification of processes underlying vaccine hesitancy that current theory does not cover. Jointly, these findings will identify new directions for theory on vaccine hesitancy and help design effective vaccine messages.

Materials and Methods
We report secondary analyses of the vaccination rift dataset [6]. The dataset was collected between 29 November 2021 and 1 December 2022 through news items in the national newspaper "Salzburger Nachrichten" on 29 November 2021 (online and print) and the public television station "ORF Salzburg" during the evening news "Salzburg heute" and on their website on 26 December 2021. As reported in the original publication, all Vaccines 2023, 11, 503 4 of 14 ethical standards, including obtaining informed consent, as put forth by the Declaration of Helsinki and the American Psychological Association, were followed. The University of Salzburg internal review board approved this study (IRB number GZ 10/2020). All materials, data (except raw free-text responses that were removed due to privacy concerns), and analyses are available at https://osf.io/c8tn3/ (access date: 20 February 2023).
We briefly summarize the methods of the original study; please refer to the vaccination rift publication [6] for further details. Participants read critical comments calling for unvaccinated people to get the COVID vaccine (see Appendix A). This comment was attributed to a former participant who was described as either being unvaccinated but intending to get vaccinated (unvaccinated source condition) or being fully vaccinated (vaccinated source condition). After reading each comment, participants completed manipulation checks on the source vaccination status, and rated message motive, message threat, and commenter characteristics. Participants then indicated if they wanted to receive more information on vaccination sites (yes/no). Key to the present analysis, participants then had the opportunity to respond to the free-text question: "What is your personal opinion about the Corona vaccination?" Participants could write as much or as little as they chose. Finally, participants provided demographic information (nationality, Austrian county of residence, vaccination status, age, and gender) and were debriefed. All participants then received official information on the COVID vaccine, as recommended by the ethics committee.
For the current analysis, we obtained the original dataset and analyzed the responses to the free-text question on participants' opinion about the coronavirus vaccination. In line with recent recommendations [29], a research assistant blind to the studies' hypotheses spell-checked and proof-read all comments. Obvious typographic errors were corrected but no other alterations were made to the texts. Specifically, all slang, swear words, and colloquial expressions were maintained. We then entered the free-text response into Linguistic Word Count and Inquiry (LIWC) 2015 [41], the industry-standard tool to conduct psycho-linguistic analyses. We used the 2015 German dictionary available for LIWC that has been validated and has measurement properties equivalent to the English version [42,43]. We used R [44] in R-Studio [45] including the packages psych [46], readxl, stringr, and dplyr to analyze our data.

Results
All linguistic measures were highly skewed, and we accordingly used a non-parametric Mann-Whitney-Wilcoxon tests for independent samples, applying continuity corrections. We, moreover, sought to find a balance between statistical power to observe (typically small) effects on linguistic markers and to control for alpha-error inflation due to multiple tests. We, therefore, decided to apply group-wise alpha corrections by dividing the threshold of p = 0.05 by the number of tests per category, yielding an effective alpha-level between 0.05 and 0.001. Moreover, we omit significant exploratory effects that were extremely small (i.e., medians of 0 across conditions). We indicate in our results section where tests were not significant due to the alpha-adjustment and report the full results in Supplementary S1. Non-parametric Spearman correlations between LIWC summary measures, source vaccination status and participant vaccination status are depicted in Figure 1; means and medians in Table 1.

Results
All linguistic measures were highly skewed, and we accordingly used a non-parametric Mann-Whitney-Wilcoxon tests for independent samples, applying continuity corrections. We, moreover, sought to find a balance between statistical power to observe (typically small) effects on linguistic markers and to control for alpha-error inflation due to multiple tests. We, therefore, decided to apply group-wise alpha corrections by dividing the threshold of p = 0.05 by the number of tests per category, yielding an effective alphalevel between 0.05 and 0.001. Moreover, we omit significant exploratory effects that were extremely small (i.e., medians of 0 across conditions). We indicate in our results section where tests were not significant due to the alpha-adjustment and report the full results in Supplementary S1. Non-parametric Spearman correlations between LIWC summary measures, source vaccination status and participant vaccination status are depicted in Figure 1; means and medians in Table 1.

Vaccination Rift
The starting point for our linguistic analyses was the observation that vaccinated sources' critical calls to get vaccinated elicited longer responses than identical comments from an unvaccinated source [6].
We first sought to explore the potential rift in the complexity of participants' responses. Compared to those who received a comment from the unvaccinated source, participants in the vaccinated source condition communicated more elaborately, that is, used more words per sentence (median Effects of 3. Person plural pronouns (addressing others) were significant but small; effects for 1. Person pronouns addressing the writer themselves ("I") or their group ("we") were non-significant, as were 2. Person pronouns addressing others directly ("you"). Apparently, the vaccination rift leads to communicating about things rather than with each other. Interestingly, no effects of "we" and "you" emerged. These pronouns are typical indicators of processes within and between groups such as cooperation and conflict [34,35]. The absence of rift effects on these linguistic markers indicates that a strong identification with the group of the unvaccinated may not be a prerequisite for vaccine hesitancy. This conclusion would be in line with recent research indicating that group identification does not influence the rejection of outgroup criticism [20,21].
We next explored the emotional, cognitive, and motivational processes underlying the vaccination rift effect in more detail. Social conflict is commonly assumed to hinge on emotional processes with reduced cognitive processing. Contrary to this popular assumption, we did not observe These findings indicate that vaccination rifts may rely less on increased emotional and reduced cognitive processes that previously assumed. It is, thus, not the case that the rejection of calls to get vaccinated stems from "all feeling and no thinking".
Taking an alternative perspective, outgroup criticism may violate participants' basic needs. Indeed, a vaccination rift effect on motivational indicators emerged, such that comments from the unvaccinated source led to using fewer drive-related words (Mdn unvaccinated source = 7.69) as compared to participants who received a comment from the vaccinated source (Mdn vaccinated source = 9.01), W = 159,022, mu = −0.27, 95%CI [−1.41; <−0.01], p = 0.035. Follow-up analyses indicated that this effect emerged for achievement-related words (Mdn unvaccinated source = 0, Mdn vaccinated source = 1.73), W = 156,967, mu < −0.01, 95%CI [<−0.01; <−0.01], p = 0.009. No effects emerged on words related to affiliation, power, risk, or reward, all Ws < 163,328, all ps > 0.150. Apparently, the vaccination rift effect hinges on perceptions of competence and achievement more than affiliation or shifting riskperceptions. Calls to get vaccinated from vaccinated sources thus cause rifts because they lead to debates about competence, for instance whether the vaccine and those promoting it are efficacious. Once these questions about the source arise, it is difficult to process new information in an unbiased manner.
To summarize, a vaccination rift effect was evident in the linguistic markers, such that responses to vaccinated (vs. unvaccinated) sources:

•
Were more elaborate (longer and more words/sentence), but simpler (fewer words with six letters or more and more words in the dictionary) • Addressed things more, but there was no indication for group processes • Were not more emotional or less thought-through but included more drive-related words, especially those related to achievement, and more words related to general social processes.

Participant Vaccination Status
Our next goal was to test whether responses varied based on participants' own vaccination status, collapsed across message source conditions. To this end, we clustered participants into two vaccination status groups: vaccinated (1, 2, or 3 doses, n = 634) and unvaccinated (recovered or unvaccinated, n = 536). For exploratory purposes, we also compared the key subgroups of fully vaccinated (3 doses, n = 466) and unvaccinated participants (n = 370). Results between the full sample and this subsample did not differ, unless noted otherwise.
We first sought to explore the length and complexity of participants' responses. Compared to those who indicated that they were vaccinated, unvaccinated participants wrote more (Mdn unvaccinated participant = 31.50 vs. Mdn Person plural pronouns (addressing others) were significant but small; effects for 1. Person pronouns addressing the writer themselves ("I") or their group ("we") were non-significant, as were 2. Person pronouns addressing others directly ("you"). Just as the observed message source effects, unvaccinated participants communicated about things rather than themselves or with each other. Interestingly, again, no group process effects of "we" and "you" emerged, indicating that the divide between those vaccinated and those unvaccinated may not hinge on strong group identification. Exploratory follow-up analyses indicated small effects (i.e., all Mdn = 0) on achievement and power but no effects on affiliation or reward. Overall, threatened needs seem less important for effects of vaccination status than for the vaccination rift effect. Indicators of cognitive processes again showed no effect.
Exploratory analyses indicated that unvaccinated participants referred more to biological processes, perception processes, and informal language, but these effects were small (i.e., all Mdn = 0). Unvaccinated participants used more words establishing the relation be- .02], p < 0.001, especially male words, although this effect was very small (i.e., all Mdn = 0). No effects emerged for female, family, and friends words, all Ws < 173,528, all ps > 0.180. Apparently, vaccination status has an impact on the social world of those who are unvaccinated. This interpretation is in line with recent research on prejudice towards those unvaccinated [39]. We return to this point in the discussion.
To summarize, in comparison to vaccinated participants, unvaccinated participants responses to calls to get vaccinated:

•
Were more elaborate (longer and more words/sentence), but simpler (fewer words with six letters or more) • Were less analytic, less confident, and less positive in tone, but more authentic • Addressed things more, but showed no greater indicators of group processes • Showed less positive and more negative emotions • Referred more to spatial as well as social relations.

Matching of Participant and Source Vaccination Status
A final possibility is that the congruence between participant and source vaccination status influences the psycho-linguistic processes in responses to calls to get vaccinated. We explored this possibility by coding whether participant and source status matched (unvaccinated source and unvaccinated participant, vaccinated source and vaccinated participant, n = 564) or not (unvaccinated source and vaccinated participant, vaccinated source and unvaccinated participant, n = 606). Note that this comparison is akin to an omnibus interaction test in parametric analyses (e.g., ANOVA).
We observed no consistent effects of the matching variable on any of the dependent measures, Ws < 181,032, ps > 0.075, with the following two exceptions: Participants whose vaccination status matched the source vaccination status used more words that were in the LIWC dictionary (Mdn match = 85. 50 .01], p = 0.028, and the observed differences on dictionary words and anger were no longer significant, Ws < 90,787, ps > 0.050.
To summarize: • The matching of source and participant vaccination status had no consistent effects on the observed psycho-linguistic processes. • Apparently, the vaccination rift elicits psycho-linguistic processes that are independent of those elicited by recipients' own vaccination status.

Discussion
We provide in-depth linguistic analyses of the vaccination rift experiment, relying on 49,259 words in voluntary free-text responses from an age-representative sample of the Austrian population (N = 1170). Calls to get vaccinated from vaccinated sources caused rifts in psycho-linguistic processes, in comparison to the same calls from so-far unvaccinated sources. Experimental manipulation of the message source provides the gold-standard for causal inferences.
Participants' vaccination status also had consistent effects that were different from those of message source. While message source effects focused on drives, especially achievement and social processes, participant vaccination status influenced a range of psycho-linguistic processes but not drive-related words. This pattern of results suggests that the message source, rather than participants' vaccination status, elicits discussions about competence and efficacy of vaccines. Although the findings regarding participants vaccination status were correlational, robustness-checks between the full and a subsample increase our confidence in the observed effects.
Finally, we did not observe consistent effects of the matching between source and participant vaccination status. This may be surprising since the vaccination rift effect on self-report scales has been reported to be substantially larger among unvaccinated participants [6]. Psycho-linguistic analyses are arguably more unobtrusive than presenting self-report scales. Accordingly, one may argue that our current analyses tap into more subtle processes that are more ubiquitous. In line with this view, rejection of criticism has even been observed among bystanders who are not members of the criticized groups [20][21][22]47,48]. This indicates that choosing message sources for calls to get vaccinated carefully may even benefit those already vaccinated.
Interestingly, there was little evidence that source and participant vaccination status influenced discussions about group membership (i.e., "we" and "they") or personal identity ("I"). Apparently, strong group memberships are no pre-requisite for rifts to emerge. One may consider this good news, since vaccination may not instill strong group boundaries; this also indicates, however, that vaccination rifts do not only emerge among highly identified fanatics but represent a common social process (cf. [49,50]). The vaccination rift effect could thus represent a much greater impediment to vaccination than previously assumed.
There are some limitations to our study that warrant discussion. First, it is important to note that the observed effect sizes were small. Effects of this size, however, are in line with past research using psycho-linguistic analyses [51]. Considering the volatility in free responses, it is remarkable how consistent the observed patterns were. Second, our dataset just contained participants from Austria. This setting was well-suited for the investigation (e.g., low structural barriers to vaccination, substantial civic liberties). The dataset is nevertheless mute to whether the observed processes occur in other cultures. Given the large variation of vaccine acceptance between different countries [52,53], studies in other cultural contexts are clearly needed. Third, we used an experimental design. While experiments allow drawing gold-standard conclusions about the causal direction of effects, they are mute to within-person changes. Similarly, the media channels advertised our study broadly, but it is possible that extreme vaccine critics may not have participated. Such a selection bias may have diminished the observed effects. Future longitudinal research with extreme vaccine critics is, therefore, needed to understand the temporal dynamics of responses to calls to get vaccinated. Finally, our study was conducted during a time when vaccination was at the center of public debate. For instance, a vaccine mandate had been issued in Austria. This attention may have increased vaccine messaging effects among those who are already vaccinated. More broadly, we explicitly referred to COVID-19 vaccination during an ongoing global pandemic. Future research should thus investigate if our results generalize to settings where COVID-19 vaccination is not at the center of public debate and to other vaccination decisions.
More broadly, our analysis provides key insights into the social determinants of vaccine hesitancy. Beyond low education and awareness or lacking opportunities [52], vaccination decisions are closely linked to one's social system. In line with this view, cultural norms and peer vaccination decisions have been observed to influence vaccine hesitancy [54]. This observation is mirrored in our findings regarding social process. It is, therefore, key to take the social world into account when designing vaccine communication programs.
Trust and other general beliefs and attitudes towards vaccination are key for vaccine acceptance [55,56], but the vaccination rift diminishes trust [6]. The current analysis sheds light on the underlying processes: Participants' counterarguing focused on achievement, highlighting the role of competence-related issues and questions. Unvaccinated participant moreover showed that the vaccine is a highly emotional topic for them that is approached with less confidence and less analytically. Considering these response patterns, it is hardly surprising that vaccine uptake is low in this population.
In extension, we speculate, that effects of participant vaccination status are a consequence of repeated rift experiences. For instance, recent research has reported extreme discriminatory attitudes against the unvaccinated [39]. Such discrimination may impede, rather than promote, vaccine uptake. Others have observed that expressed negativity can deter others from getting vaccinated [57]. We observed that those unvaccinated expressed substantial negative affect when writing about the COVID-19 vaccine, which may be indicative of these processes in free communication.
Our findings have direct implications for public health policy. Taking the vaccination status of the message source into consideration is easy to implement at scale. Ongoing campaigns could readily be adjusted, and the rift approach can be combined with existing effective approaches. For instance, using expert communicators has been shown to increase message effectiveness [53]. Unfortunately, it is all too common for expert communicators to highlight their early adoption of the vaccine and their status as being vaccinated. Our research indicates that it would be beneficial to also communicate doubts of recentlyswayed sceptics. Our research thus indicates that relatively minor changes in vaccine communication may have a substantial impact on people's trust and their responses. This finding highlights the need to convince rather than coerce those who are skeptical of public health interventions and to respect societal group boundaries.

Conclusions
Getting vaccinated saves countless lives but a substantial minority chooses to forego this opportunity. The current analysis indicates that the rifts along the lines of vaccination status emerged even in the words people use to write about the vaccine. This indicates that vaccine hesitancy is a substantial societal problem. However, our analysis also contributes to understanding these basic processes. We hope that applying this knowledge will promote constructive debates and mend the rifts that COVID-19 has torn. Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.