Next Article in Journal
SARS-CoV-2: An Analysis of the Vaccine Candidates Tested in Combatting and Eliminating the COVID-19 Virus
Next Article in Special Issue
A Retrospective Evaluation of Self-Reported Adverse Events Following Immunization with Different COVID-19 Vaccines in Türkiye
Previous Article in Journal
Morphology of Lymphoid Tissue in the Lungs of Guinea Pigs Infected with Mycobacterium bovis against the Background of Vaccine Immunity and the Action of Betulin and Its Derivatives
Previous Article in Special Issue
Suit the Remedy to the Case—The Effectiveness of COVID-19 Nonpharmaceutical Prevention and Control Policies Based on Individual Going-Out Behavior
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Balanced Reporting and Boomerang Effect: An Analysis of Croatian Online News Sites Vaccination Coverage and User Comments during the COVID-19 Pandemic

Faculty of Humanities and Social Sciences, Josip Juraj Strossmayer University of Osijek, 31000 Osijek, Croatia
Institute for Social Research in Zagreb, 10000 Zagreb, Croatia
Author to whom correspondence should be addressed.
Vaccines 2022, 10(12), 2085;
Submission received: 31 October 2022 / Revised: 30 November 2022 / Accepted: 3 December 2022 / Published: 6 December 2022
(This article belongs to the Special Issue COVID-19 Vaccination, Role of Vaccines and Global Health)


The purpose of this paper was to explore online media coverage of COVID-19 vaccination and user reactions to the different types of coverage. The authors aimed to investigate possible boomerang effects that arise when COVID-19 media coverage is assertive and confident, and to determine the effects of balanced reporting. A two-stage random sample comprised a total of 300 articles published in three Croatian online news sites during a period from 1 February 2020, through 15 January 2022. The data were categorized using human coding content analysis, while reliability of coding was checked by using two coders and calculating reliability coefficients. The data were analyzed by means of negative binomial regression analysis. The results revealed that COVID-19 reporting was mainly consensual, i.e., it provided largely affirmative information about vaccines. However, user comments were highly polarized and mostly negative, with the majority of anti-vaccination tropes linked to the “corrupt elites”. Based on the user comments, the negative influence of balanced reporting on COVID-19 vaccines and the existence of boomerang effect in cases of the overtly persuasive affirmative reporting was also established. The boomerang effect did not depend on the context, i.e., on the type of reporting. This study extends previous research on balanced reporting and boomerang effects by analyzing online comments as a potentially good parallelism of the offline discursive strategies of the pro-vaccination and anti-vaccination communication. The results of the study can be used for the adjustment of strategic communication targeting the vaccine hesitant audience. Based on the study results, it is recommended that relativization and politicization of science should be prevented by not equating scientific consensus with absolute epistemological certainty and by addressing legitimate concerns of vaccine hesitant persons without putting explicit blame on them.

1. Introduction

It has been estimated that only less than 0.013% of the total published scientific articles enjoy media presence, which means, that even in the Internet era, journalists still act as strong gatekeepers who direct the flow of scientific knowledge to the public [1]. The media does not cover all topics equally-media select, filter and interpret newsworthy topics because they do not have enough time and space [2]. Scientific topics increasingly become the subject of media interest as they are a part of broader social and political problems [3]. For instance, it has been shown that traditional and influential media outlets can have a direct persuasive effect on the science attitudes [4], while it can be assumed that the changing media environment and new forms of online science journalism might exert positive influence by providing the interested public with relevant sources of rapid science information [5]. However, issues of trust in experts and expert systems are concerns of great importance for contemporary societies. Giddens asserted that we rely increasingly on experts to manage risk, but also argued that indefiniteness of expert knowledge sometimes makes it difficult for people to have trust in experts [6]. The risk society is paradoxically characterized by increasing dependence on experts along with declining trust in these same experts [7], since new knowledge brings up questions among experts that most often cannot be resolved simply by gathering additional information; instead, the dilemmas deepen, widen and multiply [8].
Even before the current pandemic, the ideas about possible influences of the media on vaccine hesitancy had been brought forward [9,10]. In their systematic analysis of the studies which employed content analysis of traditional media vaccination coverage, Catalan-Matamoros and Peñafiel-Saiz found that most studies determined that the media were more often focused on negative aspects of vaccines [11]. For Kata, the Internet is a “postmodern Pandora’s box”, given that it provides a lot of material that enables a person to question scientific authority and legitimacy [12]. When it comes to social media, it has been established that negative tweets about vaccination receive more retweets than positive tweets [13,14], and that negative Instagram posts about vaccination receive more likes [15]. Blankenship et al. suggest that the providers of negative information are more often very committed to their cause, i.e., that the issue of vaccination is more salient to them than to those who accept vaccines [14]. This is also confirmed in other research which showed that users belonging to anti-vaccination communities consume more social media sources and are more active [16,17]. In an experimental study, Betsch et al. found that viewing vaccine-critical websites increased the perception of risk of receiving vaccines, while it decreased the perception of risk of omitting vaccination and the intentions to vaccinate [18]. It can be assumed that the negative role of the misinformation on the Internet became even more pronounced during the COVID-19 pandemic given that the situation was new and unfamiliar [19].
When it comes to the types of arguments provided by vaccination skeptics, a codebook developed by Hughes et al. [20] identified main anti-vaccination narrative tropes and rhetorical strategies to be used in subsequent research on COVID-19 vaccination. They defined narrative tropes as themes which are part of the wider worldview narratives that are implied in the manifest tropes, while rhetorical strategies are more specific devices that can be utilized in various narrative tropes in order to shape appealing and persuasive messages. They identified the following four most common narrative tropes on the US anti-vaccination social media accounts: Corrupt Elite (powerful elites disempower ordinary citizens), Vaccine Injury (health damages), Sinister Motives (shadowy motives) and Heroes and Freedom Fighters (civil rights issues). Additionally, the following four most often used rhetorical strategies are identified: (1) the Brave Truthteller, (2) Do Your Own Research, (3) Mountains and Molehills and (4) Global Movement/Sleeping Giants.
When it comes to the user reactions, the studies of science communication are focused on the ways in which various types of communication can elicit both positive and negative attitudes and behavior. In this study, we also specifically focus on two issues: balanced reporting and boomerang effects that stem from overly confident science communication. Journalistic norm of balanced reporting is often cited as an important cause of negative views about scientific consensus [21]. By conducting a media content analysis and a survey study related to the MMR debate, Lewis and Spears confirmed that the journalistic norm of balanced reporting created an impression that there are two bodies of conflicting evidence that are equally important [22]. The media coverage which emphasizes scientific dissensus over COVID-19 vaccination issues can occur in two main forms. The first one is related to the negative comments about vaccination and vaccines made by the “whistleblowers” scientists. The other way is related to the misunderstandings over the very nature of scientific consensus, which is never finished and continually evolves. For instance, Capurro et al. found that some journalists in Canada covered COVID-19 crisis by emphasizing “duelling experts” and sending mixed messages about official public health measures [23]. Part of this problem might be derived from the process within the science communication, wherein knowledge becomes solidified and translated into a “fact” [24]. The perception of the level of dissensus among scientists may be further raised by employing journalistic norms of balanced coverage, even in the cases where the weight of evidence is heavily skewed towards one side [25]. Balanced coverage helps journalists to emphasize controversy and add drama to their reporting and to provide them with a guise of objectivity [26]. Studies showed that balanced reporting is quite widespread. For instance, Boykoff and Boykoff found that the majority of US press accounts on global warming gave roughly equal attention to the opposing views [27]. Boykoff’s content analysis of the main US television channels showed that about 70% of all news stories included balanced coverage of anthropogenic climate change [28]. By analyzing the U.S. media reporting, Merkley found that fully balanced reporting is not a universal journalistic norm, but that is still present in 20% of all accounts [29]. The effect of the balanced reporting was confirmed in several experimental studies. For example, Koehler established that balanced conflicting comments from an expert on either side of an issue influenced that ability of people to distinguish between topics with high and low expert consensus, even in cases where weight of evidence comment was provided [25]. Bolsen and Druckman found that an experimentally induced scientific consensus statement increased belief in human-induced climate change [30]. Similarly, van der Linden et al. found that experimentally manipulated public perception of the scientific consensus translated into a significant increase in climate change concern [31]. Aklin and Urpelainen confirmed that even the smallest amount of scientific dissent undermined public support for new environmental regulations [32].
On the other hand, overwhelmingly positive strategic communication can cause the so-called boomerang effect. Byrne and Hart suggest that a boomerang effect can occur due to either intended or unintended construct activation [33]. The first path occurs when the receiver receives the message in the intended way, but nevertheless does not accept it. This often happens due to psychological reactance, which arises from the perceived threat to personal freedom and the willingness to restore freedom of choice [34]. On the other hand, unintended construct activation happens when the message is either too complex or not too salient for the receiver. Several research studies in the field of health communication demonstrated the existence of the boomerang effect. Namely, in some cases health communication aimed at decreasing unhealthy behavior backfired by producing boomerang effects [35,36,37]. As for the COVID-19 communication, Shoenberger, Kim and Sun found that threat messaging related to brand communication increased reactance, and consequently led to lower intention to engage in socially responsible COVID-19 related behaviors [38]. It can be expected that the boomerang effect can be very pronounced among those with vaccine conspiracy beliefs, given that the process of motivated reasoning should be very strong. Vivid reporting and the reporting which presents concrete and extreme cases and personal cases, are also more likely to be remembered and activated [39,40,41], probably by virtue of the availability heuristic [42]. Scientists have noticed the idea and use this type of communication. For instance, Holton et al. found that during the peak of MMR controversy scientists more often used opinions/points of view/personal experiences than “pure facts” [43]. However, on a more negative note, the vividness can amplify negative attitudes in cases of conflict and low trust. For instance, Van Prooijen demonstrated that existential threats stimulate conspiratorial thinking only in cases where antagonistic groups are present and salient [44]. In the case of COVID-19 communication, we can hypothesize that the presence of the concrete scientists or other experts should serve as a vivid identification of the antagonistic group, thus increasing negative attitudes.
Based on the above considerations, the goal of this study is fourfold. The first goal is to investigate the presence of various vaccination and anti-vaccination tropes and rhetorical strategies during the pandemics. This goal is largely descriptive and does not entail specific hypotheses. Second, our aim was to check whether “balanced” reporting will elicit negative reactions among the readers. Third, we aimed to investigate possible boomerang effects that arise when COVID-19 coverage is too assertive and confident. Fourth, we wanted to explore possible interaction between balanced reporting and predictors of boomerang effect. i.e., whether predictors of boomerang effect will act more strongly in cases of balanced reporting. We set out to explore these issues by analyzing user comments posted within online news sites, since they represent a good way of estimating real-life discursive strategies of pro-vaccination and anti-vaccination communication appeals [45]. As opposed to user responses on social media, the topic of online news sites’ user comments and their dependence on the nuances of science communication is largely unexplored. Having all this in mind, we put forward the following research question and hypotheses:
Research question RQ1: Which types of anti-vaccination and pro-vaccination tropes and rhetorical strategies are present within online news sites’ user comments?
General Hypothesis H1.
Implied scientific dissent about COVID-19 vaccination will elicit more negative user comments.
General Hypothesis H2.
There will be a boomerang effect when reporting on COVID-19 vaccines.
Specific Hypothesis H2.1.
Explicit call for vaccination will elicit more negative user comments.
Specific Hypothesis H2.2.
Blaming of vaccine hesitant people will elicit more negative user comments.
Specific Hypothesis H2.3.
Explicit naming of the expert will elicit more negative user comments.
General Hypothesis H3.
Predictors of boomerang effect will be stronger in cases of balanced reporting.
Specific Hypothesis H3.1.
Explicit call for vaccination will interact with balanced reporting in eliciting more negative user comments.
Specific Hypothesis H3.2.
Blaming of vaccine hesitant people will interact with balanced reporting in eliciting more negative user comments.
Specific Hypothesis H3.3.
Explicit naming of the expert will interact with balanced reporting in eliciting more negative user comments.

2. Materials and Methods

The overall research design comprised the steps delineated in Figure 1. Namely, after the initial coding book development and sampling, preliminary reliabilities were calculated, which led to the development of the final version of the coding book. After that, data collection and data analysis were conducted. All the steps are described below in more detail.
As a means of operationalization of the above-mentioned research question and hypotheses, we developed a coding book to be used in the subsequent human coding content analysis. One part of the coding book covered vaccination tropes and rhetorical strategies as described in Hughes et al. [20]. Additionally, positive, neutral and negative user comments are counted. In the other part, the specific types of issue coverage are measured, namely balanced reporting and coverage that could elicit a boomerang effect. Balanced reporting was operationalized as the implied scientific dissent on the topic of vaccination and was recorded on a 1 to 5 scale, with higher number indicating more divergence on opinions, on the basis of whether reporting on the topic was exclusively affirmative, i.e., if it presented only one side of the argument; or dissenting opinions were brought forth, most often on the basis of opposing research or the opinions of experts whose opinions were contrasted. It has been suggested that science journalists should supplement balanced reporting with the weight of evidence acknowledgement which should point out that the bulk of evidence and support of the majority of experts lie on one side of the debate [46]. However, in the current study the weight of evidence argument was completely absent, since journalists largely avoided expressing their opinion in the cases where they presented conflicted evidence and dissenting views among the scientific community. Therefore, it was not necessary to differentiate among different versions of balanced reporting, i.e., to code the weight of evidence.
As for the examination of boomerang effects, explicit calls for vaccination were present in cases where experts directly addressed the general audience and encouraged them to get the vaccine. Blaming of hesitant persons was measured as an implicit or explicit statement that the pandemic cannot be stopped unless the vaccination rate is increased, or a contention that vaccination has no alternative in the current public health situation. The measurement of the presence of the named expert was quite straightforward.
The study employed a two-stage sample design. In the first stage, we used a judgment sample by selecting three online news sites with publicly available user, 24 and are among the top five most visited online newspapers in Croatia judged by the average daily real users, either according to the Gemius rating of Croatian web domains [47] or according to the Alexa rating [48]. Both 24 and represent online versions of the national daily journals, while is an online-only media outlet. All three newspapers publish in Croatian. In the second stage, we chose 100 articles published by each portal by using random sampling, thus obtaining the total sample size of 300. The keywords used in finding the suitable articles were (English translation): “COVID-19”, “coronavirus”, “SARS-CoV2”; “scientists”, “scientific research” and various combinations of these keywords. Following the initial sample selection, the screening was conducted in order to include only those reports which entailed sufficient coverage of COVID-19 vaccination issues. The time period in which the analyzed articles were published was from 1 February 2020 to 15 January 2022, while the data were collected during February and March 2022. The study approval was obtained from the Ethics Committee of Faculty of Humanities and Social Sciences, Josip Juraj Strossmayer University of Osijek (approval number: 2158-83-02-19-2). The data are available as a supplementary file to this paper (Supplementary Materials, Supplementary File).
After two independent coders finished the analysis on a random sample of 30 cases, preliminary reliability coefficients were calculated. Given that the reliabilities below the lowest threshold value of 0.667 for the anti-vaccination tropes, major antagonists and rhetorical strategies were obtained, some of the categories were merged [49]. Namely, several categories of anti-vaccination tropes are merged into (a) corrupt elites and (b) vaccine injury. Similarly, major antagonists were merged into (a) mainstream society at large/politicians; shadowy villains and (b) the vaccine itself. Anti-vaccination rhetorical strategies were merged into (a) The Brave Truthteller; Global Movements/Sleeping Giants and (b) Do Your Own Research; Mountains and Molehills. Finally, pro-vaccination rhetorical strategies were merged into (a) Rationalism against irrationalism and (b) Presenting evidence for vaccination. After the mergers, final reliability calculations on a total sample of 80 cases were obtained. From Table 1 it can be seen that even after the merger of the categories, reliability of anti-vaccination tropes and major antagonists still remained slightly below the above-mentioned threshold.

3. Results

In Figure 2, the averages of comment types (affirmative, negative and neutral) cross-tabulated across the online news sites are shown. We can observe that the average number of comments per article across all the newspapers is about 66, and that the majority of comments are negative (40.17 per article).
The main tropes, antagonists and rhetorical strategies are shown in Table 2. We can behold that the majority of tropes are connected to the notion of corrupt elites and, consequently, that the major antagonist is not the vaccine itself, but politicians, mainstream society at large and the undefined shadowy villains. Within the tropes, the major anti-vaccination rhetorical strategy is linked to the “do your own research” approach which implies that the pandemics is an artificially and sometimes intentionally produced “mountain out of the molehill”. Regarding the pro-vaccination tropes, in the large majority of cases the proponents of COVID-19 vaccination simply pointed to the fact that vaccines work and prevent the disease. However, when doing that, in most cases the proponents implied irrationality of the vaccine hesitancy and refusal, thus indicating the high degree of polarization that could be noted in the comments.
Descriptive statistics for the variables related to the issue coverage characteristics are shown in Table 3. We can observe that in 71.24% of all cases the names of the experts are explicitly mentioned, while in the remaining cases only general findings and arguments are discussed, sometimes only with a mention of institutions where research touched upon had taken place. The take on vaccination in the analyzed articles is mainly consensual, given that only 7% of cases presented vaccination as mainly or completely controversial. Maybe surprisingly, the implicit or explicit blaming of vaccine hesitant people was present in 74.33% of all cases. However, in most cases the blame was quite implicit, most often in the form of the assertion that the pandemics cannot be stopped if people keep refusing to get vaccinated. Explicit call for vaccination was present in 54% of all cases.
In order to test the proposed hypotheses, we used several negative binomial regressions as an analytical strategy. The rationale for using negative binomial regression instead of ordinary least squares regression or Poisson regression was the fact we had count outcome variable and that there was a significant amount of over-dispersion (variance of the outcome variable was higher than the mean of the variable). In the first model (Table 4), all coverage types were entered as predictors, with newspapers as control variables. The model likelihood ratio is statistically significant, thus rejecting the null hypothesis that all regression coefficients in the model are simultaneously equal to zero, and the fit of the model is also confirmed by the Chi-Square/df ratio which is substantially higher than 0.05. We can also add that almost all residuals were under the absolute value of 2, whereas only a few cases were outliers with values slightly above absolute value of 2.0. From the results we can note that three out of four predictors were statistically significant. As for the balanced reporting, for every one-point increase in dissent, the rate of negative user comments increased by a factor of 1.69. As for the boomerang effect predictors, the presence of an explicit call for vaccination increases the number of negative comments by 1.76 times, explicit naming of the expert by 1.46 times, while the blaming of vaccine hesitant persons was not statistically significant when the other predictor variables are held constant.
In the second model (Table 5), we added interaction terms in order to test whether boomerang effects vary in different contexts, i.e., in relation to the presence of balanced reporting (scientific dissent). The results revealed that none of the three interaction effects was statistically significant, thus implying that there are no moderating effects. The model fit is also somewhat poorer in comparison to the first model.

4. Discussion

Overall, the results of our study revealed that online user comments are quite polarized and mostly skeptical about vaccination (RQ1). We also confirmed the balanced reporting effect (H1) and boomerang effect (H2) in COVID-19 reporting. The interactive effect (H3) was not confirmed.
Our findings show that negative sentiments are frequent in analyzed online user comments. These results are in line with the findings of previous studies in Croatia, which indicated that news related to the topics of vaccination, epidemic measures and COVID-19 daily reports are associated with fear [50]. Although it did not deal specifically with vaccination, an earlier Croatian study conducted in 2021 by Babić et al. [51] also showed that negative sentiment is dominant in tweets posted during the COVID-19 pandemic and findings of many other studies. A research study by Sufi et al. [52] found that anti-vax posts/tweets containing the word “hoax” had the highest level of social influence with the highest level of negativity. Another study found that anti-vaccination supporters are more engaged in discussions on Twitter [53]. We can conclude that our results are in line with similar studies [51,53,54] which have already found that negative attitudes and sentiments are dominant in social media posts during the COVID-19 pandemic, especially related to the topics of vaccination. Online user comments have the potential to strengthen pre-existing negative sentiments/attitudes [55] and public suspicions toward vaccines and be one of the important strongholds of the antivax movement.
Even though we applied the Hughes et al. coding scheme, our analysis showed that anti-vaccination tropes can be broadly divided into two types: vaccine-related and establishment-related [20]. In other words, even though Hughes et al. nuanced tropes and rhetorical strategies are valid, they are often employed together, and thus can be divided in the more general categories mentioned above [20]. Our study results show that in online user comments establishment-related tropes are much more frequent, thus confirming the issue of trust in the expert system that is vividly present throughout COVID-19 pandemics. Trust involves uncertainty and risk which are acknowledged and accepted as the part of the process of trust between competent and autonomous agents. The lack of trust in medical expert systems is not connected to the specific context of COVID-19 pandemic; it represents a long-term process of declining trust and criticism that emerged within the scientific community [56,57], mainly as the consequence of the risk societies development processes mentioned in the introductory part of this paper, wherein the trust in experts’ competence to control the risks is increasingly questioned [7,8]. Such a social environment questions “scientific monopoly” on the truth of the medical profession, which gradually loses its professional authority. However, it is important to note that majority of the scientists who were present in the Croatian media were members of various government/state bodies and that distrust towards the Government can be reflected in distrust towards experts/scientists. They are perceived as persons who are providing scientific legitimacy to government decisions without real scientific autonomy. In addition, study by Beliga et al. [50] have found that over time people become more dissatisfied with politicians and scientists as relevant actors in the COVID-19 pandemic. In this context, presence of negative sentiments in online user comments, as a reflection of distrust toward establishment/public officials, is not surprising.
Our results also revealed that balanced reporting has not been widely present during the COVID-19 pandemic in Croatia. Nevertheless, in cases where it was present, measured by the implied scientific dissent about COVID-19 vaccination, it elicited more negative reactions among the readers. Considering that, we can conclude that the influence of balanced reporting, i.e., our hypothesis H1 was confirmed. Therefore, the results of the current study cohere with the previously mentioned studies which determined that even small cues about scientific dissensus can make science attitudes more negative [31,32]. However, we have to bear in mind that this conclusion is based on the overall number of negative comments. Balanced reporting might have solidified the support for vaccination among less hesitant persons, which could not be measured within our research design.
The study results also partially confirmed the existence of the boomerang effect (hypothesis H2), given that two out of three boomerang effect predictors (presence of the explicit call for vaccination and naming of the expert) were significantly associated with more negative comments. The only predictor, which was not statistically significant, was blaming vaccine hesitant persons. This finding might be explained by the possibility that some of the readers did not pick up often subtle cues connected to the blame attribution. One of the possible explanations might lie in the fact that explicit calls for vaccination and named expert, as the other two measures predicting boomerang effect, are more often present in the article titles, thus being more accessible to the readers. Given the vast amount of information available on the Internet accompanying the current COVID-19 crisis, all the tropes that include scientists are known and readily available to the media users. Therefore, cues coming from assertive reporting are more likely to elicit negative reactions, especially in the context of the declining trust in expert systems explained earlier. It is particularly interesting that the very act of the presence of the concrete, named expert elicited more negative comments. It seems that concrete experts act as a more provoking signal for vaccine hesitant persons, probably enabling them to visualize “the enemy”. Here, we should also highlight that such a finding does not mean that experts do not have an overall positive role in communication about vaccines. This just means that it is possible that the use of experts in changing the minds of stubborn opponents of vaccination through direct dialogue might not be the best way. This is supported by a recent research study in Croatia, wherein the authors concluded that vaccination campaigns should not be focused only on the scientific arguments about safety and effectiveness of vaccine and on the authority of science and scientists because the study has shown that vaccine-hesitant groups do not trust scientists and experts [58]. However, this should be checked in future studies. Finally, we did not confirm our expectation that the boomerang effect will be stronger in cases where balanced reporting was present (hypothesis H3). A plausible explanation might be hidden in the fact that explicit calls for vaccination and placing the blame on vaccine hesitant persons very rarely went hand in hand with balanced reporting, i.e., implying scientific dissensus or mentioning vaccination harms. In other words, reporting that wholeheartedly recommended vaccination usually did not contain any challenges to the consensual view on the issue.
Overall, our study lends some support to the concept of reactance in health communication studies which, as Shoenberger, Kim and Sun noted, has been often referenced, but still not sufficiently tested [38]. The influence of the perception of scientific dissent on attitudes about vaccination can be reversed in two ways. The first is to undo the disagreement, but this is very difficult to achieve and it is questionable how desirable it is to do so. Grasswick suggests that the importance and impact of “scientific whistleblowers” stem from the failed expectations of the lay public, i.e., the expectations from the scientists to communicate and share significant knowledge [59]. Whistleblowers fill in the gap between science and the lay public, making the lay public feeling less marginalized. As Goldenberg notes, by failing to bridge the gap and to engage into meaningful conversation and to address the concerns which are relevant to the lay public, scientists fail to transfer the consensus that exists within the scientific community over the issues that worry the lay public and to earn epistemic merit and overall trust for pursuing public health measures [60]. As Aklin and Urpelainen suggested, probably the more promising venue is to try to “normalize” scientific disagreement, i.e., to prevent the relativization and politicization of science that occurs when even the smallest amount of uncertainty and scientific dissent is equated with the epistemological failure of science and the futility of following its public recommendations [32]. For example, Collins and Pinch [61] also argue that the public image of science as completely certain, neutral and objective is dangerous because it creates public expectations which cannot be fulfilled. If citizens are encouraged in such thinking, they will expect too much from science or be horrified by disagreement among scientists [62].

5. Conclusions

Our study extends previous research on balanced reporting and the boomerang effects by analyzing online new sites user comments as a potentially good approximation of the real-life discursive strategies of the pro-vaccination and anti-vaccination communication, thus being complementary to the mainly experimental research of the aforementioned effects. The results revealed that COVID-19 reporting was mainly consensual and pro-vaccination. However, user comments were highly polarized and mostly negative, with the majority of anti-vaccination tropes linked to the “corrupt elites”. We also established the negative influence of balanced reporting and the existence of a boomerang effect in cases of overtly persuasive affirmative reporting, thus confirming our main hypotheses and the experimental evidence from previous studies. The main general conclusion of our study is that the journalistic norm of balanced reporting does some damage to the credibility of science. However, aggressive communication based on the blaming and overtly persuasive communication can also elicit negative reactions, which hinder the achievement of the public health goals. The results of the study can be used for the adjustment of strategic communication, especially when communicating with vaccine hesitant audience. Based on this study’s results, it is recommended that the relativization and politicization of science should be prevented by not equating scientific consensus with an absolute epistemological certainty and by addressing legitimate concerns of vaccine hesitant persons without putting explicit blame on them.
As for the main limitation of the study, we should reiterate that the number of negative comments is not a good projection of the general attitudes, given that vaccination opponents can be assumed to be more motivated to respond to mostly positive vaccination coverage. Therefore, the conclusions of this study can hardly be generalized to the general population, but rather reflect the regularities that exist in the attempts to influence people with more negative attitudes about vaccination. Another limitation is relatively modest sample size (N = 300), which was dictated by the very strenuous and time-consuming human coding content analysis, which also included pre-testing and reliability analysis.
Based on the overall conclusions and limitations, future studies should aim to replicate the results using different indicators of balanced reporting and boomerang effects in order to increase the robustness of the results. In addition, it would be advisable to replicate the results on larger samples and in other countries.

Supplementary Materials

The following supporting information can be downloaded at:, Supplementary File.

Author Contributions

Conceptualization, Ž.P., A.Š. and J.J.; methodology, Ž.P. and J.J.; validation, Ž.P., A.Š. and J.J.; formal analysis, Ž.P. and J.J.; investigation, Ž.P. and J.J.; data curation, Ž.P. and J.J.; writing—original draft preparation, Ž.P.; A.Š. and J.J.; writing—review and editing, Ž.P., A.Š. and J.J.; visualization, A.Š.; supervision, Ž.P. and A.Š.; project administration, Ž.P.; funding acquisition, Ž.P. All authors have read and agreed to the published version of the manuscript.


This work has been supported by Croatian Science Foundation under the project HRZZ IP-2019-04-7902.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Ethics Committee of the Faculty of Humanities and Social Sciences Josip Juraj Strossmayer University of Osijek approved the study (Ethical approval code: 2158-83-02-19-2).

Informed Consent Statement

Not applicable.

Data Availability Statement

Available on the journal’s website.

Conflicts of Interest

The authors declare no conflict of interest.


  1. Suleski, J.; Ibaraki, M. Scientists are talking, but mostly to each other: A quantitative analysis of research represented in mass media. Public Underst. Sci. 2010, 19, 115–125. [Google Scholar] [CrossRef]
  2. Friedman, S.; Dunwoody, S.; Rogers, C. Scientists and Journalists; American Association for the Advancement of Science: Washington, WA, USA, 1988. [Google Scholar]
  3. Hansen, A. Journalistic practices and science reporting in the British press. Public Underst. Sci. 1994, 3, 111–134. [Google Scholar] [CrossRef]
  4. Feldman, L.; Maibach, E.; Roser-Renouf, C.; Leiserowitz, A. Climate on Cable. Int. J. Press/Politics 2011, 17, 3–31. [Google Scholar] [CrossRef]
  5. Su, L.; Akin, H.; Brossard, D.; Scheufele, D.; Xenos, M. Science News Consumption Patterns and Their Implications for Public Understanding of Science. J. Mass Commun. 2015, 92, 597–616. [Google Scholar] [CrossRef]
  6. Giddens, A. The Consequences of Modernity; Wiley: Hoboken, NJ, USA, 2013. [Google Scholar]
  7. Doyle, A. Introduction: Trust, Citizenship and Exclusion in the Risk Society. In Law Commission of Canada Eds. Risk and Trust: Including or Excluding Citizens; Fernwood: Halifax, UK, 2007; pp. 7–22. [Google Scholar]
  8. Beck, U.; Bonss, W.; Lau, C. The Theory of Reflexive Modernization. Theory Cult. Soc. 2003, 20, 1–33. [Google Scholar] [CrossRef] [Green Version]
  9. Rosselli, R.; Martini, M.; Bragazzi, N.L. The old and the new: Vaccine hesitancy in the era of the Web 2.0. Challenges and opportunities. J. Prev. Med. Hyg. 2016, 57, E47–E50. [Google Scholar]
  10. Chirico, F. Vaccinations and media: An on-going challenge for policy makers. Health Soc. Sci. 2017, 2, 9–18. [Google Scholar]
  11. Catalan-Matamoros, D.; Peñafiel-Saiz, C. How is communication of vaccines in traditional media: A systematic review. Perspect. Public Health 2018, 139, 34–43. [Google Scholar] [CrossRef]
  12. Kata, A. A postmodern Pandora’s box: Anti-vaccination misinformation on the Internet. Vaccine 2010, 28, 1709–1716. [Google Scholar] [CrossRef]
  13. Yu, H.; Yang, C.C.; Yu, P.; Liu, K. Emotion diffusion effect: Negative sentiment COVID-19 tweets of public organizations attract more responses from followers. PLoS ONE 2022, 17, e0264794. [Google Scholar] [CrossRef]
  14. Blankenship, E.B.; Goff, M.E.; Yin, J.; Tse, Z.T.H.; Fu, K.W.; Liang, H.; Saroha, N.; Fung, I.C.H. Sentiment, contents, and retweets: A study of two vaccine-related twitter datasets. Perm. J. 2018, 22, 17–138. [Google Scholar] [CrossRef] [Green Version]
  15. Basch, C.H.; MacLean, S.A. A content analysis of HPV related posts on Instagram. Hum. Vaccin Immunother. 2019, 15, 1476–1478. [Google Scholar] [CrossRef]
  16. Schmidt, A.L.; Zollo, F.; Scala, A.; Betsch, C.; Quattrociocchi, W. Polarization of the vaccination debate on Facebook. Vaccine 2018, 36, 3606–3612. [Google Scholar] [CrossRef] [Green Version]
  17. Bessi, A.; Coletto, M.; Davidescu, G.; Scala, A.; Caldarelli, G.; Quattrociocchi, W. Science vs Conspiracy: Collective Narratives in the Age of Misinformation. PLoS ONE 2015, 10, e0118093. [Google Scholar] [CrossRef] [Green Version]
  18. Betsch, C.; Renkewitz, F.; Betsch, T.; Ulshöfer, C. The Influence of Vaccine-critical Websites on Perceiving Vaccination Risks. J. Health Psychol. 2010, 15, 446–455. [Google Scholar] [CrossRef]
  19. Stewart, R.; Madonsela, A.; Tshabalala, N.; Etale, L.; Theunissen, N. The importance of social media users’ responses in tackling digital COVID-19 misinformation in Africa. Digit. Health 2022, 8, 20552076221085070. [Google Scholar] [CrossRef]
  20. Hughes, B.; Miller-Idriss, C.; Piltch-Loeb, R.; Goldberg, B.; White, K.; Criezis, M.; Savoia, E. Development of a Codebook of Online Anti-Vaccination Rhetoric to Manage COVID-19 Vaccine Misinformation. Int. J. Environ. Res. Public Health 2021, 18, 7556. [Google Scholar] [CrossRef]
  21. Binder, A. Routes to Attention or Shortcuts to Apathy? Exploring Domain-Specific Communication Pathways and Their Implications for Public Perceptions of Controversial Science. Sci. Commun. 2009, 32, 383–411. [Google Scholar] [CrossRef]
  22. Lewis, J.; Speers, T. Misleading media reporting? The MMR story. Nat. Rev. Immunol. 2003, 3, 913–918. [Google Scholar] [CrossRef]
  23. Capurro, G.; Jardine, C.; Tustin, J.; Driedger, M. Communicating scientific uncertainty in a rapidly evolving situation: A framing analysis of Canadian coverage in early days of COVID-19. BMC Public Health 2021, 21, 2181. [Google Scholar] [CrossRef]
  24. Bucchi, M. Of deficits, deviations and dialogues: Theories of public communication of science. In Handbook of Public Communication of Science and Technology; Bucchi, M., Trench, B., Eds.; Routledge: London, UK; New York, NY, USA, 2008; pp. 57–76. [Google Scholar]
  25. Koehler, D. Can journalistic “false balance” distort public perception of consensus in expert opinion? J. Exp. Psychol. Appl. 2016, 22, 24–38. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. Zehr, S. Public representations of scientific uncertainty about global climate change. Public Underst. Sci. 2000, 9, 85–103. [Google Scholar] [CrossRef]
  27. Boykoff, M.; Boykoff, J. Balance as bias: Global warming and the US prestige press. Glob. Environ. Chang. 2004, 14, 125–136. [Google Scholar] [CrossRef]
  28. Boykoff, M. Lost in translation? United States television news coverage of anthropogenic climate change, 1995–2004. Clim. Chang. 2007, 86, 1–11. [Google Scholar] [CrossRef] [Green Version]
  29. Merkley, E. Are Experts (News) Worthy? Balance, Conflict, and Mass Media Coverage of Expert Consensus. Polit. Commun. 2020, 37, 530–549. [Google Scholar] [CrossRef]
  30. Bolsen, T.; Druckman, J. Do partisanship and politicization undermine the impact of a scientific consensus message about climate change? Group Process Intergroup Relat. 2018, 21, 389–402. [Google Scholar] [CrossRef]
  31. van der Linden, S.; Leiserowitz, A.; Feinberg, G.; Maibach, E. The Scientific Consensus on Climate Change as a Gateway Belief: Experimental Evidence. PLoS ONE 2015, 10, e0118489. [Google Scholar] [CrossRef] [Green Version]
  32. Aklin, M.; Urpelainen, J. Perceptions of scientific dissent undermine public support for environmental policy. Environ. Sci. Policy 2014, 38, 173–177. [Google Scholar] [CrossRef]
  33. Byrne, S.; Hart, P. The Boomerang Effect: A Synthesis of Findings and a Preliminary Theoretical Framework. Ann. Int. Commun. 2009, 33, 3–37. [Google Scholar] [CrossRef]
  34. Brehm, S.; Brehm, J. Psychological Reactance; Elsevier Science: Amsterdam, The Netherlands, 1981. [Google Scholar]
  35. Campo, S.; Cameron, K. Differential Effects of Exposure to Social Norms Campaigns: A Cause for Concern. Health Commun. 2006, 19, 209–219. [Google Scholar] [CrossRef]
  36. Wolburg, J. College Students’ Responses to Antismoking Messages: Denial, Defiance, and Other Boomerang Effects. J. Consum. Aff. 2006, 40, 294–323. [Google Scholar] [CrossRef]
  37. Schwartz, M.B.; Thomas, J.J.; Bohan, K.M.; Vartanian, L.R. Intended and unintended effects of an eating disorder educational program: Impact of presenter identity. Int. J. Eat. Disord. 2007, 40, 187–192. [Google Scholar] [CrossRef]
  38. Shoenberger, H.; Kim, E.; Sun, Y. Advertising during COVID-19: Exploring Perceived Brand Message Authenticity and Potential Psychological Reactance. J. Advert. 2021, 50, 253–261. [Google Scholar] [CrossRef]
  39. Iyengar, S. The accessibility bias in politics: Television news and public opinion. Int. J. Public Opin. Res. 1990, 2, 1–15. [Google Scholar] [CrossRef]
  40. Shrum, L.J. Media Consumption and Perceptions of Social Reality: Effects and Underlying Processes. In Media Effects: Advances in Theory and Research; Bryant, J., Oliver, M.B., Eds.; Routledge: New York, NY, USA; London, UK, 2009; pp. 50–73. [Google Scholar]
  41. Zillmann, D. Exemplification theory of media influence. In Media Effects: Advances in Theory and Research; Bryant, J., Zillmann, D., Eds.; Erlbaum: Mahwah, NJ, USA, 2002; pp. 69–95. [Google Scholar]
  42. Tversky, A.; Kahneman, D. Availability: A heuristic for judging frequency and probability. Cogn. Psychol. 1973, 5, 207–232. [Google Scholar] [CrossRef]
  43. Holton, A.; Weberling, B.; Clarke, C.; Smith, M. The Blame Frame: Media Attribution of Culpability About the MMR–Autism Vaccination Scare. Health Commun. 2012, 27, 690–701. [Google Scholar] [CrossRef]
  44. van Prooijen, J. An Existential Threat Model of Conspiracy Theories. Eur. Psychol. 2020, 25, 16–25. [Google Scholar] [CrossRef]
  45. Gallagher, J.; Lawrence, H. Rhetorical Appeals and Tactics in New York Times Comments About Vaccines: Qualitative Analysis. J. Med. Internet Res. 2020, 22, e19504. [Google Scholar] [CrossRef]
  46. Wilson, K. Drought, debate, and uncertainty: Measuring reporters’ knowledge and ignorance about climate change. Public Underst. Sci. 2000, 9, 1. [Google Scholar] [CrossRef]
  47. Gemius Website Rating. Available online: (accessed on 20 December 2021).
  48. Alexa Rating. Available online: (accessed on 18 December 2021).
  49. Krippendorff, K. Content Analysis: An Introduction to Its Methodology; Sage: Thousand Oaks, CA, USA, 2004. [Google Scholar]
  50. Buhin Pandur, M.; Dobša, J.; Beliga, S.; Meštrović, A. Topic modelling and sentiment analysis of COVID-19 related news on Croatian Internet portal. In Proceeding of Conference on Data Mining and Data Warehouses; Grobelnik, M., Mladenić, D., Eds.; Information Society: Ljubljana, Slovenia, 2021; pp. 5–9. [Google Scholar]
  51. Babić, K.; Petrović, M.; Beliga, S.; Martinčić-Ipšić, S.; Matešić, M.; Meštrović, A. Characterisation of COVID-19-Related Tweets in the Croatian Language: Framework Based on the Cro-CoV-cseBERT Model. Appl. Sci. 2021, 11, 10442. [Google Scholar] [CrossRef]
  52. Sufi, F.K.; Razzak, I.; Khalil, I. Tracking Anti-Vax Social Movement Using AI based Social Media. IEEE Trans. Technol. Soc. 2022. [Google Scholar] [CrossRef]
  53. Germani, F.; Biller-Andorno, N. The anti-vaccination infodemic on social media: A behavioral analysis. PLoS ONE 2021, 16, e0247642. [Google Scholar] [CrossRef] [PubMed]
  54. Lwin, M.O.; Lu, J.; Sheldenkar, A.; Schulz, P.J.; Shin, W.; Gupta, R.; Yang, Y. Global Sentiments Surrounding the COVID-19 Pandemic on Twitter: Analysis of Twitter Trends. JMIR Public Health Surveill. 2020, 6, e19447. [Google Scholar] [CrossRef] [PubMed]
  55. Meyer, S.B.; Violette, R.; Aggarwal, R.; Simeoni, M.; MacDougall, H.; Waite, N. Vaccine hesitancy and Web 2.0: Exploring how attitudes and beliefs about influenza vaccination are exchanged in online threaded user comments. Vaccine 2019, 37, 1769–1774. [Google Scholar] [CrossRef] [PubMed]
  56. Wolfensberger, M.; Wrigley, A. Trust in Medicine. Its Nature, Justification, Significance, and Decline; Cambridge University Press: Cambridge, UK, 2019. [Google Scholar]
  57. Hausman, B.L. Anti/Vax: Reframing the Vaccination Controversy; IRL Press: Oxford, UK; Cornell University Press: Ithaca, NY, USA, 2019. [Google Scholar]
  58. Bagić, D.; Šuljok, A.; Ančić, B. Determinants and reasons for coronavirus disease 2019 vaccine hesitancy in Croatia. Croat. Med. J. 2022, 63, 89–97. [Google Scholar] [CrossRef]
  59. Grasswick, H. Scientific and lay communities: Earning epistemic trust through knowledge sharing. Synthese 2010, 177, 387–409. [Google Scholar] [CrossRef]
  60. Goldenberg, M. Public Misunderstanding of Science? Reframing the Problem of Vaccine Hesitancy. Perspect. Sci. 2016, 24, 552–581. [Google Scholar] [CrossRef]
  61. Collins, H.M.; Pinch, T. The Golem: What Everyone Should Know about Science; Cambridge University Press: Cambridge, UK, 1993. [Google Scholar]
  62. Yearly, S. Understanding science from the perspective of the sociology of scientific knowledge: An overview. Public Underst. Sci. 1994, 3, 245–258. [Google Scholar] [CrossRef]
Figure 1. Research design.
Figure 1. Research design.
Vaccines 10 02085 g001
Figure 2. Comment types by online newspapers (average number).
Figure 2. Comment types by online newspapers (average number).
Vaccines 10 02085 g002
Table 1. Reliability measures.
Table 1. Reliability measures.
VariablePercentage AgreementKrippendorff Alpha (Nominal or Interval)
Anti-vaccination tropes87.340.63
Major antagonist91.140.66
Anti-vaccination rhetorical strategy83.540.70
Pro-vaccination tropes91.140.79
Pro-vaccination rhetorical strategy86.080.74
Number of negative comments-0.83
Number of neutral comments-0.88
Number of positive comments-0.79
Scientific dissent-0.81
Explicit call for vaccination88.610.77
Blaming hesitant persons92.410.82
Is the expert named97.480.94
Table 2. Descriptive statistics for tropes, antagonists and rhetorical strategies.
Table 2. Descriptive statistics for tropes, antagonists and rhetorical strategies.
VariableAnswer CategoryPercentage
Anti-vaccination tropesCorrupt elites69.67
Vaccine injury22.00
Major antagonistMainstream society at large/politicians; shadowy villain81.00
Vaccine itself11.00
No clear antagonist8.00
Anti-vaccination rhetorical strategyThe Brave Truthteller; Global Movements/Sleeping Giants16.34
Do Your Own Research; Mountains and Molehills71.33
No clear rhetorical strategy12.33
Pro-vaccination tropesVaccines work/prevent diseases64.67
Medical workers as heroes3.67
Not available31.67
Pro-vaccination rhetorical strategyRationalism against irrationalism51.00
Presenting evidence for vaccination16.67
Not available32.33
Table 3. Descriptive statistics for the characteristics of the issue coverage.
Table 3. Descriptive statistics for the characteristics of the issue coverage.
VariableAnswer CategoryPercentage
Scientific dissentComplete agreement67.67
Mainly agreement15.67
Both agreement and disagreement9.67
Mainly disagreement3.00
Complete disagreement4.00
Explicit call for vaccinationYes54.00
Blaming hesitant personsYes74.33
Is the expert namedYes71.24
Table 4. Negative binomial regression—model 1.
Table 4. Negative binomial regression—model 1.
BS.E.pExp (B)95% Confidence Intervals for Exp (B)
Newspapers = Index0.–1.75
Newspapers = 24 sata1.–3.66
Newspapers = Večernji list-----
Scientific dissent0.520.070.001.691.48–1.92
Blaming = Yes0.–1.65
Explicit call = Yes0.560.140.001.761.34–2.30
Main expert named = Yes0.380.140.011.461.10–1.94
Outcome variable: count of negative user comments. Likelihood ratio χ2(6) = 158,03, p = 0.00; Pearson Chi-Square/df = 0.96; AIC = 2562.35; BIC = 2587.97.
Table 5. Negative binomial regression—model 2.
Table 5. Negative binomial regression—model 2.
BS.E.pExp (B)95% Confidence Intervals for Exp (B)
Portal = Index0.300.150.051.340.99–1.82
Portal = 24 sata1.–3.78
Portal = Večernji list-----
Scientific dissent0.700.–2.86
Blaming = Yes0.630.260.021.881.13–3.12
Explicit call = Yes0.560.280.041.751.02–3.02
Expert named = Yes0.630.310.041.871.02–3.44
Blaming X Scientific dissent−0.360.190.070.700.48–1.03
Explicit call X Scientific dissent0.–1.75
Expert named X Scientific dissent0.030.230.891.030.66–1.61
Outcome variable: count of negative user comments. Likelihood ratio χ2(13) = 163.26, p = 0.00; Deviance/df = 0.97; AIC = 2563.12; BIC = 2599.72.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Pavić, Ž.; Šuljok, A.; Jurlina, J. Balanced Reporting and Boomerang Effect: An Analysis of Croatian Online News Sites Vaccination Coverage and User Comments during the COVID-19 Pandemic. Vaccines 2022, 10, 2085.

AMA Style

Pavić Ž, Šuljok A, Jurlina J. Balanced Reporting and Boomerang Effect: An Analysis of Croatian Online News Sites Vaccination Coverage and User Comments during the COVID-19 Pandemic. Vaccines. 2022; 10(12):2085.

Chicago/Turabian Style

Pavić, Željko, Adrijana Šuljok, and Juraj Jurlina. 2022. "Balanced Reporting and Boomerang Effect: An Analysis of Croatian Online News Sites Vaccination Coverage and User Comments during the COVID-19 Pandemic" Vaccines 10, no. 12: 2085.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop