1. Introduction
As is widely known, agenda setting theory was formally developed in 1972 by US sociologist Maxwell McCombs, together with Donald Shaw, based on the earlier work of journalist Walter Lippmann [
1,
2]. This theory clearly shows that by selecting which topics to include and which to omit from their agenda, the media exert great influence over the public, along with citizens’ ability to express their opinions and debate public affairs, due to the fact that the media establish the topics of interest, the importance of those issues, and the space devoted to each one [
3,
4].
The nuance offered by scientist and historian Bernard Cohen [
5] (p. 13) is also worth noting. Cohen argued that the press is not very effective in telling audiences what to think yet is highly successful in telling its readers what to think, meaning that each media outlet has its own followers and that each one can influence the thinking of its readers according to its own editorial perspective.
Since 1972, there have been a large number of articles and publications in this field, and agenda setting has become one of the most important theoretical frameworks related to the media, which continues to hold an influential position in communication research, as pointed out below. One of the most striking aspects of the agenda setting theory is that it crosses geographical borders and has an international scope [
6,
7]. Moreover, it encompasses studies related to topics as diverse as illegal immigration [
8] and the way in which media scepticism moderates the agenda setting effect [
9].
More recent studies have addressed the effects of agenda setting carried out by the media on the habits of information searches that take place online [
10,
11,
12] and on the relationship between the media agenda and topics of conversation on social networks [
13]. In short, over the past half a century, hundreds of studies have been published, and even today, the agenda setting theory is still being discussed, despite the considerable transformation that the media have undergone in recent years [
14].
In addition, as mentioned above, disinformation has recently become a serious topic of conversation in the media, and the theory of agenda setting is making its presence felt once again. From the moment that the media’s influence on public opinion [
15] was discovered, disinformation started to become part of the information domain. According to Manfredi [
16] (p. 24), along with many others, the “values inherent to news reporting” are the foundation upon which the media provides the audience with the information they must have, or at least be aware of.
The media are the ones who understand that disinformation is a topic of interest to society. As such, this issue now holds a place in the media and has indeed become a newsworthy event in itself. Directors and editors-in-chief are responsible for the media’s agenda [
13]. Moreover, the fact that they publish reports about disinformation is partly an acceptance of the media’s own responsibility regarding this issue, thereby establishing solidarity with society in recognizing the importance of the problem.
Disinformation is not a new phenomenon, yet the outcry against information disorders, in which we are currently immersed, is so strong that organizations are starting to address the issue, one of which is Spain’s
Centro de Investigaciones Sociológicas (
CIS) [Center for Sociological Research]. In fact, this centre has already included the topic of disinformation in its list of issues in the April 2022 barometer [
17]. In fact, this was the first time that the issue was placed on the list of problems that Spaniards believe currently exist in the country.
Periodically, this institution carries out a survey that includes a list of 68 concerns that Spanish people have, in which the respondents are required to answer the following question: “In your opinion, what is the most serious problem that currently exists in Spain?” In the April 2022 barometer [
17], disinformation, information manipulation, and the diffusion of hoaxes occupied the 25th position. One year later, according to the same source [
18], disinformation moved up to 23rd place, with an increase of nearly 1% in the number of Spaniards who considered it to be a major problem in the country.
This situation is also reflected in the 2023 Eurobarometer [
19] with regard to disinformation: the vast majority of Spaniards are fully aware that fake news is a problem in Spain (83%). This figure is very similar to the average for European countries, where 78% of the citizens believe that disinformation is a problem. However, these percentages of growth are significant when looking at the 2018 Eurobarometer on fake news and disinformation [
20]. At that time, 51% of Spaniards surveyed said that disinformation could be a problem for the country. This means that in just the last five years, a period in which we have experienced a pandemic, a war against Russia, and milestones such as Brexit and Trump’s election victory, disinformation has been placed high on society’s list as an additional problem.
Given the presence of disinformation on the agendas of both the media and citizens, it is not surprising that the fake news phenomenon has been one of the most closely examined issues in recent years in the field of communication research [
21]. During the pandemic, a significant increase in supposedly informative content related to science was observed, which has led to great concern, not only regarding disinformation in general but about scientific disinformation in particular. Google Scholar lists more than 24,400 entries in which the terms “pandemic and disinformation” appear, and 21,700 if the term “communication” is added, which provides a rough idea of the role that information disorders played during the pandemic and how researchers have sought to understand and explain the growing presence of disinformation in both traditional and digital media.
Although social media is considered one of the main points of entry for disinformation [
22], authors such as Blanco Alfonso, García Galera, and Del Hoyo Hurtado [
23] have also highlighted the decisive role that both traditional and digital journalists play in the disinformation process. This is significant, because although the media consumption habits of Spaniards are changing as a result of digitization, traditional media in digital formats continue to have a stronger influence on the way we stay informed compared to social networks [
24]. Before beginning with the primary research, it is of interest to provide a conceptual approach to the two terms that stand out the most in this work: disinformation and fake news. Following these authors [
23], the term “disinformation” is generally used to refer to deliberate attempts to confuse or manipulate people by providing dishonest information. The term ‘fake news’ is recurrently used by political actors for ulterior motives, to discredit journalists and the press whose information they find ‘unpleasant.’
From this standpoint, the present article raises the following research questions: (1) What presence have terms like disinformation or synonyms (fake news, hoaxes) had in both scientific journals and print media during the first two decades of the 21st century? Or, Do preferences for one term or another (fake news, disinformation, hoax) vary depending on the type of publication using it (scientific journals, print media)? (2) Has COVID-19 impacted the quantity and treatment of information regarding science-related disinformation in academic publications and the press?
2. Materials and Methods
Based on the theoretical assumptions of agenda setting, this research aims to verify the degree of interest in disinformation among the mainstream press and academic journals of scientific communication based on frequency, and whether or not disinformation has been observed in the field of science. For this purpose, two time periods have been established within the first quarter of the 21st century, with the dividing line being the COVID-19 pandemic: the first period is from 2001 to 2019, and the second is from 2020 to 2023.
Given this proposal, the following specific objectives have been set based on a comparison of the type of publication (scientific communication journal or newspaper) and the established time periods: (1) determine which terms (fake news, hoax, or disinformation) are most frequently used in the publications; (2) discover the terms that are directly related to the field of science and, within this domain, reveal the subfields to which they belong; (3) verify how disinformation is treated based on an examination of the methodological techniques used in scientific articles; and, as an additional objective, (4) confirm whether the content in newspapers maintains an objective focus regarding information or if it is subjective, all of which are a complementary approach to the processing of content in the publications analysed.
Based on these objectives, the following hypotheses are developed: (H1) despite the controversy that has arisen in recent times regarding the expression “fake news”, which is considered an oxymoron, this term might be the most widely used by the media themselves, even more so than words such as “disinformation” or “hoax”, probably due to the former term being so familiar to the public; (H2) the media’s impact on citizens regarding COVID-19 reflects a greater concern for disinformation by the media; and (H3) the treatment of disinformation in academic communication journals is mainly based on content analysis, while in newspapers it is presented in the form of information. Therefore, there is only scant interest on the part of scientific journals to focus on research topics, and a tendency for newspapers to move away from subjectivity.
Methodology
Beyond the different definitions of quantitative content analysis set forth by pioneering authors such as Berelson [
25] and Holsti [
26], the approach taken by Krippendorff [
27,
28] is the most appropriate for this work, as the expected results have allowed us to make reproducible and valid inferences from the data within their context.
Although content that is journalistic, and scientific in this case, can be analysed in many different ways, quantitative content analysis involves the systematic application of predefined rules. These guidelines are used to measure the frequency with which certain elements of interest appear in a mass of information, which the authors have previously selected in order to study some of the aspects that seem useful for the purposes of our research [
28].
The starting point for the first part of this research was to select the national newspapers that have both digital and print versions, which are the most widely read by Spanish people [
29]: these include
El País,
ABC, and
El Mundo. A search was also conducted for Spanish scientific journals in the field of communication, which were included in the SJR-SCImago Journal Rank database from 2000 to 2023. To do so, a search engine filter was applied using the terms “Communication”, “Spain”, “Journal”, and “2022” (the last year of registration at the time of this research), resulting in a total of 32 scientific journals.
In the second phase, using the Scopus and Google Scholar databases, a search and selection of scientific articles since 2000 was carried out using the Boolean operator “OR” to link it with the terms “Disinformation”, “Hoax/es”, “Fake News”, and “False News”. The two latter expressions have been combined in the analysis. In the case of newspapers, the Lexis Nexis database was used to apply the same terms and the same operator, as well as the same timeframe to enable the selection of newspaper reports published in Spain with headlines that included the aforementioned terms. Initially, a sample of n = 30 was tested (15 articles published in scientific communication journals and 15 journalistic pieces) in order to check the adequacy of the analysis categories.
The final sample was n = 732. Of these units of analysis, 527 (71.9%), were from newspapers and 205 (28.1%) from scientific journals. Of the 32 science journals selected, less than half (14, or 43.75%) included one or more of the search terms, “Disinformation”, “Hoax/es” “Fake News”, and “False News”. On the other hand, all the newspapers included one or more of these terms in the headlines of some of their content.
For this analysis, it is important to remember that the authors have considered whether the terms used appear in the title of the article or in the keywords, and that multiple categorizations were created as follows: (1) based on the terms used and (2) whether the content addressed the field of science or not. If that were the case, the subfield in which they were included was investigated. For the subcategorization, the authors referred to a study conducted by the Spanish Foundation for Science and Technology (
FECYT) based on research into disinformation and science [
30], which served as a benchmark for the present study. The subcategories are as follows: Nutrition and Physical Wellness; Technology; Medicine and Health; Environment and Ecology.
Finally, both the type of publication (V1) and the time of publication (V2) were taken into account as independent variables in order to carry out a comparative study. Regarding the time of publication, the timeframe was established before and after the COVID-19 pandemic. Thus, the demarcation was set from 2001 to 2019, and from 2020 to 2023, in order to verify whether the number of publications increased after the pandemic. Although information on this health crisis started to appear in 2019, and the pandemic started in March of 2020, the timeframe was based on the calendar year in order to take into account the diverse periodicity of scientific journals. Regarding the type of publication, a comparison was carried out based on whether the writing was from a scientific journal or a newspaper.
Bearing in mind that these units of analysis were taken from publications with different perspectives (scientific research for academic journals and divulgatory information for newspapers), a third variable was used to complement the research, which is generically called “Treatment” (V3). In the case of scientific journals, this variable was applied through a classification involving the main methodological techniques used, which yielded the following categories: content analysis (discourse or lexical analysis); survey-questionnaire; literature review; direct systemic or participant observation; and others. Although other methodological techniques were previously included in this categorization, their limited presence with percentages that never went higher than 3% in any of the cases led the authors to group them under the heading of “Other”.
With regard to the newspapers selected, the variable was based on the degree of objectivity required by the journalistic genre that reflected the selected headlines, which is divided into two categories: (1) Information: articles with an objective approach (news, reports, and briefs); (2) Opinion: reports with a subjective approach (editorials, columns, and contributors’ bylines).
The search was carried out between November and December of 2023, and the data analysis was performed using SPSS v.23. For the statistical analysis, contingency tables were used, and significant differences were established at the level of statistical validity: chi-square χ2 < 0.05.
4. Discussion
Disinformation is a source of widespread concern [
30], which has increased among governments and societal groups in recent years [
31]. Although it is not easy to establish a cause and effect stipulated by the theory of agenda setting regarding the media’s role in setting the public agenda on issues such as disinformation, this paper highlights two important issues. Firstly, there is the media’s use of terms such as disinformation and fake news. Secondly, there is a perception that the academic literature and press reports frequently address this issue, which is of growing concern to Spanish society, as indicated by the
CIS report [
18].
Regarding the first research question (What presence have terms like disinformation or synonyms (fake news, hoaxes) had in both scientific journals and print media during the first two decades of the 21st century? Or, Do preferences for one term or another (fake news, disinformation, hoax) vary depending on the type of publication using it (scientific journals, print media)?), the present study confirms that both the press and scientific journals are concerned about disinformation. This finding is possibly linked to its prominence as a topic of conversation and worry among citizens, who believe that they are frequently exposed to disinformation. In fact, one in four people say that they have received false information on scientific topics in the last week [
30]. However, as the findings are based on content analysis, the results do not allow the authors to establish neither the cause nor the effect, and this is one of the limitations of this study. Nevertheless, the theoretical support offered by the agenda setting theory does provide a clear link between what the media considers relevant and the interests of citizens.
The prominence of the term disinformation in the units analysed can also be seen in the wide range of terminologies used. In fact, this research has revealed that the terms addressed in this study are used indistinctively at rates of approximately 33%, yet there is a slight tendency to use fake news more often. The greater precision attributed to the term disinformation is relevant, as opposed to the oxymoron contained in the expression Noticias Falsas, or its English counterpart, fake news. It also allows a broader conceptual scope that encompasses diverse phenomena of information distortion. However, the research concludes that fake news is more frequently used to talk about this problem, which is most likely due to the familiarity of this term by the public.
Regarding the type of publication type in question, the media’s preference for hoax/es over the other two expressions indicates their tendency to use terms that are as clear and brief as possible. In addition, and more importantly, this term avoids the harmful confusion entailed in using the expression fake news, which taints the concept of “news” as being distorted, with the consequent loss of credibility this implies for the most distinctive format in the journalistic profession. The conduct of scientific journals might seem to be an even a greater paradox, where the demand for conceptual accuracy should result in the predominance of the term disinformation, yet the more controversial term is used instead, most likely due to its use in non-specialized fields, and because of its effectiveness as a synonym. Therefore, the first hypothesis of this research is partially confirmed: the expression fake news, which has been utilized more frequently in the pandemic and post-pandemic phase, is the expression that is most heavily used in scientific journals, and the second most used in newspapers.
Concerning the second research question (Has COVID-19 impacted the quantity and treatment of information regarding science-related disinformation in academic publications and the press?), as reflected in this study, the media itself seems to play an important role in the presence of disinformation, as they have taken the initiative to present topics of conversation related to the lack of veracity in information. The results presented above confirm that the 2020–2023 period revealed an increase in the frequency with which newspapers and academic journals have dealt with disinformation, especially in the field of science. This finding is in line with the conclusions of Lagares Díez, López-López, and Pereira López [
32], who have compiled an interesting classification of the publications regarding COVID-19, in addition to the disinformation disseminated in that period. In fact, at that time, there was such a massive amount of information and disinformation related to science and health that the term
Infodemic was coined [
33].
The time variable is a determining factor in the concern for this phenomenon in relation to the category of Medicine and Health, which is a result of its direct relationship with the threat posed by COVID-19. In fact, authors such as Costa Sánchez, Vizoso and López-García [
34] have referred to a health disinformation agenda that originated with COVID-19. At that time, issues in the category of Environment and Ecology took a back seat to the concerns of citizens and researchers, and even Nutrition and Physical Wellness were considered less important as well.
Therefore, in the present study, we can see that once again, the pandemic represents a before and after: the first consequence of the health crisis is a general increase in disinformation and a specific augmentation of false information related to the field of science, especially that of health; the second result is an increase in public interest in disinformation. Finally, regarding the preference of certain expressions or terms over others in referring to this phenomenon, the second hypothesis is confirmed, since COVID-19 indeed increased the media’s concern for information related to science issues, and there has been a larger amount of content linking science and disinformation.
The slight shift toward Opinion in the newspapers analysed in the second stage could have a rationale similar to the one observed in the preceding paragraph. The debate and social concern about the magnitude of disinformation in the second timeframe has been channelled more through opinion genres than in all the years of the preceding phase. All of this seems to reflect the increase in concern and public discussion surrounding information disorders and this could be due to the inclusion of more assessments from experts who have provided their points of view to newspapers regarding the consequences of the global health crisis, which may also be related to the increase in public concern and debate over the phenomenon of disinformation. However, what was expected was a major effort toward this approach from spaces dedicated to Information, which is reflected in the findings of the present research.
Finally, this study has also obtained interesting conclusions regarding disinformation and scientific publications in academic communication journals. In this regard, the time variable again plays an important role, as pointed out by Staender and Humprecht [
35]) in a very recent paper. According to these authors, 400 articles on this topic were published in 2016, yet the average figure rose dramatically to 1500 articles in 2019. The same researchers point out that in the domain of social science, surveys and experiments have become predominant in recent years. This is partly in line with the results obtained in the present study, possibly due to society’s need to answer urgent questions regarding the exposure to disinformation [
36,
37,
38], concern about it [
39,
40], digital literacy regarding the issue, and the ability to recognize it [
41,
42,
43]. However, in the studies analysed in this research, there is a clear preference for content analysis as a research method, as posed by our third hypothesis.
In short, the results clearly show the impact of the media, as they were essential in divulging the link between disinformation and the global health crisis that affected citizens directly. This proliferation of disinformation had a major impact on the pandemic, and neither the press nor scientific research were unaffected by it. In both cases, a strong effort was made to address the highly relevant issue. Although one of the limitations of this study is the failure to reveal the motivations behind the interest in science-related disinformation in newspapers and academic journals, the findings suggest that the problem is being addressed in order to demonstrate its seriousness and to detect and differentiate it from factual content. Therefore, this research contributes to upholding the agenda setting theory. With the descriptive scope of this study, we can only observe which techniques are used with the bias of the time limitation that marked COVID-19. Nevertheless, this question is raised as an additional objective. In future lines of research, we propose to investigate this question further to see if other special circumstances also lead to a difference in the methodological techniques employed.
Moreover, despite the fact that the concept was set forth more than 50 years ago by McCombs and Shaw [
1], and enhanced two decades later by one of the original authors [
44], the concept is still very much alive and relevant to this day. Moreover, in addition to reporting on disinformation, academic research also attempts to uncover the causes of its prevalence, viralisation, origins, and consequences. Nevertheless, in the case of newspapers and scientific journals, whether directly or indirectly motivated, they are definitely influencing society to remain vigilant in the face of an avalanche of disinformation, which is not only worrisome in itself but is capable of producing effects that are even more dangerous when it involves a public health crisis as serious as COVID-19.