Fighting Disinformation: How to Measure the Impact of Pre- and Debunking on Dutch Primary School Children’s Media Literacy?

Abstract

To empower citizens against disinformation in a sustainable way, teaching media literacy is important, especially among young children, enabling them to use this competence throughout their lifetime. Prebunking and debunking are two different teaching methods to enhance media literacy and fight disinformation. Prebunking is a technique based on teaching people to recognize disinformation in advance so that they are aware of it. Debunking is characterized by refuting disinformation after the target group has been exposed to disinformation. To compare their impact, our empirical study conducted an exploratory experiment at two primary schools in the Netherlands in Spring 2024. One group of primary school children (10–13 years old) was exposed to a prebunking intervention (based on inoculation, a psychological approach directed against the disinformation manipulation techniques) while the other group was exposed to a debunking intervention (based on a traditional media literacy lesson). Both groups were questioned about the reliability of digital information (news items) to measure the effectiveness of prebunking compared to debunking. We addressed the following question: To what extent can the impact of pre- and debunking as an educational method for media literacy among Dutch primary school children be measured by an exploratory experiment?

1. Introduction

The rise in ICT has brought many opportunities to our everyday life, such as the availability of information 24/7. However, the rapid scale-up of ICT, including social media platforms, has also brought harmful developments. Without the intervention of gatekeepers such as journalists, it is important to question the quality of information shared on social media and platforms such as X, Instagram, and Facebook. This risks the reduction in reliability of such digital information and may lead to disinformation: “False information which is deliberately intended to mislead—intentionally misstating the facts” [1].

The increasing presence of social media platforms in our everyday lives means that disinformation can reach anyone. Therefore, countering the spread of disinformation is of utmost importance. Legal measures or smart ICT tools could check the reliability of digital information. It is important to check how reliable such measures are and who decides what criteria are used to conduct this check [2]. Moreover, according to Dumitru et al. (2022) [3], the rapid developments around social media platforms also need a more sustainable solution: citizens themselves should be able to adequately assess the reliability of digital information. To achieve this, citizens’ media literacy, defined by Livingstone (2004) [4] as “the ability to access, analyze, evaluate and create messages across a variety of contexts” (p. 3), needs to increase. But how can we improve citizens’ media literacy? Dumitru et al. (2022) [3] discussed the effectiveness of media literacy training and interventions for different generations. In this article, we focus on primary school children, as it is important to start as early as possible in life to enhance media literacy.

A relatively recent remedy against disinformation was elaborated by Roozenbeek et al. (2022) [5]. Their work introduces the inoculation theory as a way of increasing resilience to disinformation. Inoculation in this sense is a metaphor, psychological in nature, and directed against manipulation techniques most often encountered in disinformation. They have incorporated inoculation against manipulation techniques in short videos (see Section 2.1). Inoculation theory relies on prebunking, a technique based on teaching people to recognize disinformation in advance so that they are aware of it, rather than debunking, a technique that is characterized by refuting disinformation after the target group has been exposed to disinformation. Their research shows that such videos improve the ability to recognize manipulation techniques, increase confidence in recognizing them, and help people distinguish between unreliable and reliable information. According to Roozenbeek et al. (2022) [5] (P. 1), “the effects [of these videos] are robust across the political spectrum and a wide variety of covariates” (p. 1). They claim that psychological inoculations based on manipulation techniques are effective in improving resilience to disinformation. ”To further complicate this problem, correcting misinformation after it has spread (for example, through fact-checking) comes with several challenges: establishing what counts as factual information is epistemologically difficult, particularly in the context of politics [6]; fact-checks are unlikely to reach everyone who was exposed to the initial misinformation [7]; encouraging people to believe fact-checks is challenging [8]; effective interventions are hard to scale to a population level [9]; and testing the effectiveness of interventions in the real world (as opposed to a laboratory setting) is complicated [10]. Debunking misinformation is also problematic because correcting misinformation does not always nullify its effects entirely, a phenomenon known as the “continued influence effect” [11]. Accordingly, in contrast to debunking, prebunking has gained prominence as a means to preemptively build resilience against anticipated exposure to misinformation (Roozenbeek and Van der Linden, 2019 [12]).”

To empower citizens against disinformation in a sustainable way, it is important to look at how we teach media literacy. Improving media literacy education among young children is of great importance, according to the Dutch saying Jong geleerd is oud gedaan (learned young is done old). To compare the impact of prebunking and debunking as a teaching method to enhance media literacy to fight disinformation, our empirical study conducted an exploratory experiment at two primary schools in the Netherlands in Spring 2024. One group of primary school children (10–13 years old) was exposed to a prebunking irvention (based on inoculation) while the other group was exposed to a debunking intervention (based on a traditional media literacy lesson). Both groups were questioned about the reliability of digital information (news items) to measure the effectiveness of prebunking compared to debunking. Our empirical study addressed the following question: To what extent can the impact of pre- and debunking as an educational method for media literacy among Dutch primary school children be measured by an exploratory experiment?

2. Methodology

2.1. Inoculation Theory

Roozenbeek et al. (2022) [5] introduced the inoculation theory in their article ‘Psychological inoculation improves resilience against misinformation’ related to social media, relying on prebunking. Inoculation metaphorically uses the medical procedure of a vaccine against viruses while the patient is protected by injecting a small dose of that virus, thus producing antibodies to enhance resistance to the disease. Their empirical study tested the effectiveness of inoculation by using short videos covering five manipulation techniques: “emotionally manipulative language, incoherence, false dichotomies, scapegoating, and ad hominem attacks” [5]. “Each video instantiates the inoculation procedure by first providing a forewarning of an impending misinformation attack, then issuing a preemptive refutation of the manipulation technique used in this attack, and lastly presenting a “microdose” of misinformation in the form of innocuous and humorous examples (such as an example of incoherence from the animated television series Family Guy). All examples are nonpolitical and fictitious, in addition to being humorous, to avoid any appearance of partisan bias and prevent triggering defensive motivated cognition. The videos can be viewed at https://inoculation.science/inoculation-videos/” [5] (pp. 1–2).

Roozenbeek et al. (2022) [5] not only investigated the effectiveness of these videos but also conducted an experiment on YouTube for this purpose that involved almost 30,000 participants. After watching the videos, they asked participants to rate ten fictitious social media posts. These randomly selected messages were similar in length and topic, and contained a neutral or manipulated message. Roozenbeek et al. (2022) [5] designed the messages, each time with one of the manipulation techniques, to measure “participants’ ability to recognize manipulation techniques in social media content (Cook & Lewandowsky, 2017 [13]); their confidence in their ability to recognize these techniques (Basol et al., 2020) [14]; their ability to distinguish trustworthy from untrustworthy content (McGrew et al., 2020 [15]); and the quality of their sharing decisions, i.e., willingness to share (Pennycook et al., 2021) [10]” (p. 2). They concluded that the short videos improved their ability to identify manipulation techniques in a social media context, which could easily reach millions of users. They not only increased viewers’ self-confidence in recognizing manipulation techniques, but they were also less likely to share manipulative messages. However, their empirical study had some limitations. It was not possible to draw conclusions on how long inoculation remains effective, and the content of the videos was only focused on US citizens, which means that no conclusions can be drawn about other countries. To sum up, according to Roozenbeek et al. (2022) [5], prebunking tools based on inoculation theory enhance resistance to false information.

2.2. Bad News Game

A good illustration of the effectiveness of using prebunking for inoculation was offered by the online Bad News game. Roozenbeek and Van der Linden (2019) [12] used this online game in their empirical study ‘Fake news game confers psychological resistance to online misinformation’. This game was developed by DROG (a Dutch platform for critical news consumers and creators of the Bad News Game, https://www.getbadnews.com/en accessed on 22 March 2025) in collaboration with Roozenbeek and Van der Linden. This game uses inoculation theory and playfully introduces users to the manipulation techniques used in the production and dissemination of false information. During this 15 min game, players crawl into the role of disinformation disseminators and are introduced to what Roozenbeek and Van der Linden [12] (p. 4) call the six misinformation techniques: “(1) floating conspiracy theories, (2) impersonating people online, (3) using emotional language, (4) discrediting opponents, (5) trolling people online, and (6) group polarization.”

The goal of the Bad News game is to obtain the highest possible number of followers and, at the same time, keep trustworthiness high. The number of followers and trustworthiness are influenced by the choices players make while spreading messages. By taking the role of a disinformation spreader, players become familiar with the manipulation techniques and start using them themselves to gain as many followers as possible. Players are punished for making ‘ethically correct’ choices by losing followers and trustworthiness. The player loses when the trustworthiness score reaches zero, while the total follower count also serves as a score at game end.

The assumption is that going through this process while playing improves cognitive immunity when people encounter real disinformation in their daily lives. So, the Bad News game used by Roozenbeek and Van der Linden (2019) [12] is based on inoculation by actively using prebunking tools where the player is introduced to false information in advance with the aim of recognizing it in everyday situations.

According to Roozenbeek and Van der Linden (2019) [12], “in contrast to debunking, prebunking has gained prominence as a means to preemptively build resilience against anticipated exposure to misinformation” (p. 2). For this reason, they decided to adopt a prebunking approach for their empirical study by making use of the Bad News game.

The game measured players’ ability to recognize manipulation techniques by collecting data during a three-month period. Players were asked to rate the trustworthiness of news stories on a scale of 1 (unreliable) to 7 (reliable) before and after playing the game. They rated six news headlines and two control headlines.

According to Roozenbeek and Van der Linden (2019) [12] the Bad News game has proven to be an effective inoculation tool against manipulation techniques, and they concluded “We provide initial evidence that people’s ability to spot and resist misinformation improves after gameplay, irrespective of education, age, political ideology, and cognitive style” (p. 1).

Finally, they argued that playing the Bad News game improves the ability to recognize false information regardless of age. Although their empirical study distinguished different age categories of participants (18 or younger, 19–29, 30–49, 50+), Roozenbeek and Van der Linden (2019) [12] did not elaborate on the composition of their youngest group of participants. Therefore, to contribute to insight into media literacy education for primary school children, our empirical study investigates the impact of the active prebunking Bad News game on Dutch primary school children (10–13 years old) compared to the impact of a more passive debunking traditional lesson.

2.3. Prebunking Versus Debunking

Lewandowsky and Van der Linden (2021) [16] state that false claims spread faster than real news and that these messages then also ‘stick’ with recipients for a long time. The effectiveness of corrective education on disinformation (debunking) is often questioned (e.g., [16]). So, one can ask to what extent offering an alternative explanation or a critical examination of the source (debunking) is indeed suitable to make people resilient against disinformation. Maybe people do not always accept alternative explanations, for instance, due to a lack of evidence. Ideology or political views could also hinder the acceptance of an alternative explanation. And they also mention that “a final problem with debunking is that it is often forced to adapt a disadvantageous framing at a disadvantageous time. One often unavoidable attribute of corrections is that they tacitly accept someone else’s rhetorical framing, thereby permitting the actor who promulgated the original falsehood to set the agenda” (p. 355).

To what extent prebunking as a media literacy intervention is indeed more effective than debunking to fight disinformation in primary school children is a question that needs attention and, therefore, will be addressed by the explorative media literacy experiments we present in our empirical study. As stated in Section 1, we follow Livingstone’s (2004) definition of media literacy: “the ability to access, analyze, evaluate and create messages across a variety of contexts” (p. 3). Because of the scope of our empirical study, the measurement focused on the ability to analyze and evaluate social media, which aligns with the inoculation theory that Roozenbeek et al. (2019) [12] used for their empirical study, based on the Bad News game. As we explained in Section 2.2, the Bad News game highlights different manipulation techniques during its play. The underlying aim of the game is to improve the ability to recognize these techniques. Analyzing and evaluating online content relates to this ability.

3. Methodology

3.1. The Primary Schools

In Spring 2024, the first author of this publication approached two Dutch primary schools through personal contacts. After an introductory meeting, she discussed its aims and procedure with the teachers. The children’s parents/carers then received a letter, in which she explained the aims and procedure and asked them to sign an informed consent form to confirm that their child was allowed to participate in the empirical study. Both primary schools participated in the Dutch Media Literacy Week 2023–2024, paying attention to media literacy themes such as online bullying or deepfakes (https://www.weekvandemediawijsheid.nl/), offering the classes a similar starting position regarding media literacy.

The two primary schools are similar, both in media literacy education and, in general, composition. In each of the two conditions of this empirical study (playing the Bad News game (prebunking condition) and participating in a traditional lesson including an online memory game (debunking condition)), a relatively low number of primary school children participated (see

Table 1. Number of primary school children for each condition.

	Total Number of School Children	Girls	Boys
Prebunking condition	21	11	10
Debunking condition	20	10	10

). But the number was sufficient for an exploratory experiment like this to examine whether the research design we developed can be used to gain insight into the extent to which prebunking as a media literacy intervention is more effective than debunking to fight disinformation in primary school children.

The two primary schools were visited twice. The first visit served to become acquainted with the children. It is important to establish a rapport with the children so that they have a good understanding of the media literacy project. The content of the investigation was explained in broad terms. The first author told the primary school children that they were going to follow a programme about disinformation and fill in questionnaires. She deliberately did not tell them yet that we were going to split up the group and have a different programme for both groups because we did not want to influence the results.

During the second visit to each primary school, she randomly divided the class into two groups. The children in the debunking condition group participated in a traditional lesson, including an online memory game, while those in the prebunking group played the Bad News game (prebunking condition). Although the empirical study by Roozenbeek and Van der Linden (2019) [12] was conducted in English, DROG also developed the game in Dutch (https://www.getbadnews.com/nl/ accessed on 22 March 2025). For this reason, nothing needed to be translated into Dutch. The Bad News game could therefore be used for our empirical study without translations being a hindering factor.

Before splitting the group, the primary school children were asked to complete a questionnaire. They were told that they were going to take some kind of test that they should conduct in silence. This questionnaire served as a baseline measurement to provide insight into the primary school children’s ability to recognize disinformation in six news headlines. After having played the Bad Game or having followed the traditional lesson, including an online memory game, the primary school children were asked to complete the same questionnaire again, with different but similar questions. They were again told that they were filling in a questionnaire that served as a kind of test. We clearly mentioned that the test did not imply grading.

In their empirical study, Roozenbeek and Van der Linden (2019) [12] did not make measurements that could capture the long-term effect of the game. Dumitru et al. (2022) [3] highlighted both the importance and the lack of training and interventions that focus on long-term solutions. To investigate the sustainability of the interventions of this empirical study, we decided to send a questionnaire with six other news headlines to the classes four weeks after the interventions in the primary schools. See also Section 3.3 for more information about the questionnaires.

3.2. Conducting the Exploratory Experiment in the Classroom

To conduct our explorative experiment in a rigorous way, it was important that the intervention conditions were as similar as possible. The only difference was the prebunking and debunking character of the media literacy intervention in each condition. The following is important for the research design:

1. It is of utmost importance that the ‘fun’ of the game is not a determining factor. Bad News is a game where children are able to take on the role of the bad guy. The game is played online and allows the player to behave ‘badly’ to learn about manipulation techniques. As a player, you make your own free choices. These elements make the game interesting and funny for children. To ensure that this ‘fun factor’ does not have an influence on the children and how they do their best, it is important that the traditional lesson is conducted in a similar way, so it should also have a ‘fun factor’, which was performed by letting the school children play an online memory game (see point 3b below).

2. Both groups spent a comparable period of time during the lesson to keep it as comparable as possible. So, it was important to manage time well and match the duration of the Bad News game to the duration of the traditional lesson with the online memory game.

3a. The Bad News game is an individual game and forces the players to think for themselves about the choices being made. It was important that the traditional lesson also had an individual component: the online memory game. Of course, it was not possible to have no interaction at all during the traditional lesson, but it was required that primary school children complete the questionnaires individually, without any collaboration.

3b. Like the digital game Bad News game, the traditional lesson covered the six manipulation techniques, but here the focus was on a presentation to explain these six techniques (1) floating conspiracy theories, (2) impersonating people online, (3) using emotional language, (4) discrediting opponents, (5) trolling people online, and (6) group polarization. In this way, the lesson consists of a ‘traditional’ teaching method in which the focus is on absorbing the material through listening and reading. To keep the traditional lesson comparable to the Bad News game, the primary school children played an online memory game on their iPad after the traditional lesson, in which the six manipulation techniques had to be paired with their descriptions. The pairs of the online memory game consisted of pictures with the name of the manipulation technique and its corresponding description at the primary school children’s reading level (see Figure 1). The memory game was designed to have the same visual elements as the Bad News game to keep the conditions as similar as possible. Children who were fast at the game were allowed to play it multiple times until time ran out. The addition of a relatively short (5 min) online memory game to the traditional lesson made the debunking condition comparable to the fun element of the Bad News game. Moreover, the presentation looked visually interesting and contained examples that were humorous, just as the Bad News game.

It is important to note that the online memory game is active only to a very limited extent, so the vast majority of the traditional lesson remains passive and uses debunking. This allowed us to examine the extent to which the impact of prebunking and debunking as an educational method for media literacy among Dutch primary school children can be measured.

Before the game started, there was a brief explanation to the group about what they were going to do. For this, we used the basic explanation that can be found on the Bad News website itself (see Figure 2). We adapted it to fit the primary school children’s reading level, as the Bad News game is not specifically developed for children. Nevertheless, the language used is certainly not incomprehensible to children, and with explanations, they were able to play the game. To be absolutely sure of this, the first author presented the game to the teachers and four children (two boys and two girls). She then asked these children whether the game was easy to follow and understand. According to the teachers, the game was easy to play, but there was room for questions about certain terms. The children we asked for help also said they understood the game, but found some words difficult or did not know them. To ensure primary school children received help when a term was not clear, their teacher walked around the room during the gameplay to answer questions and offer help.

After having completed the Bad News game (prebunking group) and the online memory game (debunking group), the school primary children were given a link and completed this second questionnaire. It was ensured that they did not confer with each other while completing the list.

3.3. Measuring the Primary School Children’s Media Literacy

To measure media literacy related to disinformation based on the six manipulation techniques among primary school children, we made use of the questionnaires used in Roozenbeek and Van der Linden’s questionnaires (2019) [12] (see Section 2.2). We contacted Van der Linden via e-mail to ask if we could use and adapt this questionnaire because the questions in their empirical study were not suitable for the primary school children from our empirical study, as they were 10–13 years old (Van der Linden, personal communication, 13 March 2024). This was approved and even strongly recommended since we were working with a very different target group than was the case during Roozenbeek and Van der Linden’s study [12].

The first questionnaire started with a series of 6 news headlines. The primary children had to rate these news headlines based on reliability using a 7-point Likert scale (1 = very unreliable, 7 = very reliable). The 7-point Likert scale was based on the one used by Roozenbeek and Van der Linden (2019) [12]. We adapted it visually for children’s use (see Figure 3).

Then, the second questionnaire consisted of six other news headlines for which the primary school children rated the reliability. After four weeks, they received the third questionnaire from their teachers with six other news headlines.

So, for each questionnaire, the primary school children assessed six news headlines, four of which contained a manipulation technique. The two neutral messages served as the control questions and contained truthful information. Roozenbeek and Van der Linden (2019) [12] argued that the control questions should be neutrally formulated and come from a legitimate sender such as the ‘New York Times’ (p. 4). The control questions in Roozenbeek and Van der Linden’s (2019) [12] empirical study reflected (world) news relevant to English-speaking persons. Thus, it is important that the news items, both fake and real, contain relevant information reflecting the primary school children’s, in this case, Dutch, lifeworld. The control questions serve to check social desirability; after all, it is conceivable that the children could become sceptical and, for that reason, would judge all news headlines as unreliable, and therefore, not critically assess the content. Roozenbeek and Van der Linden (2019) [12] argue that when the control questions are rated as unreliable after the interventions, this could possibly mean that the participants developed an overall sceptical attitude towards questionnaires rather than actually thinking that the questions are not reliable.

Prior to administering the questionnaires, it was important to take into account the primary school children’s living environment while creating the questions. We drew inspiration from articles and episodes of the Youth News by the Dutch public broadcasting foundation NOS. While drafting the news headlines, we made sure that the messages were applicable and understandable to the children by pre-testing them with two girls and two boys, aged 10–13 years old. In addition to this pre-test with the four children, we also presented the news headlines and lesson materials to the teachers. After this check, we adjusted the news headlines where necessary to ensure readability at the right reading level. We also conducted such a check for the explanation of manipulation techniques in both conditions.

Other than understandable language and relevant topics, it was also important to ensure that the news headlines clearly included one of the manipulation techniques. To incorporate the manipulation technique ‘disguise’ into the questionnaire, we chose to use well-known channels and brands: the Dutch public broadcasting foundation NOS (and its Youth television programme) and Jumbo supermarkets, a well-known supermarket chain with lots of stores and frequent advertisement on television. These logos and names are easy for the primary school children to recognize, and this way, we prevented children from not being able to recognize a ‘disguise’ manipulation technique because they have never heard of a channel or company, for example. For the other techniques, it was important to come up with realistic channels and newspapers because the primary school children have to judge messages and not the channel or the newspaper that disseminates them. The news headlines should not be implausible (see also [12]). Instead, the aim is for the primary school children to be able to discover the technique used in the news headline and judge its reliability on that basis. So, we created news headlines that come from realistic, but fake, channels and that do not appear particularly implausible despite the techniques they contain.

To combat the spread of fake posts and users’ identity fraud or channels, many social media platforms such as Instagram, Facebook, X, and TikTok use an authentication tool, often a blue check mark behind the username. Accounts can receive such a blue check mark when they are verified. While this can be a helpful means of accepting or not accepting information, this effect is negated by the fact that one can buy the blue check on some platforms. To prevent primary school children from labelling all news headlines as unreliable due to the lack of such a blue check mark, we emphasized at the beginning of all questionnaires that there are no blue check marks giving away the reliability of messages, so that they really have to judge the message themselves.

4. Discussion

First of all, it is important to emphasize that due to the exploratory nature of the study, along with the limitations posed by sample size, it was not the aim of our study to generalize the results (see also limitations in Section 5). We used our unique dataset for an exploration to see if our method can be used to compare two kinds of media literacy interventions to fight disinformation.

The debunking group analyzed news headlines better than the prebunking group immediately after the intervention. Although the prebunking group showed a slight improvement during the measurement for the manipulation techniques ‘polarization’ and ‘conspiracy’ immediately after the intervention, the measurement four weeks after the intervention showed that this improvement was not sustained. It appeared from the measurement four weeks after the intervention that the primary school children in the debunking group neither showed a sustained improvement. However, the debunking group showed more improvement immediately after the intervention: it analyzed the manipulation techniques ‘disguise’ and ‘conspiracy’ better than the prebunking group. Our explorative empirical study thus showed that the 10–13-year-old primary school children did not analyze news headlines better after following a teaching method with prebunking compared to a traditional lesson based on debunking. Media literacy did not improve in both groups. What is striking is that the primary school children appeared to have mastered the manipulation technique ‘disguise’ well, unlike the manipulation techniques ‘emotion’, ‘polarization’, and ‘conspiracy’. During all measurements, it could be seen that both groups had a good command of the ability to correctly analyze fake accounts. To draw conclusions about the reason behind the mastery of the manipulation technique ‘disguise’ would require further research. One possible explanation for this outlier is the use of recognizable and familiar channels and companies. This was a deliberate choice so that the primary school children’s environment was represented in the questionnaires. Awareness about fake accounts and identity fraud on the internet is also a topic that is covered during the Dutch Media Literacy Week, to which both schools participated. This means that the primary school children are familiar with this topic. Moreover, recognizing logos and names is a relatively concrete act compared to the other manipulation techniques.

The ability to evaluate content was measured by asking the children for each news headline why they attributed their rating. In this way, it became clear for what reason they found a message (un)reliable. The school children in both groups were well able to evaluate news headlines with the ‘disguise’ manipulation technique: in the open-ended responses, the vast majority showed that they could name this manipulation technique. The debunking group was even better at recognizing fake accounts during the measurement immediately after the intervention, and maintained this improvement four weeks after the intervention quite well, although it only weakened slightly. Both groups evaluated this technique well, indicating a high level of media literacy. However, the prebunking group did not improve after the intervention as the debunking group did. This means that the primary school children of the debunking group started to evaluate the content better than the prebunking group. In this sense, the debunking group, on average, became more media literate in terms of evaluating content.

Although both groups also improved in evaluating the manipulation technique ‘conspiracy’ during the measurement immediately after the intervention, the measurement four weeks after the intervention showed that this improvement was not sustained. The ability to evaluate declined and reached the pre-measurement level. Neither the prebunking group nor the debunking group showed long-term improvement in evaluating news headlines. The manipulation techniques ‘emotion’ and ‘polarization’ were neither better evaluated by the groups. However, it became clear that the debunking group made the biggest progress during the measurement immediately after the intervention, even if this was not sustained after four weeks. As well as for the ‘conspiracy’ technique, the ability to evaluate ‘emotion’ and ‘polarization’ declined and reached the pre-measurement levels. So, the primary school children do not evaluate news headlines better after taking a traditional lesson or playing the Bad News game for the manipulation techniques ‘conspiracy’, ‘polarization’, and ‘emotion’.

Nevertheless, it is valuable to examine the unexpected success of the debunking group more closely. Despite the fact that the increase was not sustained in the final measurement, it appeared that the primary school children from the debunking group performed best, according to the questionnaire, immediately following the interventions. A possible explanation for this is that they had the opportunity to engage in a conversation with the teacher and fellow primary school children during class. Dialogue ensued based on questions asked about the content of the lesson. Also, the group was smaller. This could also have an effect on their engagement during class compared to a full class. The interaction with teacher and fellow primary school children needs to be studied on a larger scale to identify to confirm the unexpected success of the debunking group.

5. Conclusions, Limitations, and Implications for Future Research

The empirical study [17] addressed the following question: To what extent can the impact of pre- and debunking as an educational method for media literacy among Dutch primary school children be measured by an exploratory experiment?

The results show that the research design we developed allowed us to measure the impact of prebunking and debunking as an educational method for media literacy among Dutch primary school children. The pre-tests with four children and the consultation of the teachers helped us to develop and refine the questionnaires. The questionnaires we finally used for the group of primary school children who played the Bad News game (prebunking group) and those who took the traditional lesson including the online memory game (debunking group) made it possible to draw conclusions about the effect of prebunking versus debunking in teaching media literacy related to their ability to analyze and evaluate news headline. See the end of the section above for more specific information.

So, our method can be used to compare two kinds of media literacy interventions to fight disinformation. As our empirical study was conducted at only two primary schools with a limited number of children (41) in one country, we recommend a large-scale follow-up experiment in more primary schools with more children and in different countries. That would also offer insight into the question of whether pre- or debunking is the right strategy to follow if we want to fight disinformation in an effective way.

As mentioned, it is important to note that while observing the teaching in the traditional lesson, we noticed that primary school children asked questions, sometimes followed by a conversation about these questions. Although the Bad News game was introduced to the primary school children in an activating way, they had no interaction with each other or the teacher. The prebunking group played the game individually. So, it is activating on the one hand, but it isolates the learners from each other. In the traditional lesson, the design did not encourage classroom discussion in order to keep the conditions of the interventions comparable. But it contained an interactive element between the primary school children and their teacher, as this is inherent to this type of lesson. To further investigate the extent to which this affects the effectiveness of pre- and debunking interventions, we recommend setting up a follow-up empirical study with a third condition. This should be constructed as a classroom lesson in which primary school children interact, stimulated by media literacy materials, with each other and the teacher, followed by an activating element, for example, the Bad News game, or shorter, game-based modules, which could provide promising avenues for subsequent research.