MAthE the Game: A Serious Game for Education and Training in News Verification

Abstract

During the last years, there has been a growing multidisciplinary interest in alternative educational approaches, such as serious games, aiming at enhancing thinking skills and media literacy. Likewise, the objective of this study is to present the design and the development of an educational web application for learning the necessary steps towards the detection of bogus content, according to the fact-checking procedures. The game presents news articles, which have to be characterized as fake or real by the players. During the effort to reach the correct decision, the players can use tools and practices for identifying relevant information regarding the clues, which frame a news story (title, date, creator, source, containing images). After presenting the progress of interface design and development, this paper reports the results of a randomized online field study (n = 111), which provides some preliminary evidence. Specifically, it is validated that the game can raise awareness, teach about authentication tools, and highlight the importance of patterns that include evidence regarding the authenticity of articles. Additionally, thorough discussion was conducted within a media class (n = 35) to receive useful feedback/evaluation about the offered utilities and their usability. The findings suggest that educational games may be a promising vehicle to inoculate the public against misinformation.

1. Introduction

The knowledge and the means to acquire and analyze it are exceptionally vital in any aspect of human life. Learning paths should always be evolving, mainly because web-based knowledge have flooded online reality, and people face difficulties to evaluate it. Widespread misinformation on the Internet is a consequence of the above challenge in information evaluation and makes everyone suspicious or prone to debate everything [1]. Content that is entirely false is manipulated to look like a real journalistic report and fake headlines go viral [2]. These posts usually have powerful symbolism so that people are encouraged to share them. Simultaneously, creators of content, in an effort to increase their profitability, support the reproduction of false materials, which most of the time, cause much more reactions than ordinary news (emotional, political, etc.) [3].

Misinformation forms a demanding and complicated research field, where multiple scientific and applied disciplines are involved. It is a fertile ground for scientific articles, novel technological proposals, business/startups, newspapers articles, and others. Although there are plenty of approaches trying to resolve the problem, the real, societal impact is difficult to handle. The above partly occurs because the issue is discussed behind the closed doors of the “verification industry”, which includes researchers deriving from differentiated fields, debunking websites, digital forensics techniques, institutes, and consortiums that try to analyze and propose solutions [4]. It goes without saying that in the case of such particularly complex and multidisciplinary problems, more than one partners have to cooperate. Thus, in fighting against false information, the number of initiatives, web environments, organizations, platforms, and tools of verifying actual information have increased enormously. By studying related research, it is easy to observe that most attempts to face the problem of misinformation through automated solutions seem sole and/or incidental [5]. More specifically, examples of false news have been identified, discussed, analyzed, and used to train classification algorithms that recognize inaccurate information in favor of humans [5,6]. For this reason, most of the already implemented web-services focus on one kind of multimedia content, i.e., text, image, audio, video, URLs [7]. Generally, existing environments try to automate the encompassed tasks of human-operated validation of sources and stories. However, following the above approach, people learn to depend on software for coming to conclusions and decisions, a habit which has never proved to be adequate and beneficial in the long run [7,8].

Among others, cross-validation procedures and initiatives have emerged as the most useful practices to separate fact from fiction. In this direction, platforms like Facebook have taken steps to help citizens access verified information, connecting them to related procedures and repositories. Since, fact-checkers have played an important role in this battle by critically evaluating claims, thus informing citizens about their integrity and credibility [9]. These groups of media experts are usually equipped with abilities and methods to identify unverified stories and to look for evidence that raise neutrality. However, the volume and speed of information traveling on social media are beyond the influence of any network of fact-checkers, mainly because most of the curation assessment is still done manually. This is further deteriorated by the fact that the visibility and “viral-ness” of the debunked posts are significantly lower than the hoaxes [10]. Therefore, the target services do not really reach the citizens, resulting in lack of awareness of the problem and its potential solutions. However, research has shown that being aware of the factual information is not enough to stop misinformation dissemination.

Recent studies have indicated that individuals often resist fact-checking messages that oppose their prior beliefs [10]. In a landscape of media pluralism and lack of censorship, the solution would be to trigger people to critically think, estimate risks, and judge the impact of information on society. Hence, instead of censoring or forbidding content, tools should enhance users’ curiosity, inviting them to behave as active thinkers, to investigate details and explanations and, consequently, to learn to distinguish qualitative content [4]. While it is difficult to cultivate skills and know-how for facing contemporary issues in which the audience lacks basic literacy, experiential learning strategies have appeared as a viable alternative. Specifically, gamification approaches are considered a useful tool for informal training, taking advantage of the appealing storytelling, their engaging character, the power of the visual media, and overall, the dynamics of the technological capacities and the rich media experience [7]. Among others, information accuracy can be investigated through a gamification model, offering the additional advantage of unveiling related mechanisms and best practices. Far away from the hyperactive Internet world, we may be able to establish the foundations for upgrading digital literacy that will make us stop confronting any information that passes in front of our eyes [10].

In recent years, gamification has become a favorite subject for academic research and development (R&D), since it is considered a promising future educational approach [11]. Most of the studies in education contexts consider the learning results of gamified processes or activities as mostly positive, increasing motivation and engagement in the implicated tasks, while also offering entertainment [12]. Though the main goal of games is to generate pleasant moments/feelings, infotainment values and their widespread applicability gave more new functions in various aspects of everyday life, such as training and knowledge sharing. Hence, serious games have launched, with the purpose to primarily train, investigate, or advertise [13]. Featured activities are designed to promote the desired actions, ushering an enjoyable way to solve real-world problems [11]. Overall, approaches of gamification and serious games are mutually bounded, both trying to leverage gaming aspects to achieve something beyond playfulness.

Nah, Zeng, Telaprolu, Ayyappa, and Eschenbrenner (2014), in their review paper [14], analyze the impact on learner outcomes, which are the engagement, the participation, the motivation, the enjoyment, the productive learning experience, the sense of achievement, accomplishment, and performance. The above-stated results can be attained by the application of design game elements such as points, levels/stages, badges, leaderboards, prices and rewards, progress bars, storyline, and feedback. However, besides the above-mentioned principles, further studies emphasize the necessity for developing gamified systems that trigger intrinsic motivations rather than replacing them with extrinsic rewards (points and badges) [11]. Summing up, these theoretical perspectives were considered with the inception of the game idea and the associated analysis tasks (along with the rest of the technical and functional requirements, i.e., availability of debunked fake news items, grouped at differentiated difficulty levels.

Apart from the content itself (i.e., the news items representing the gaming assets), the need for semi-automated authentication solutions emerges, to provide ways and digital literacy on how to validate journalistic material in the media ecosystem. In all cases, learning is considered a continuous personal process, in which new technological capacities could also be exploited. Hence, in today’s ubiquitous society, interactive services and serious games can facilitate life-long learning, helping individuals to keep up with the new trends [15]. Undeniably, serious games offer new experiences, and we are still at the beginning of exploring the real capabilities that will emerge. The existing interdisciplinary implementations confirm the importance of this new paradigm for learning. Besides the traditional educational games, which aim at transferring knowledge to the players for a scientific field (geography, history, music, etc.) [16,17], the range of serious games that have already been developed is extensive; often because of the educational character, they have active social ingredients [18]. For example, there are games that aim at helping children with autism to understand and express their emotions [19,20,21]. Though most approaches primarily utilize the visual sense, there are also interesting designs that exploit the diegetic nature of audio, thus relying on sound perception, either solely (audio-only games) or in combination with other human senses [22,23,24,25,26]

The current paper introduces a serious gaming approach to combat the misinformation issue, aimed at offering the required familiarizing on all aspects of this unwanted phenomenon. “MAthE the Game” (media authentication education) [27] deals with a very urgent necessity in today’s digital world, making it a good match with the aims of technology-enhanced learning. The name of the project is inspired by the Greek word “μάθηση” (=mathisi), which conceptually means education, i.e., the process of knowledge acquisition. This topic fits comprehensively and concisely with the gamification concept, gathering all the knowledge on how to evaluate information authenticity, while putting users in the game role of the verification character. Hence, it will turn digital media from a time-consuming process into a pleasant challenge. The range and characteristics of potential players are broad, from simple users of social networks to journalists and scientific field specialists. The player acquires detailed knowledge, and also controls, evolves, activates, and acclimatizes to the new information hyperactive reality. Eventually, the goal of the game is for the users to acquire the knowledge and know-how to identify the realness of information with partial machine assistance and to be capable of transmitting accurate information instead of propagating misinformation [28]. But how do we cultivate confidence about news and sources? Of course, by upgrading our critical ability to recognize the true nature of the information. The point is to stop fearing the information, to accept it, and understand it.

Similar serious games on the misinformation subject have been created before, but there was not a Greek version that utilizes scenarios that derive from the associated news agenda. Taking into consideration that the battle against misinformation must take place in restricted extent (in terms of country, language, topics agenda, etc.) in order to have the best possible results [3], MAthE can be a significant contribution, especially since it brings forward some novel ideas. However, before the discussion focuses on these features, it is essential to conduct an analysis of related products. Fake News is a board game (not application) testing people’s ability to distinguish between what’s true and not, known with the slang expression “what’s just been tweeted”. Regarding its rules and structure, first, players are divided into groups of 2–4 people. These groups are then randomly assigned to one of four key characters: (1) The denier, (2) the alarmist, (3) the clickbait monger, and (4) the conspiracy theorist. Each group is given a “source card”, presenting the background of the article that the players will produce, and also a “fact-sheet”, in which the issue is explained in detail. Additionally, the fact-sheet lists possible reasons behind the rise in incidents. Based on the specific goals and motivations of their character, players are then instructed to use the information from the fact sheet to create a fake news article [29,30]. Another already developed online game is the BBC IReporter, in which the players pretend to be BBC journalists [31]. Their task is to create a breaking news story and thereafter publish it to a “BBC Live” site. Their story is judged on metrics of balance accuracy, impact, and speed. An interesting approach is the Factitious, a three-step game in which the players should read the containing articles, swipe to the right if they think that it is a real story, and swipe left if they think that it is a fake story. The noteworthy point is that, according to the fact-checking rules, Factitious encourages players to read the article to the end (by providing a small bonus) and to dive deeper (by offering some other tips and suggestions for spotting fake news) [32]. The Bad News game takes the user behind the scenes and into the mind of a fake news distributor. Through various options, the player creates fake stories and tries to introduce them into an online news site to increase the number of its followers. By the above procedure, game-creators hope to teach the dangerous societal consequences of misinformation [33]. Last but not least, Fake it to make it is a game that hopes to make players more aware of how and why fake news are written and distributed [34], formulating as outmost target to enable users to be more skeptical of what they encounter in the future. A different kind of application, trying to familiarize the audience with the problem, refers to online games and quizzes that test our ability to identify the true or false nature of an image, based on our cognitive skills and experience. Examples of the above platforms are the Real or Photoshop quiz by Adobe and the Real or fake Photo game [35,36].

Up until now, the discussion was about the field of news verification and the related gamification approaches. The rest of the paper is organized as follows: Section two focuses on the four stages (analysis, design, development, evaluation), which have been followed for the implementation of MAthE. Section three presents the interface and the two evaluation procedures. Section four discusses the findings and the conclusions.

2. Methodology

This work involves the methodological and technological framework of a serious game design, with the purpose of offering education and training in media authentication. Prior to actual implementation, it was necessary for us to acquire knowledge and skills in creating and evaluating interactive interfaces. The application production was completed in 6 months, following five main phases, which are analysis, design, development, implementation, and evaluation, according to the instructional systems design (ISD) framework, known as ADDIE model, which is an acronym for the five phases it defines for building training and performance support tools [37]. The exploratory process of analysis involves information gathering about the current situation and already existing games. This would help to determine the tasks to be completed, to select an applicable theme, and to investigate available technologies that could be deployed at the development step. The design phase is about preparing the scenarios and their educational content, the verification tools, and the overall user experience (UX), setting up the associated design layout (colors, fonts, arrangement of images and buttons etc.). During development and implementation, the selected technologies with the associated design and authoring assets are combined into an integrated software solution. The final phase, evaluation, ensures that the product achieved the desired goals. This employs the review and testing scenarios of MAthE for subsequent modifications and necessary adjustments.

More specifically, the hypotheses that are investigated during the conducted research refer to the percentage of social media users that:

RH1:

Do not have the skillsets and know-how to fully exploit tools and practices for news verification.

RH2:

Could learn things through the gaming process more efficiently.

Based on the above hypotheses, the implicated research questions that are investigated are:

RQ1:

Is the audience interested in the fight against misinformation and are they familiar with the news authentication practices?

RQ2:

Is the audience interested in a game aimed at enhancing their authentication skills, while also cultivating media literacy?

RQ3:

Will the users enjoy playing a gamification application on media authentication like MAthE?

RQ4:

Will the users learn the dedicated steps to indicate bogus content, according to the fact-checking procedures by playing MAthE?

To answer the above RQ (Research Questions) after building MAthE, we run a randomized online field study, exploiting the possibilities offered by the platform Lime survey, which is a free and open source online statistical web-app. Specifically, targeted users were asked to play MAthE before and after they had to fill out questionnaires. For this reason, we included the website of the game in our Lime Survey questionnaire. Additional feedback about the interface utility and usability was collected from students that attend the undergraduate course Human–Computer Interaction, which is taught as an optional subject at School of Journalism and Mass Communications of Aristotle University of Thessaloniki. The interface development and the results of the above evaluation processes are presented below.

3. Results

3.1. The Main Interface of MAthE the Game

The graphical user interface (GUI) was carefully designed as the main font-end, through which the user interacts with the system, navigates, draws information, and completes the purpose of the game. Since critical factors for the effectiveness of the application are the achieved comprehensibility and accessibility of the offered browsing options that shape the overall user experience design (UX), much attention was paid to create a user-friendly GUI [15]. Specifically, the main interface of the game (Figure 1) is divided into four sections. Section (A) contains the presented article. The second section (Β) is the part where the icons of the four supplementary tools are displayed (search engine, reverse image search, image verification assistant, debunking site). The third section (C) is the panel where the descriptions of the provided tools are presented. The fourth section (D) contain all the answering controls (“real” or “fake” buttons) and the associated indicators that are displayed in individual fields along with the game progress (i.e., the article scenarios and levels numbering, the score, and the elapsed time). Time recording for game completion was incorporated, for analyzing the duration of the user engagement in the articles and interaction with the provided services.

As already mentioned, the game presents real and fake articles. Since visual information is crucial during the verification process, the exact layout of the articles was utilized strictly inside the game. For the above reason, the selected fake articles contain overloaded language, excessive punctuation, a lot of capital letters, or weird colors for emphasizing purposes. The real items derived both from the regular and weird daily agenda. For example, Figure 1A contains the presented article written in Greek language. The specific example is a case retrieved from the webpage tokoulouri.com [38], a Greek satirical source, adopting similar philosophy with the well-known satirical website theonion.com [39]. The article entitled “Study: Up to 40% less likely to have astigmatism for those who writes Greeklish” is considered unreliable, since it satirizes the widely spread Greeklish phenomenon (from the words Greek and English), i.e., writing Greek words with the use of Latin alphabet [40].

The experience showed that the simplest way to validate information online is through search engines, which constitute the primary way of Internet navigation and one of the most common online activities. After reviewing the available tools in related studies [3], it was decided to equip MAthE with popular search engines (Google, Bing, DuckDuckGo, Yandex, Swisscows) (Figure 2A). Visual elements (images) are the most frequent subject of investigation. The (re)distribution history of a picture on the web can be retrieved by a reverse image search, i.e., based on the visual data and not on textual keywords. Widely known reverse image search engines are Google Image, TinEye, Bing, Yandex, and Baidu, with Google already having many pictures stored; anyone can upload an image file or paste its corresponding URL into the search bar for retrieving similar content (Figure 2B) [5]. Apparently, not all the image doctoral forgeries can be located with the above engines that were incorporated in MAthE interface. Hence, the “Image Verification Assistant” toolbox (accessed as web service at http://reveal-mklab.iti.gr/reveal/), containing online toolkits for forensic investigation of pictures, has been embedded in the game (Figure 2C) [41,42]. The last service offered by MAthE provides an answer to the following question: Has the “title” of the article already been investigated by the debunking sites? For example, if the title has already been captured/recorded by the debunking site “Ellinika Hoaxes” [43] it would probably be unreliable (Figure 2D). The bundle of the above utilities offers forensic solutions that can adequately support typical authentication tasks (especially if they are combined), so they were incorporated in the MAthE environment, as presented in Figure 2.

Το better understand the utilization of the tools provided, we might explain the presented scenarios in Figure 2. The article of Figure 2A refers to a dying man, who defeated cancer with wheatgrass (a pseudoscience article that can be debunked with the search engine help). Figure 2B presents an article about a Syrian little girl, who covers the eyes of her doll so that she does not “see” the horror. The presented image is used out of the context and it can be debunked with the aid of reverse image search. The case in Figure 2C indicates that, in France, citizens set guillotines and hangers and urges the Greek people to do the same. The presented image is manipulated and can be identified by using the image verification assistant tool. Last but not least, the article in Figure 2D uses hate speech to intimidate citizens that “they want our children to be Muslims”. The author of the article offers a horrifying cover page of newspaper as evidence but the image used is out of context. Thus, with the aid of the debunked site ellinikahoaxes, the player can easily understand that this front page is crafted and propagandistic.

The presented educational approach is considered suitable and effective since it gathers/combines all the basic existed knowledge on how to deal with the phenomenon of misinformation. Given that the main goal of the game is to transform digital educational media from a time-consuming process to a fun challenge, in this context, users are not obliged to learn by heart the authentication practices and rules, but they obtain the essential knowledge through experiential education and playful activities. In addition, they do not need to have a particularly high technological background to play the game (with the exception that tools of image forensic analysis are far more demanding, even for technology-familiar users). Overall, these combined infotainment goals are put into test, in the subsequent evaluation section.

Concerning the development of the application, the fundamental design requirements were: a) To be compatible with the majority of computing devices, making the game accessible for all users; b) to be upgradable, localizable, and able to support typical data structures for content management; c) to support integration with online survey platforms, so as to favor common and easy-to-follow evaluation procedures. Thus, it was decided to select a modular design utilizing web technologies. Specifically, the application is structurally split into two components, the back-end and the front-end. The former is responsible for data manipulation/management, while the latter provides the required user interface functionalities, as presented in Figure 3. The back-end is implemented in PHP (Hypertext Preprocessor) and MySQL technologies for programming and data management respectively, while HTML and JavaScript were used for constructing the user interface and facilitating user interaction. Much attention was paid in providing simple, well-known content management strategies. For this reason, the common comma-separated-values (CSV) file format is selected for content manipulation and graphical user interface (GUI) translatable text elements. While the GUI is not fully optimized for smartphones yet, a responsive design approach is under development, along with the appropriate modifications, as the outcome of the current evaluation phase. Finally, the complete source code of the game can be retrieved from GitHub (http://tiny.cc/wtyd8y).

3.2. The Online Playtest Survey

The game development step was followed by the necessary assessment procedure. As mentioned above, a dedicated questionnaire was formulated, targeted to collect evaluation answers of potential users. For the creation of the survey, data from various sources were used, the largest part of which is derived from “The game experience questionnaire”, originating from Technische Universiteit Eindhoven [44] and from the MEEGA+, an evolution of a model for the evaluation of educational games [45]. Moreover, content- and topic-related queries were incorporated, associating the perceived experience with the learning targets, to validate the stated research hypotheses (i.e., to educate and support audience in the battle against misinformation) [3]. The selected investigation items were tested and validated from an experts’ group, who also served as beta testers of MAthE the Game: (a) An art director in a UX design agency; (b) a researcher, specializing in software and game development; (c) a researcher in the field of media veracity and authentication practices; (d) a Post-doc researcher on audiovisual arts, with expertise on digital games and multimodal/auditory interfaces; and (e) a communication professional working as data analyst with multivariate experience in conducting public opinion surveys. This board of specialists provided useful feedback regarding the structure and the content of the initially formed questionnaire to its final version (presented in

Table 1. Experience questions and associated metrics for game playing.

#	Description	Mean	Std
EM1	The contents and structure helped me to become confident that I would learn with this game.	3.5545	0.8695
EM2	This game is appropriately challenging for me.	3.3636	1.0156
EM3	Appropriate and in right time transitions	3.8182	0.7408
EM4	The game does not become monotonous as it progresses (repetitive or boring tasks).	3.7091	0.8979
EM5	It is due to my personal effort that I managed to advance in the game.	3.6545	0.8027
EM6	I feel satisfied with the things that I learned from the game.	3.5636	0.8689
EM7	I would recommend this game to my colleagues.	3.8182	0.8861
EM8	I was able to interact with other players during the game.	3.7364	0.9879
EM9	The game promotes cooperation and/or competition among the players.	3.0545	1.0941
EM10	I had fun with the game.	3.6545	0.9764
EM11	Something happened during the game (game elements, competition, etc.) which made me smile.	3.5	1.051
EM12	There was something interesting at the beginning of the game that captured my attention.	3.3909	0.8749
EM13	I was so involved in my gaming task that I lost track of time.	2.5091	1.0595
EM14	I forgot about my immediate surroundings while playing this game.	2.6091	1.054
EM15	The game contents are relevant to my interests.	3.3636	0.9217
EM16	It is clear to me how the contents of the game are related to misinformation phenomenon	4.1636	0.7327
EM17	I prefer learning with this game to learning through other ways (e.g., other teaching methods).	3.5364	0.9599

and

Table 2. Perceived experience questions and associated metrics, within the learning targets.

#	Description	Mean	Std
PM1	The game tries to address an important contemporary problem, misinformation	4.2727	0.6311
PM2	The game has social and educational character	4.2455	0.5590
PM3	The game contributed to my training on verification of news content.	3.7364	0.8166
PM4	The game allows effective learning in other activities get involved.	3.4545	0.9109
PM5	The game enhances my digital literacy	3.6636	0.8009
PM6	The game help me to learn about the verification tools	3.8182	0.8963
PM7	The game help me to learn how to use verification tools	3.3727	0.9803
PM8	The game helped me to understand when I should use each verification tool to confirm if a news item is real or fake	3.4273	0.9387
PM9	The game helped me to learn the must checked elements of an article	3.6273	0.8619
PM10	The game helped me to understand the “patterns” of false news	3.6818	0.7738
PM11	The game helped me to identify fake images	3.3909	0.8644
PM12	The game helped me to identify the signs of manipulation at images.	3.2091	0.8954
PM13	The game helped me to become familiar with them characteristics of fake articles	3.6818	0.7620
PM14	The game will help me to recognize fake articles more easily when I meet them.	3.5636	0.7450
PM15	The game helped me have a better view of the phenomenon of misinformation	3.6818	0.8413

).

A total of 111 subjects completed the study online. Participants were asked to answer some initial questions, then to play the game, and thereafter, to respond to an additional set of questions regarding their impressions in respect to fun, excitement, and clarity of objectives. The first goal was to reveal useful information regarding their online habits. Participants were asked if they hold an account on social media (yes = 106, no = 5), how many hours they use their accounts actively (<1 h = 22, 1–3 h = 37, 3–5 h = 23, 5–7 h = 14, >7 h = 10), how often they share news on their profile (every day = 7, every 2–3 days = 10, once a week = 13, 2–3 a month = 16, once a month = 18, 1–2 a year = 26, never = 16), and how often they post their opinion/comment on the daily agenda (every day = 6, every 2–3 days = 6, once a week = 12, 2–3 a month = 9, once a month=12, 1–2 a year = 28, never = 33).

The next subset of questions investigates users’ interest in news reporting (yes = 97, no = 14) and their previous experience with the dissemination of fake news on social media. They were asked whether they can identify a fake story (yes = 76, no = 35) and if they have ever realized that an article they posted was fake (yes = 24, no = 87). From the above, 24 people admitted that they have posted at least one fake story, 16 argued that they deleted the post, while only 6 of them made a corrective action, such as posting the debunked article.

Furthermore, the participants’ familiarity in misinformation issues was also assessed via binary variables (yes/no), along with previous experience on verification tools/practices and gamification approaches. In general, the participants were aware of the “fake news” matter and its consequences (yes = 101, no = 10), being informed for the existence of debunking websites that cross-validate articles (yes = 84, no = 27). However, their knowledge regarding specialized applications in the authentication of multimedia content (audio, video, image) is limited (yes = 57, no = 54). Moreover, only 18 of the participants (out of 111) are aware of the existence of games that aim to upgrade our critical ability to recognize the true nature of information (yes = 18, no = 93). An interesting finding regarding the main subject of the research (i.e., development and evaluation of a serious “fake news” game) is that moving to the core of our combined target (verification literacy through gamification), the familiarity score collapses, as presented in Figure 4.

After the interaction with the game, we measured the extent to which they liked the game as a playing experience on a 5-point Likert scale (1 = strongly disagree to 5 = strongly agree) with a set of 17 questions concerning how easy and fun was the playing experience, as presented in Table 1. Furthermore, this evaluation subset was correlated to basic demographic variables of the sample and, specifically, gender (female = 63, male = 48), Age (<18 = 0, 18–28 = 41, 29–39 = 48, 40–50 = 11, <50 = 11), education (PhD holder = 10, Master holder = 41, graduates = 46, high school = 13), and profession (self-employed = 15, employee = 56, student = 29, Unemployed = 10). Because of the categorical nature of the input data, during correlation analysis, Chi-Square tests were utilized with a confidence level of 95%, therefore comparing the related Pearson coefficient to the threshold of 0.05. An interesting statistical difference was spotted in the results, i.e., in the answers of male/female users for items EM1 (p-value = 0.0031), EM3 (p-value = 0.0001), EM4 (p-value = 0.0004), EM5 (p-value = 0.0011), EM6 (p-value = 0.0029), EM7 (p-value = 0.0001), EM8 (p-value = 0.0001), EM10 (p-value = 0.0003), EM11 (p-value = 0.0041), EM12 (p-value = 0.0339), EM15 (p-value = 0.0337), EM16 (p-value = 0.0001), and EM17 (p-value = 0.0031). Moreover, education is corelated to answers EM13 (p-value = 0.0001) and EM14 (p-value = 0.0006). The same tests were repeated for another subset of 15 questions, shown in Table 2, evaluating effectiveness within the learning experience (1 = strongly disagree to 5 = strongly agree). In the results, only age had statistical correlation with this evaluation metrics and particularly with items PM1 (p-value = 0.0001), PM2 (p-value = 0.0001), PM3 (p-value = 0.0002), PM4 (p-value = 0.0083), PM5 (p-value = 0.0006), PM6 (p-value=0.0001), PM7 (p-value = 0.0134), PM8 (p-value = 0.0073), PM9 (p-value = 0.0006), PM10 (p-value=0.0003), PM11 (p-value=0.0187), PM13 (p-value = 0.0004), PM14 (p-value = 0.0036), and PM15 (p-value = 0.0003).

3.3. Evaluation from Students

According to Whitney Quesenbery [46], the definition of usability often ends in “ease of use”, resulting in an oversimplification of the problem and providing little guidance to the designer of the user interface. A more precise definition can be used to understand the user’s requirements in order to formulate usability objectives and decide on the best techniques for assessing it. The same refers to the need for understanding the so-called “5 Es of usability” [47]:

• Effectiveness: To what extent has it been completed and how accurately the work or experience (or whether the goal of the job) is achieved.
• Efficiency: How fast the job you have set can be completed.
• Engagement: How well the interface leads the user to interact with the system and how enjoyable it is as a process.
• Error tolerance: How well the product predicts and protects against errors, and how it can help the user to recover from a possible error.
• Easy of learning: How well the product supports both initial orientation and lifelong learning throughout its life.

To access and assess the above information, we conducted an analysis of the game environment into a related media course, focusing on the specific expertise of human–computer interaction. The students had to play the game and make suggestions regarding its utility and usability. The next step was to propose improvements through the creation of prototype screens (mock-up design). The UXPin application was used (https://www.uxpin.com/), since it is an online collaborative platform, facilitating the design of high-fidelity interactive prototypes.

In general, the interface was considered functional by the students, with a simple structure without unnecessary and intricate graphics elements that disorient the player during interaction. Moreover, its minimalistic design makes it simple for the user to navigate in the page, i.e., easily and quickly learn the offered browsing routes, thus, without requiring a high memory capacity. The functionality of the available tools was also judged as visible and understandable, features that were attributed to the remaining indications as well (level, time, score). The most frequent negative review was related to the layout of the answer buttons, which had the same blue color. The suggestion from almost all the workgroups was to change the colors to green and red, respectively, for the true and false, so as to become better distinguishable from the players (Figure 5 and Figure 6). Some teams proposed broader rearrangement of the MAthE panel and its components, resulting in more holistic GUI (and back-end) refactoring, as presented in Figure 6.

Among others, some subjects would prefer to interact with other players during the game, favoring collaboration and competition between players. Another personalization-driven suggestion is related to the option of creating a profile, as presented in Figure 7. In general, considering that MathE is not a conventional/mainstream game, i.e., is different from games played for entirely entertaining purposes, the criticism made by the media students is considered extremely motivating, matching the already predefined steps for the development of an improved game version. Based on the MathE evaluation experience, it is verified that gamification could enhance learning and engage learners in a more social and context-rich decision making for problem solving in educating tasks, as someone might find in related bibliography [48].

4. Discussion and Conclusions

The present study deals with gamification assessment, focusing on the news verification topic. In this context, it is attempted to unveil useful information regarding the playing experience and its educatory impact. Overall, MAthE, the created game, was positively evaluated, justifying the innovation of the approach, its simple design, and the interesting themes/content of the implicated scenarios. While measuring the achieved goals of the whole project, it became clear (88.2% of the respondents replied agree/totally agree) that the game is related to the field of fact-checking, trying to address an important contemporary problem, the spread of false news (91.9%). The main conclusion (RQ1) is that users usually do neglect to check information and are not very familiar with contemporary authentication practices; nevertheless, they are still interested in enhancing their verification skills. Particularly, they are interested in obtaining knowledge and knowhow to detect the characteristics of false news (69.1%), thus, to be able to recognize fake news (57.3%). Furthermore, they would prefer to grow awareness and expertise through a gamification experience compared to other learning approaches, aiming at cultivating related media literacy (RQ2). The above statement is confirmed by the pleasure that they felt after playing the game (yes = 102, no = 9) (RQ3). Recalling RQ4 and the estimated educational effectiveness, more than 95.5% of the participants understand that MAthE has a social and educational character.

Additional interesting results on how gaming systems impact on people’s behavior can be extracted by analyzing the answers of the online survey. Specifically, the participants agreed that the game contributes to their news-verification training (66.4%), enhances their digital literacy (62.7%), teaches them about the existence of verification tools (70.9%), and helps them to learn how to use such tools (50.9%). Moreover, to a much lower extent, they answered that the game helped them to spot fake images (52.8%) and to understand the trails of visual tampering (39.1%). The above results prove that further effort is required towards image-centered training on the recognition of doctored photos (and multimedia content in general).

Summing up, the current study is based on the motivational model, confirming that serious games do form a key tool for influencing cognitive evaluation [48]. Indeed, the findings showed that different gamification approaches could induce different decisions. This can be grounded on the fact that players (66.3%) are willing to alter their behavior in a more critical attitude, concerning online information evaluation and judgement. In conclusion, the vast majority of the participants stated that MAthE helped them to get a better picture of the misinformation situation and that will try to be more careful in the future, which was the ultimate goal put into test.