Designing and Evaluating a Serious Game for Learning Artificial Intelligence Algorithms: SpAI War as a Case Study
Abstract
:1. Introduction
How do undergraduates and graduates of Informatics evaluate the player experience, scenario, and perceived short-term learning of AI algorithms in the first-person serious game SpAI War?
2. Related Work
2.1. Difficulties in Learning AI Algorithms
2.2. Methods of Teaching AI Algorithms
2.3. The Role of Serious Games in Teaching Algorithms
3. Design of SpAI War
3.1. Game Design
- Clear goals: At the beginning of each level, the player is presented with the mission that he/she has to complete with clear steps, depictions and photos of the spaces he/she will encounter (Figure 4). In this way, the player is fully focused on learning each algorithm and is not burdened with extra effort to understand what should follow.
- Uncertain outcomes: The game ensures the avoidance of any possible sense of monotony when using it, as almost every level includes different challenges with different approaches to applying the algorithms. In addition, during the crossing of the levels, there are several, unpredictable and unexpected waves of enemies of increasing difficulty, and the player is called upon to repel them. Their existence aims at the constant vigilance of the player but also at the stimulation of his/her interest.
- Raising the player’s self-esteem: The game tries to reward the player and give him/her confidence in fulfilling the goal of the scenario. This is achieved by ensuring that at each level, he/she constantly learns at least one new, simple and understandable piece of information about the final goal which at higher levels are combined to reach a final conclusion regarding the main purpose. In addition, within the levels, when the player makes a right choice, a reward message appears urging the player to continue to the next challenge with more confidence.
3.2. Pedagogy
3.3. Learning Content Modeling
4. Materials and Methods
4.1. Context of the Study
4.2. Research Question
How do undergraduates and graduates of Informatics evaluate the player experience, scenario, and perceived short-term learning of AI algorithms in the first-person serious game SpAI War?
4.3. Participants
4.4. Data Collection Instruments and Analysis
5. Results
5.1. Perceived Knowledge and Difficulties on Algorithms and Programming
5.2. Game-Play Habits and In-Game Performance
5.3. Perceived Player Experience
5.4. Game Scenario
5.5. Perceived Short-Term Learning
6. Discussion
7. Limitations and Future Work
8. Conclusions
Author Contributions
Funding
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Abt, C. Serious Games; Viking Press: New York, NY, USA, 1970. [Google Scholar]
- Michael, D.R.; Chen, S.L. Serious Games: Games That Educate, Train, and Inform; Thomson Course Technology: Boston, MA, USA, 2006. [Google Scholar]
- Barab, S.A.; Scott, B.; Siyahhan, S.; Goldstone, R.; Ingram-Goble, A. Transformational Play as a Curricular Scaffold: Using Video Games to Support Science Education. J. Educ. Psychol. 2009, 101, 291–310. [Google Scholar]
- Zhonggen, Y. A Meta-Analysis of Use of Serious Games in Education over a Decade. Int. J. Comput. Games Technol. 2019, 2019, 4797032. [Google Scholar] [CrossRef]
- Giannakoulas, A.; Xinogalos, S. A Review of Educational Games for Teaching Programming to Primary School Students. In Handbook of Research on Tools for Teaching Computational Thinking in P-12 Education; Kalogiannakis, M., Papadakis, S., Eds.; IGI Global: Hershey, PA, USA, 2020; pp. 1–30. [Google Scholar]
- Kroustalli, C.; Xinogalos, S. Studying the effects of teaching programming to lower secondary school students with a serious game: A case study with Python and CodeCombat. Educ. Inf. Technol. 2021, 26, 6069–6095. [Google Scholar] [CrossRef]
- Galgouranas, S.; Xinogalos, S. jAVANT-GARDE: A Cross-Platform Serious Game for an Introduction to Programming with Java. Simul. Gaming 2018, 49, 751–767. [Google Scholar] [CrossRef]
- Malliarakis, C.; Satratzemi, M.; Xinogalos, S. CMX: The Effects of an Educational MMORPG on Learning and Teaching Computer Programming. IEEE Trans. Learn. Technol. 2017, 10, 219–235. [Google Scholar] [CrossRef]
- Xinogalos, S.; Satratzemi, M. The Use of Educational Games in Programming Assignments: SQL Island as a Case Study. Appl. Sci. 2022, 12, 6563. [Google Scholar] [CrossRef]
- Xinogalos, S.; Eleftheriadis, S. Office Madness: Investigating the impact of a game using a real life job and programming scenario on player experience and perceived short-term learning. Entertain. Comput. 2023, 44, 100521. [Google Scholar] [CrossRef]
- Van Eck, R. Digital game-based learning: It’s not just the digital natives who are restless. EDUCAUSE Rev. 2006, 41, 16–30. [Google Scholar]
- Bellotti, F.; Ott, M.; Arnab, S.; Berta, R.; de Freitas, S.; Kiili, K.; De Gloria, A. Designing serious games for education: From pedagogical principles to game mechanisms. In Proceedings of the 5th European Conference on Games Based Learning, Athens, Greece, 20–21 October 2011; University of Athens: Athens, Greece, 2011; pp. 26–34. [Google Scholar]
- De Gloria, A.; Bellotti, F.; Berta, R. Serious Games for education and training. Int. J. Serious Games 2014, 1. [Google Scholar] [CrossRef]
- Silva, F.G.M. Practical Methodology for the Design of Educational Serious Games. Information 2020, 11, 14. [Google Scholar] [CrossRef]
- Natucci, G.C.; Borges, M.A. Balancing Pedagogy, Emotions and Game Design in Serious Game Development. In Proceedings of the Anais Estendidos Do XX Simpósio Brasileiro de Jogos e Entretenimento Digital, Gramado, Brazil, 18–21 October 2021; SBC: Porto Alegre, Brazil, 2021; pp. 919–922. [Google Scholar]
- Brisson, A.; Pereira, G.; Prada, R.; Paiva, A.; Louchart, S.; Suttie, N.; Lim, T.; Lopes, R.; Bidarra, R.; Bellotti, F.; et al. Artificial Intelligence and Personalization Opportunities for Serious Games. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Palo Alto, CA, USA, 8–12 October 2012; Volume 8, pp. 51–57. [Google Scholar] [CrossRef]
- Ibrahim, R.; Jaafar, A. Educational games (EG) design framework: Combination of game design, pedagogy and content modeling. In Proceedings of the International Conference on Electrical Engineering and Informatics, Bangi, Malaysia, 5–7 August 2009; Volume 1, pp. 293–298. [Google Scholar] [CrossRef]
- Petri, G.; von Wangenheim, C.; Borgatto, A. MEEGA+: An Evolution of a Model for the Evaluation of Educational Games; Technical Report; Brazilian Institute for Digital Convergence: Florianópolis, Brazil, 2016; pp. 1–40. [Google Scholar]
- Alpaydin, E. Introduction to Machine Learning; MIT Press: Cambridge, MA, USA, 2020. [Google Scholar]
- Shute, V.J.; Ventura, M.; Bauer, M. Melding the power of serious games and embedded assessment to monitor and foster learning: Flow and grow. In Serious Games; Routledge: Abingdon, UK, 2009; pp. 317–343. [Google Scholar]
- Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning; MIT Press: Cambridge, MA, USA, 2016. [Google Scholar]
- Lauscher, A. Life 3.0: Being human in the age of artificial intelligence. Internet Hist. 2019, 3, 101–103. [Google Scholar] [CrossRef]
- Passarelli, M.; Dagnino, F.M.; Earp, J.; Manganello, F.; Persico, D.; Pozzi, F.; Bailey, C.J.; Perrotta, C.; Buijtenweg, T.; Haggis, M. Educational games as a motivational tool: Considerations on their potential and limitations. In Proceedings of the CSEDU 2019—11th International Conference on Computer Supported Education, Heraklion, Greece, 2–4 May 2019; Scitepress: Setúbal, Portugal, 2019; pp. 330–337. [Google Scholar]
- Greipl, S.; Moeller, K.; Ninaus, M. Potential and limits of game-based learning. Technol. Enhanc. Learn. 2020, 12, 363–389. [Google Scholar] [CrossRef]
- Wassila, D.; Tahar, B. Using serious game to simplify algorithm learning. In Proceedings of the International Conference on Education and e-Learning Innovations, Sousse, Tunisia, 1–3 July 2012; IEEE: Piscataway, NJ, USA, 2012; pp. 1–5. [Google Scholar]
- Hainey, T.; Baxter, G.; Stanton, A. A Serious Game to Teach Rudimentary Programming: Investigating content integration. In Proceedings of the ECGBL 2019 13th European Conference on Game-Based Learning, Odense, Denmark, 3–4 October 2019; Academic Conferences and Publishing Limited: Reading, UK, 2019; p. 298. [Google Scholar]
- Kokotajlo, T.C. ‘Obsolescence’: Evaluating an Educational Serious Game on Artificial Intelligence Impacts to Military Strategic Goals; Air Force Institute of Technology: Wright-Patterson Afb, OH, USA, 2022. [Google Scholar]
- Geleta, M.; Xu, J.; Loya, M.; Wang, J.; Singh, S.; Li, Z.; Gago-Masague, S. Design Factors of Maestro: A Serious Game for Robust AI Education. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education, Toronto, ON, Canada, 15–18 March 2023; Volume 2, p. 1318. [Google Scholar]
- Li, Z.; O’Brien, L.; Flint, S.; Sankaranarayana, R. Object-oriented Sokoban solver: A serious game project for OOAD and AI education. In Proceedings of the 2014 IEEE Frontiers in Education Conference (FIE), Madrid, Spain, 22–25 October 2014; IEEE: Piscataway, NJ, USA, 2014; pp. 1–4. [Google Scholar] [CrossRef]
- Petri, G.; von Wangenheim, C.G. MEEGA+: A Method for the Evaluation of the Quality of Games for Computing Education. In Proceedings of the SBGames, Rio de Janeiro, Brazil, 28–31 October 2019; pp. 28–31. [Google Scholar]
- Malliarakis, C.; Satratzemi, M.; Xinogalos, S. Designing educational games for computer programming: A holistic framework. Electron. J. e-Learn. 2014, 12, 181–281. [Google Scholar]
- Salen, K.; Zimmerman, E. Rules of Play: Game Design Fundamentals; The MIT Press: Cambridge, MA, USA, 2004. [Google Scholar]
- Kiili, K. Content creation challenges and flow experience in educational games: The IT-Emperor case. Internet High. Educ. 2005, 8, 183–198. [Google Scholar] [CrossRef]
Game Design | Obsolescence | Maestro | Sokoban Solver | SpAI War |
---|---|---|---|---|
Game type | Digital board game (competing militaries fight for gaining Influence Points) | Attack and defense simulation environment | Puzzle game | First-Person Shooter |
Singe/Multiplayer | Single player | Multiplayer | Single player | Single player |
Available actions | Move forces, build forces, interact with technology cards (AI-based technologies) | Provide a solution based on the player’s role (attacker/defender) | Navigate the maze, move boxes | Player navigation, Pick up a weapon, Use weapon, Reload weapon, Open–Close doors, Interact with messages |
Time pressure | In-game turn timer | No time pressure | No time pressure | No time pressure |
Feedback | Post-game scores | Scores/ranks | Feedback is provided for every wrong or correct choice | |
AI concepts/algorithms | AI impacts in military operations and strategies | Robust AI | Basic search algorithms; heuristics | Nine AI algorithms; Introduction to Machine Learning |
Question | Mean | Sd | Median |
---|---|---|---|
How difficult is it for you to comprehend algorithms in general (e.g., sorting/searching and graph theory algorithms)? * | 2.4 | 0.85 | 2 |
How difficult is it for you to comprehend AI algorithms? * | 2.8 | 0.87 | 3 |
How would you describe the level of your programming knowledge? ** | 3.2 | 0.98 | 3 |
How would you describe your interest in programming? ** | 4 | 1.2 | 4.5 |
Dimension | Subdimension | Statement | Median |
---|---|---|---|
Usability | Aesthetics | The game design is attractive (interface, graphics, etc.). | 4 |
The text font and colors are well blended and consistent. | 4 | ||
Learnability | I needed to learn a few things before I could play the game. | 2 | |
Learning to play this game was easy for me. | 5 | ||
I think that most people would learn to play this game very quickly. | 5 | ||
Operability | I think that the game is easy to play. | 5 | |
The game rules are clear and easy to understand. | 5 | ||
Accessibility | The fonts (size and style) used in the game are easy to read. | 5 | |
The colors used in the game are meaningful. | 5 | ||
Error prevention and recovery | The game prevents me from making mistakes. | 4 | |
When I make a mistake, it is easy to recover from it quickly. | 4 | ||
Confidence | When I first looked at the game, I had the impression that it would be easy for me. | 4 | |
The contents and structure helped me to become confident that I would learn with this game. | 5 | ||
Challenge | This game is appropriately challenging for me. | 4 | |
The game provides new challenges (offers new obstacles. situations or variations) at an appropriate pace. | 4 | ||
The game does not become monotonous as it progresses (repetitive or boring tasks). | 4 | ||
Satisfaction | Completing the game tasks gave me a satisfying feeling of accomplishment. | 5 | |
It is due to my personal effort that I managed to advance in the game. | 4 | ||
I feel satisfied with the things that I learned from the game. | 5 | ||
I would recommend this game to my colleagues. | 5 | ||
Fun | I had fun with the game. | 4 | |
Something happened during the game (game elements, competition, etc.) which made me smile. | 4 | ||
Focused Attention | There was something interesting at the beginning of the game that captured my attention. | 4 | |
I was so involved in my gaming task that I lost track of time. | 3 | ||
I forgot about my immediate surroundings while playing this game. | 4 | ||
Relevance | The game contents are relevant to my interests. | 4 | |
It is clear to me how the contents of the game are related to the course. | 5 | ||
This game is an adequate teaching method for this course. | 5 | ||
I prefer learning with this game to learning through other ways (e.g., other teaching methods). | 5 |
Statement | Median |
---|---|
The scenario of the game raised my interest. | 4 |
The elements of the scenario (i.e., attacks, zombies) give a player experience analogous to that of entertainment game. | 4.5 |
The way of integrating the educational material in the game is completely attuned to the scenario. | 4 |
The scenario and the graphics are appropriate for the age of the players that the game is targeted to (students or generally adults interested in the field of AI). | 5 |
Statement | Median |
---|---|
I learned something new with this game. | 4 |
The game helped me comprehend the basic AI algorithms presented in its context. | 5 |
The activities of the game helped me comprehend my errors and misconceptions in relation to the AI algorithms. | 5 |
During game play, I devoted time to reading the information provided for the algorithms. | 5 |
The game allowed for efficient learning compared with other activities in the course (such as a tutorial or textbook). | 5 |
The knowledge of AI acquired through the game can motivate someone to study further the subject. | 5 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Barmpakas, A.; Xinogalos, S. Designing and Evaluating a Serious Game for Learning Artificial Intelligence Algorithms: SpAI War as a Case Study. Appl. Sci. 2023, 13, 5828. https://doi.org/10.3390/app13105828
Barmpakas A, Xinogalos S. Designing and Evaluating a Serious Game for Learning Artificial Intelligence Algorithms: SpAI War as a Case Study. Applied Sciences. 2023; 13(10):5828. https://doi.org/10.3390/app13105828
Chicago/Turabian StyleBarmpakas, Anastasios, and Stelios Xinogalos. 2023. "Designing and Evaluating a Serious Game for Learning Artificial Intelligence Algorithms: SpAI War as a Case Study" Applied Sciences 13, no. 10: 5828. https://doi.org/10.3390/app13105828
APA StyleBarmpakas, A., & Xinogalos, S. (2023). Designing and Evaluating a Serious Game for Learning Artificial Intelligence Algorithms: SpAI War as a Case Study. Applied Sciences, 13(10), 5828. https://doi.org/10.3390/app13105828