Usage of Gamification Techniques in Software Engineering Education and Training: A Systematic Review

Abstract

Gamification, the integration of game design elements into non-game contexts, has gained prominence in the software engineering education and training realm. By incorporating elements such as points, badges, quests, and challenges, gamification aims to motivate and engage learners, potentially transforming traditional educational methods. This paper addresses the gap in systematic evaluations of gamification’s effectiveness in software engineering education and training by conducting a comprehensive literature review of 68 primary studies. This review explores the advantages of gamification, including active learning, individualized pacing, and enhanced collaboration, as well as the psychological drawbacks such as increased stress and responsibility for students. Despite the promising results, this study highlights that gamification should be considered a supplementary tool rather than a replacement for traditional teaching methods. Our findings reveal significant interest in integrating gamification in educational settings, driven by the growing need for digital content to improve learning.

1. Introduction

Gamification, the integration of game design elements into non-game contexts, has emerged as a significant approach in various domains, including education and training. It employs elements such as points, badges, quests, and challenges to motivate and engage users, thereby enhancing their experience and performance. Its application ranges from universities to industries.

In the realm of software engineering education and training (SEET), gamification has the potential to transform traditional learning methods by making them more interactive and engaging [1]. SEET encompasses the instructional methodologies, curricular designs, and practical experiences aimed at equipping students and professionals with the knowledge, skills, and competencies required in the field of software engineering. It includes both formal academic programs, such as university degrees and professional certifications, and informal learning opportunities, such as workshops, online courses, and bootcamps. The objective of SEET is to prepare individuals to effectively design, develop, test, and maintain software systems, ensuring they meet user needs and adhere to quality standards. SEET traditionally focuses on imparting technical skills and knowledge necessary for developing software systems. However, the conventional teaching methods often struggle to maintain student engagement and motivation [2]. By incorporating gamification, educators can create a more dynamic and stimulating learning environment, which can lead to improved learning outcomes. Gamification not only makes learning more enjoyable but also fosters a deeper understanding of complex concepts through active participation and immediate feedback [3].

In the context of professional training, gamification can play a crucial role in continuous learning and skill development. As the software industry evolves rapidly, professionals need to constantly update their skills and knowledge. Gamified training programs can offer a more compelling and effective way to achieve this, promoting sustained engagement and continuous professional development [4]. Moreover, gamification can increase the awareness of employees in critical scenarios [5] by letting them take action in less time to avoid dangerous situations [6].

Despite the promising potential of gamification, there is a need for a systematic evaluation of its application in the SEET topic. This involves examining both the result improvements and the technical challenges associated with implementing gamified systems. To address this gap, we conduct a systematic literature review to evaluate the maturity and impact of gamification in SEET. We propose six research questions aimed at exploring the effectiveness of gamification in enhancing learning and training outcomes in software engineering. This review is based on an extensive analysis of 68 primary studies, identified and filtered through the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology [7]. We assess the evolution of this topic over time, the application areas, and the results obtained after the implementation of gamification techniques in university courses. Additionally, we discuss future research directions and practical implications for educators and trainers in the software engineering domain.

This paper is structured as follows: Section 2 highlights the main points of the chosen methodology, Section 3 analyzes the background on gamification and SEET topics, Section 4 summarizes results, and Section 5 shows the current literature and the differences between other reviews and the one proposed in this work. Section 6 concludes the paper.

2. Research Methodology

Our research adheres to the guidelines for a Systematic Literature Review (SLR), as described in [8]. This study aims to explore the application of gamification in SEET, covering the literature published from 2015 to 2023, using the PRISMA methodology [7] and useful tools for paper gathering, such as Zotero https://www.zotero.org/ (accessed on 1 June 2024), and for tagging and data extraction, such as Python and Microsoft Excel.

2.1. PRISMA Methodology

This section outlines the systematic review methodology employed in this study, following the PRISMA guidelines. This approach ensures a rigorous and transparent review process, allowing for comprehensive identification, selection, and analysis of relevant studies.

The review is guided by some research questions (RQs) formulated to focus the scope of this study. These RQs are designed to capture the essence of the investigated topic and to guide the systematic review process.

The Population, Intervention, Comparison, Outcome (PICO) framework is used to refine the RQs and set clear criteria for study selection:

• Population: the group or individuals targeted by the intervention.
• Intervention: the specific intervention or exposure being investigated.
• Comparison: the control or comparison group, if applicable.
• Outcome: the outcomes or effects measured in this study.

A search strategy is developed to identify all relevant literature. The search string is constructed using keywords and phrases pertinent to the research question and is applied across multiple databases to ensure thorough coverage. Searches are conducted in major academic databases.

To ensure the selection of relevant and high-quality studies, specific inclusion and exclusion criteria are established. They are needed to ensure that filtered papers are relevant to the scope of the study.

The extracted data, based on defined RQs, are then synthesized to provide a comprehensive overview of the current state of research on the topic, identify trends, and highlight gaps in the literature.

The process for identifying relevant papers, based on the PRISMA guidelines, is illustrated in Figure 1, generated using the tool explained in [9].

Figure 1. PRISMA search methodology.

2.2. Research Questions Definition

The objectives of this study are twofold: (a) to identify the current state of gamification in SEET, and (b) to provide a foundation for highlighting gaps and trends in this field, as well as suggesting future research directions. To achieve these goals, we formulate the following research questions (RQs):

• RQ1: What is the publication trend in the area of gamification applied to SEET?
  This question investigates the trend in publication quantity and the structure of publication venues, which are useful for understanding the progression of this topic.
• RQ2: In which areas of software engineering is gamification used?
  This question aims to identify the key areas of study and their contributions to the scientific community.
• RQ3: What are the analyzed application areas?
  This question explores the benefits of using gamification, considering its impact on learner engagement and performance.
• RQ4: What contribution does gamification offer when it is applied to SEET?
  This question examines the specific contribution and integration of gamification into educational practices.
• RQ5: On which continents is gamification mostly analyzed?
  This question aims to identify the continents that are most interested into gamification applied to SEET.
• RQ6: What are the advantages and disadvantages of gamification when applied to SEET?
  This question seeks to understand the the pros and cons to evaluate the success of gamification in educational settings.

2.3. Paper Selection

Relevant databases were systematically searched to ensure comprehensive coverage of the literature. A PICO approach was utilized to define the search keywords, grouped into two main categories:

• Population-related search terms: “Software Engineering education”, “training”.
• Intervention-related search terms: “gamification”, “game-based learning”.

The search string used was “gamification AND (Software Engineering OR programming) AND (education OR training OR teaching OR learning)”.

This search returned a total of 1756 results. The following databases were searched: ACM Digital Library (only Open Access content), IEEE Xplore, ScienceDirect, Scopus, SpringerLink, Wiley Online Library (only Open Access content), and Google Scholar (only Open Access content). This volume of results is considered appropriate for the scope of the review.

2.4. Inclusion and Exclusion Criteria

To enhance the reliability and relevance of the studies included in our review, specific inclusion and exclusion criteria were applied.

Inclusion criteria:

• Studies published between 2015 and 2023.
• Studies written in English.
• Studies published in peer-reviewed journals or conference proceedings.
• Studies focused on the application of gamification in software engineering education or training.
• Studies that present empirical evidence or substantial theoretical contributions.

Exclusion criteria:

• Studies for which the full text is not available (e.g., article not available online or DOI not found or not readable without subscriptions): 364 studies.
• Secondary or tertiary studies (e.g., reviews or surveys): 175 studies.
• Studies where gamification is not the main focus but is only mentioned: 284 studies.

By adhering to these criteria, we ensured that the selected studies were pertinent and of high quality, thereby providing a solid foundation for our systematic review.

A pool of 68 studies is included in the analysis after the application of the screening procedure. A replication package is available at https://github.com/Mackerkun/Usage-of-Gamification-techniques-in-Software-Engineering-Education-and-Training-A-Systematic-Review (accessed on 3 July 2024).

3. Background

This section introduces gamification by giving a common definition gathered from the analyzed papers. Moreover, it explores the main components of gamification and its role in SEET.

A brief definition of gamification can be given, based on different analyzed papers [10,11,12,13,14]: gamification is an approach characterized by the application of game design elements and principles in non-game contexts to enhance user engagement and motivation. Unlike traditional educational methods, which often rely on passive learning, gamification leverages the interactive and stimulating nature of games to create more dynamic learning experiences [15].

In the domain of SEET, gamification has shown significant potential. Traditional software engineering education typically involves theoretical learning and practical exercises designed to build technical skills. However, maintaining student motivation and engagement can be challenging with conventional teaching methods. By incorporating gamification, educators aim to make learning more interactive and enjoyable, thereby increasing student participation [1] and improving learning outcomes [16].

The core components of gamification include the integration of game mechanics, dynamics, and aesthetics [17].

• Game mechanics refer to the rules and feedback systems that drive gameplay, such as scoring, levels, and rewards.
• Game dynamics involve the emotions and behaviors induced by the mechanics, such as competition, collaboration, and achievement.
• Aesthetics pertain to the overall look and feel of the gamified experience, which can enhance its appeal and immersion.

Application of Gamification in Software Engineering Education

Figure 2 illustrates a typical architecture for gamified educational platforms. This architecture includes components such as the learning management system, game engine, and user interface [18]. The learning management system handles educational content and tracks student progress, while the game engine manages game mechanics and dynamics. The user interface presents the gamified experience to students, providing them with interactive and engaging learning activities, typically accessible as web apps [19], created using some frameworks (e.g., Angular, ReactJS).

Figure 2. A common architecture for gamified educational platforms.

The unique characteristics of gamification make it well-suited for SEET. Gamification can transform traditional learning environments by promoting active participation, immediate feedback, and a sense of progression. These features help address common challenges in education, such as student disengagement and the difficulty of maintaining sustained interest over time [20].

Gamification fosters an interactive learning environment where students can engage in problem-solving activities that mirror real-world software engineering tasks. This practical application of knowledge helps reinforce learning and develops critical thinking skills. For example, gamified platforms might simulate coding challenges or project management tasks, allowing students to apply theoretical concepts in a controlled, game-like setting.

Moreover, gamification can enhance collaborative learning. By incorporating team-based challenges and competitive elements, students are encouraged to work together, share knowledge, and develop essential soft skills such as communication, teamwork, and leadership [21]. This collaborative aspect is crucial in software engineering, where teamwork and communication are key to successful project completion.

Despite its benefits, the application of gamification in SEET also presents several challenges. Designing effective gamified systems requires a deep understanding of both game design and educational pedagogy. There is a need to balance game elements with educational content to ensure that learning objectives are met without compromising the fun and engagement aspects [22].

AC-contract is a systems design approach that uses cognitive psychology concepts, such as schemas, to ensure that systems remain adaptable and reliable during changes. It inserts logical propositions into the source code, verified by a preprocessor, ensuring that the adaptable code meets the requirements even during changes. Applied to gamified educational platforms in software engineering (SEET), the use of methodologies such as AC-contract can ensure that these platforms are effective and reliable by adapting to various educational contexts and user interactions. This is crucial in dynamic environments where educational needs and interactions can change rapidly. In the field of gamification, the adoption of AC-contract principles allows gamified platforms to maintain effectiveness and reliability, addressing challenges such as active learning, personalization of study rhythms, and collaboration while reducing stress and responsibilities for students. This underscores the importance of adaptable and reliable systems and suggests that gamification should complement, not replace, traditional methods [23].

Moreover, tools like PrOnto, an ontology-driven business process mining tool, demonstrate the importance of identifying and modeling processes within organizations to improve their effectiveness and competitiveness. PrOnto’s approach of utilizing business ontologies to classify and abstract business processes can be analogously applied to educational settings. In gamified educational environments, understanding and modeling the learning processes can enhance the personalization and effectiveness of gamified interventions. By leveraging ontologies to dynamically exploit knowledge at runtime, similar to PrOnto, educational systems can better adapt to the needs and contexts of learners. This dynamic adaptation is crucial in gamified platforms where the engagement and motivation of learners are influenced by how well the system can personalize the experience based on real-time data analysis and contextual understanding [24].

To address the proposed challenges and explore the potential of gamification in SEET, a systematic evaluation of existing studies is necessary. This involves assessing the design, implementation, and outcomes of gamified educational tools and identifying best practices and areas for improvement.

4. Results and Discussion

The following section analyzes results of the performed review. Each subsection answers one of the six proposed research questions.

4.1. RQ1: What Is the Publication Trend in the Area of Gamification Applied to SEET?

To analyze the temporal evolution of research on gamification in SEET, we examined the publication trends over the past years. Figure 3 illustrates the number of publications per year from 2015 to 2023.

Figure 3. Publication trends over time for gamification in SEET.

The analysis reveals several notable trends. Starting in 2015, there were a modest number of publications, increasing over the next few years. A significant rise is observed in 2018, with the number of publications peaking at 11. This increase indicates growing interest and recognition of the potential benefits of gamification in SEET during this period.

Interestingly, the number of publications remained stable in 2019, again with six papers, before experiencing another rise in 2020 with eight publications. The most substantial growth occurred in 2021, with a peak of 16 publications. This surge can be attributed to the COVID-19 pandemic, which necessitated remote learning solutions and the adoption of innovative teaching methodologies, including gamification, to engage students in virtual environments. The continued interest in gamification indicates that it remains a relevant and important area of research in SEET, together with the higher number of journal publications in recent years.

Figure 4 presents the distribution of publication types, differentiating between conference papers and journal articles.

Figure 4. Distribution of publication types for gamification in SEET.

The analysis reveals that the majority of the research output on gamification in SEET has been in the form of conference papers. This is indicative of the dynamic and rapidly evolving nature of the field, where researchers prefer the relatively faster dissemination route offered by conferences to share their latest findings and innovations. Approximately 80% of the total publications were conference papers, reflecting the community’s emphasis on quick dissemination and discussion of new ideas.

In contrast, journal articles, which typically undergo a more rigorous and lengthy peer-review process, constituted around 20% of the publications. This lower percentage suggests that while there is a substantial amount of exploratory and preliminary research being conducted, fewer studies have reached the level of maturity required for journal publication. The presence of journal articles, however, highlights that some research in this domain has achieved significant depth and rigor, contributing to a more formal and comprehensive understanding of gamification in SEET.

4.2. RQ2: In Which Areas of Software Engineering Is Gamification Used?

Figure 5 illustrates the sectors where gamification has been applied in SEET.

Figure 5. Sectors of application for gamification in SEET.

The analysis indicates that the predominant application area for gamification in SEET is within university-level software engineering programs, which accounts for 30 studies, about 44% of the analyzed studies. This reflects a significant focus on integrating gamified approaches to enhance the learning experiences of students in higher education, particularly in courses related to software engineering, as shown in papers [11,14,25].

Schools and related educational institutions represent the second most common sector, with 23 studies, 34% of the analyzed papers. This includes primary, secondary, and other non-university educational contexts where gamification is used to make learning more engaging and effective.

Companies are also exploring the use of gamification (9% of the papers). In the corporate sector, gamification is employed to improve employee training, professional development, and motivation, as shown in [26]. The lower number of studies in this sector could be due to the proprietary nature of corporate training programs, which might not be as widely documented in academic literature.

The “Other” category, encompassing nine studies, includes various applications that do not fit neatly into the previously mentioned sectors. This involves informal learning environments, online courses, or interdisciplinary studies where gamification is applied.

4.3. RQ3: What Are the Analyzed Application Areas?

To explore how gamification is used within SEET, we first understand the search type of selected papers, then we analyze the various application areas where gamified approaches are implemented.

To evaluate the depth of analysis on gamification in SEET, we categorized the 68 primary studies into five distinct groups based on their focus: Proposal, Analysis, Implementation/Tool, Validation, and Other. The distribution of studies across these categories provides insight into the current state and focus areas of research in this field. Results are shown in Figure 6.

Figure 6. Distribution by search type.

• Proposal: This category includes studies that introduce new concepts, frameworks, or methodologies for applying gamification in SEET. We identified 10 papers that primarily focus on theoretical foundations and suggest innovative approaches to integrating gamification into educational contexts.
• Analysis: In this category, 10 studies provide detailed examinations of existing gamification techniques and their impacts on learning outcomes.
• Implementation/Tool: The largest category, with 24 studies, focuses on the practical aspects of implementing gamification. These papers describe the development and deployment of specific tools, platforms, or software that incorporate gamification elements into SEET. They often include case studies or reports on pilot projects.
• Validation: Comprising 17 studies, this category includes empirical research that evaluates the effectiveness of gamification through experiments, surveys, or longitudinal studies. These papers provide evidence-based insights into how gamification influences student engagement, motivation, and learning outcomes.
• Other: The remaining seven studies cover various other aspects of gamification that do not fit neatly into the above categories. This includes research on the broader impacts of gamification, such as its effects on educational policy, its role in lifelong learning, and interdisciplinary applications.

The categorization reveals a balanced approach to exploring gamification in SEET, with significant emphasis on practical implementation and empirical validation, and while a substantial number of studies propose and analyze gamification frameworks, the majority focus on real-world applications and their validation, highlighting a strong interest in understanding the practical benefits and challenges of gamification in education.

Figure 7 illustrates the distribution of the application areas. The analysis reveals that the primary use of gamification in SEET is for teaching support, which constitutes 56% of the studies. This significant proportion indicates that gamification is predominantly applied to enhance educational experiences, making learning more engaging and interactive for students. Examples of this include the incorporation of game elements into lectures, assignments, and assessments to motivate students and improve their learning outcomes. Work improvement represents 19% of the applications. This area focuses on using gamification to enhance productivity, collaboration, and efficiency in software engineering practices. For instance, gamified tools and platforms are used to improve team dynamics, project management, and individual performance within software development teams. Approach analysis accounts for 18% of the studies. This area involves evaluating and analyzing the effectiveness of different gamified approaches and methodologies in SEET. Research in this domain aims to understand the impact of gamification on learning outcomes, student engagement, and overall educational quality. Serious games constitute 7% of the applications. These are games designed for purposes beyond mere entertainment, specifically to educate and train individuals in software engineering concepts and practices. Serious games provide immersive and interactive learning environments that simulate real-world software engineering challenges. For instance, a serious game might involve a scenario where players must collaboratively debug a software application or manage a software project with constraints and deadlines [27]. These games are categorized based on their educational objectives and the inclusion of realistic software engineering tasks. An example is “SimSE” [28], a game that simulates software engineering processes, allowing students to experience the impact of their decisions on the project’s outcome. Another example is “CodeSpells” [29], where players write code to cast spells, learning programming logic and problem-solving skills in a magical context. Serious games enhance understanding and retention of complex software engineering principles.

Figure 7. Application areas of gamification in SEET.

4.4. RQ4: What Contribution Does Gamification Offer When It Is Applied to SEET?

To answer this RQ, we summarize the main contribution of each paper in Table 1. This table provides an overview of the key findings from each study, highlighting the specific ways in which gamification has been applied to the education topic. By examining these contributions, we can identify common themes, benefits, and challenges associated with gamification in this context, offering insights into its effectiveness and areas for further research.

Table 1. Main finding for each analyzed paper.

Paper	Contribution
Ortega-Arranz et al. [13]	This paper discusses the use of gamification in MOOCs for a large number of students, using automatic tools to assign rewards (flipped-classroom tickets, quiz benefits, extra learning content) for a course on Spanish history from the 11th to the 16th century.
Arif et al. [30]	This paper covers the use of gamification for web programming in high schools, specifically for learning HTML, CSS, and JavaScript. The gamification involves a web app with rewards such as avatars, lives, and time constraints.
Rahim et al. [31]	This paper focuses on using gamification to learn linear algebra, with storytelling as the main gamified element, along with avatars (king or queen), levels representing different topics, and scoring.
Hajarian and Diaz [32]	This paper describes creating an application with gamification techniques, emphasizing a reward-based system with customizable items, score saving, and leaderboards.
Iquira et al. [33]	This paper presents a mobile application using gamification to understand software engineering, particularly extreme programming (XP), with points and level progression as elements. Positive results were achieved in testing.
Robledo-Rella et al. [34]	This paper describes a mobile and web application for learning discrete math, physics, and chemistry through gamification, using quizzes, points, customizations, and avatars, with positive feedback from students.
Gomes Fernandes Matsubara and Lima Corrêa Da Silva [35]	This paper mentions using a gamified platform to learn software engineering, utilizing missions, experience points (XPs), and level progression.
Rodrigues et al. [36]	This paper surveys software engineering professors to determine if GBL and gamification improve learning outcomes, with positive results and gamified elements such as quizzes, points, levels, and badges.
Quinde et al. [37]	This paper uses gamification in a penitentiary for digital literacy, with tutorials guiding inmates through basic literacy and computing concepts.
John and Fertig [38]	This paper uses Moodle for gamification in agile and scrum model learning, with points, badges, anonymous leaderboards, quizzes, and storytelling, though the latter was less engaging over time.
Ivanova et al. [39]	This paper utilizes various existing platforms with gamification for software engineering learning, including Kahoot and “Who Wants to Be a Millionaire?” for theoretical testing, and platforms for project role division and coding.
Gasca-Hurtado et al. [40]	This paper describes creating a tool for educators to develop educational content with gamification elements, tested on a software engineering course using Happy Faces for points and Kahoot for web 2.0.
Carreño-León et al. [41]	This paper applies gamification to basic programming learning using playing cards with commands, forming groups to solve assigned algorithms, with different difficulty levels.
Sherif et al. [42]	This paper describes a platform (CoverBot) using gamification to teach code debugging, with levels, graphics, and sounds to enhance user experience.
Norsanto and Rosmansyah [43]	This paper applies gamification to civil service training with a custom application using missions, points, ranking systems, levels, and badges.
Call et al. [44]	This paper uses gamification for understanding algorithms and data structures in C++, with Moodle incorporating points and leaderboards to motivate faster assignment completion, and a Q&A forum for extra points.
Trinidad et al. [45]	This paper analyzes a multi-context, narrative-based platform (GoRace) for educational and workplace use, with storytelling, challenges, rewards, penalties, rankings, and a shop for advantageous items.
Prasetya et al. [46]	This paper uses a tower defense game for learning formal languages, where users defend a processor from bugs while creating abstract syntax trees for assigned commands.
Bucchiarone et al. [47]	This paper discusses gamification in programming and modeling (UML diagrams) using PolyGlot and PapyGame platforms with points, XP, levels, coins, and rewards.
Lema Moreta et al. [48]	This paper applies gamification to a risk management course with a web app using points, levels, and leaderboards for competition, with positive results.
Ouhbi and Pombo [49]	This review surveys instructors, identifying gaps in SEE teaching and proposing solutions like SWEBOK guidance, Mentimeter, and Flipped Classroom.
Villagra et al. [50]	This paper provides gamification implementation examples like Flipped Classroom, recorded short lessons, and group projects.
Moser et al. [51]	This paper uses gamification for university project development, suggesting characteristics like negative points for wrong code and positive points for solving software quality issues.
Rattadilok et al. [52]	This paper presents “iGaME”, a bot for teaching machine learning algorithms in classrooms using gamification.
Bucchiarone et al. [53]	This paper uses gamification in “Minecraft” to teach Scrum development methods to electrical engineering students.
Ebert et al. [54]	This paper describes applying gamification in Vector to develop software applications, enhancing user engagement and learning.
Maxim et al. [55]	This paper describes teachers using gamification principles like realistic stories for students to immerse in software creation tasks.
Jiménez-Hernández et al. [56]	This paper presents the serious game “Tree Legend” for studying trees/graphs.
Nagaria et al. [57]	This paper describes MOOC platforms like Moodle using the “CodeRunner” plugin for coding questions and “Pacman” for pathfinding algorithms.
Margalit [58]	This paper describes “Capture the Flag” for understanding AI, machine learning, and microprocessor decoding.
Stol et al. [59]	This paper discusses gamification in software engineering training to expand knowledge of new development technologies, with younger SWE more receptive than seniors. Stackoverflow’s gamification with badges and reputation is also mentioned.
Fulcini and Torchiano [25]	This paper proposes using ChatGPT to find strategies for implementing gamification in software engineering Education.
Đambić et al. [60]	This paper presents an experiment in a Croatian university during COVID-19, using a mobile app for short lessons and gamified elements like leaderboards, points, and rewards.
Mi et al. [61]	This paper discusses GamiCRS, a web application using PBL (Points-Badges-Levels) for coding skill improvement and student motivation, tested in a Hong Kong university with positive feedback.
Monteiro et al. [14]	This paper presents MEEGA+, a framework for evaluating educational games in software engineering using the GQIM approach, evaluated by three researchers in five phases.
Takbiri et al. [11]	This paper discusses gamification’s impact on students and teachers in software engineering, education, and psychology, highlighting improvements in individual skills and teamwork.
Molins-Ruano et al. [12]	This paper discusses e-valUAM, an adaptive gamified system tested in a Madrid university using the MUD model to enhance engagement.
Tsunoda and Yumoto [10]	This paper compares the PRBL (points-ranking-badges-levels) gamification method with traditional teaching, highlighting its advantages and disadvantages.
Skalka et al. [62]	This paper discusses Microlearning, an action-oriented approach with short lessons, combined with interactive gamification elements using the Octalysis Framework.
Silvis-Cividjian [63]	This paper discusses a course for medical, aerospace, and IT equipment testers using gamification to address various teaching challenges and enhance realism.
Makarova et al. [64]	This paper highlights the advantages and disadvantages of e-learning, showing how gamification can improve teaching and training with role-playing, exercise games, and simulation games.
de Paula Porto et al. [65]	This paper characterizes how gamification has been applied in software engineering, identifying benefits and challenges.
Vlahu-Gjorgievska et al. [66]	This paper discusses the inclusion of computational thinking in curricula and the need for an educational approach involving various stakeholders.
Chan et al. [67]	This paper examines a course on professional software development and the integration of gamification to enhance learning outcomes.
Figueiredo and García-Peñalvo [68]	This paper highlights the motivational power of games and explores gamification’s potential to increase student engagement in programming courses.
Pratama et al. [69]	This paper describes the development and impact of Rimigs, a gamification system aimed at improving student engagement and learning outcomes.
Naik and Jenkins [70]	This paper reviews the role of agile methodologies in software development education and how gamification can enhance collaborative learning.
Swacha and Szydłowska [71]	This paper evaluates the effectiveness of gamification in computer programming education through various case studies and learning outcomes.
Sousa-Vieira et al. [72]	This paper analyzes the impact of social learning and gamification on higher education, focusing on activity levels and learning results.
Ren and Barrett [26]	This paper explores the importance of communication in software management and how gamification can improve team interactions and project outcomes.
Monteiro et al. [73]	This paper presents the recurring theme of gamification in software engineering education literature and its influence on student engagement.
Jusas et al. [74]	This paper assesses the potential of gamification to enhance student engagement, drive learning, and support sustainable educational practices.
Maher et al. [75]	This paper introduces the Personalized Adaptive Gamified E-learning (PAGE) model, which extends MOOCs with enhanced learning analytics and visualization to support learner intervention. The results indicate a positive potential for learning adaptation and the necessity of focusing on gamification.
Bachtiar et al. [76]	This paper develops an e-learning system named Code Mania (CoMa) that integrates gamification elements like leaderboards and badges to increase student engagement in a Java Programming course. The system performs well as specified, demonstrating the potential of gamification in enhancing e-learning environments.
Laskowski [77]	This paper investigates the applicability of gamification across different higher education courses through an experiment involving computer science students. The study shows the comparative results of gamified and non-gamified groups, indicating the impact of gamification on student performance.
Fuchs and Wolff [78]	This paper presents an online learning platform with gamification elements designed for software engineering education. It combines formative assessment with gamification to enhance learning experiences, providing detailed examples and system design.
Bucchiarone et al. [79]	This paper reports on the outcomes of the 6th International Workshop on Games and Software Engineering, highlighting the growing complexity and need for theoretical frameworks in gamification. The workshop covered perspectives on software projects, testing, and design, with insights from keynotes and panel discussions.
Bucchiarone et al. [80]	This paper presents POLYGLOT, a gamified programming environment targeting programming languages education and text-based modeling languages like SysML v2. The approach allows for the creation of heterogeneous gamification scenarios, enhancing the learning experience.
Poecze and Tjoa [81]	This paper explores the relevance of publication bias tests in meta-analytical approaches to gamification in higher education. It discusses the challenges in conducting meta-analyses due to heterogeneity and compares methods for correcting publication bias.
Cabezas [82]	This paper introduces a continuous improvement cycle for teaching scenarios in engineering, combining gamification theory and ABET criteria. The proposed cycle is applied in a computer programming course, showing a positive impact on student engagement and learning outcomes.
Bucchiarone et al. [83]	This paper discusses the convergence of game engineering, software engineering, and user experience to create solutions blending game strengths with real-world applications. It highlights the potential benefits of gamification and serious games in various domains such as education and healthcare.
Ristov et al. [84]	This paper presents a gamification approach in a hardware-based course on microprocessors and microcontrollers for computer science students. The approach improved course grades and motivated students to enroll in other hardware courses, demonstrating the positive impact of gamification on student interest and performance.
Bernik et al. [85]	This paper presents empirical research on the use of gamification in online programming courses. A gamified e-course was designed, and its impact on student engagement and use of learning materials was examined, showing potential benefits of gamification in e-learning.
Schäfer [86]	This paper reports on a gamification approach using Minecraft to train students in Scrum, an agile project management method. The study compares two teaching periods, highlighting findings and lessons learned from using game-based learning to teach Scrum principles.
Petrov et al. [87]	This paper analyzes gamification software for promoting minority languages. It provides an overview of current educational software and assesses the need for new gamification solutions to support regional and minority languages.
Tsalikidis and Pavlidis [88]	This paper presents jLegends, an online multiplayer platform game designed to teach programming with JavaScript. The game employs a role-playing approach to enhance learning through game mechanics, demonstrating the effectiveness of game-based learning in programming education.

4.5. RQ5: In Which Continents Is Gamification Mostly Analyzed?

An analysis on the geographical distribution of relevant papers reveals that Europe is the leading region, with 41% of the studies, as shown in Figure 8. This significant proportion indicates a robust interest and investment in gamification strategies within European educational institutions and research communities.

Figure 8. Geographical distribution of studies on gamification in SEET.

The prominence of European research in this field suggests that many universities and educational bodies in Europe are actively exploring and implementing gamification to enhance learning outcomes. This focus could be driven by several factors, including the strong support for educational innovation in European countries, the availability of funding for educational research, and the collaborative networks among European researchers. The “hybrid” field includes papers where authors have affiliations from two or more continents, to avoid inconsistencies in the data extraction.

4.6. RQ6: What Are the Advantages and Disadvantages of Gamification When Applied to SEET?

The application of gamification as an educational tool has been the subject of extensive analysis, revealing both its advantages and disadvantages. We examine the potential benefits and drawbacks of this technique below.

Among the advantages, gamification has been shown to significantly enhance student participation [89] and interest [68] compared to traditional teaching methods. For instance, in the study conducted at the University of Applied Sciences Würzburg-Schweinfurt [38], 40 students were surveyed, with 27 respondents. Initially, 35% of students reported moderate motivation levels before engaging with the gamified course. Post-intervention, 40% of students indicated increased motivation due to the gamification content, although nearly 20% did not find it motivating at all. This finding aligns with the broader research on gamification, highlighting that motivation varies among individuals and is influenced by different types of gamified elements [35].

Furthermore, the economic growth associated with gamification is notable. As reported in [45], the gamification market is projected to expand from USD 9.1 billion in 2020 to USD 30.7 billion by 2025, with a growth rate of 27.4% per year. This underscores the increasing interest and investment in gamification as a promising educational approach. Gamification also provides intrinsic benefits for student motivation. By offering rewards and real-time feedback, students can visualize their progress and achievements, thereby enhancing their learning experience.

However, the application of gamification is not without its challenges. Some limitations of gamification in SEET are summarized below [36]:

• Difficulty in measuring performance improvements;
• Increased workload for educators [75];
• Lack of digital platforms to implement gamified techniques [36];
• Challenges in engaging all students;
• Difficulty for some students to understand the gamification method;
• Lack of appreciation for the method by some students;
• Difficulty for students in gaining human feedback, for platforms in which gamification is used alongside Artificial Intelligence [25];
• Insufficient knowledge of gamification approaches;
• Limited time and interest: students can sometimes become annoyed or disengaged by gamification elements if they find them distracting [90];
• Scarcity of materials and resources [75];
• Ensuring proper use of gamification by students.

The analysis indicates that gamification, like any pedagogical method, presents both strengths and weaknesses. The primary challenge lies in the nascent stage of this technique, which precludes definitive conclusions about its long-term efficacy [51]. Nevertheless, short- to medium-term studies and experiences in educational settings provide valuable insights into its potential benefits and limitations [91]. Furthermore, the novelty of gamification means that large-scale, comprehensive studies are currently lacking, making it difficult to obtain a fully reliable overview of all its advantages and disadvantages. Continued research and practical implementation are essential to fully understand the impact of gamification on education.

5. Comparison with Other Review Papers

Different reviews on the topic of gamification applied to SEET were examined. The main ones are proposed below, together with analyzed databases and main findings.

Dal Sasso et al. [92] propose a critical overview of gamification and its application in supporting software engineering tasks, starting from the IEEEXplore database. They highlight how to implement gamified approaches and propose a method to evaluate gamification systems. However, the presented work is not a systematic review, but a first approach in evaluating the literature.

Pedreira et al. [93] carry out a systematic mapping study to analyze papers between 2011 and mid-2014. They use Scopus, Science Direct, IEEEXplore, ACM Digital Library, and Springer databases to gather papers and find 29 primary contributions: main results show that research in this field is still preliminary, and most of the considered papers focus on software development and not on proving empirical evidence of pros and cons of gamification.

Barreto et al. [94] carry a mixed-approach literature review, composed of both an ad hoc review in which they manually select relevant papers and a systematic review following software engineering guidelines based on ACM Digital Library, IEEEXplore, and Science Direct databases. They conclude that “researchers in the field tend towards a strict view of gamification, the practical results of gamification are unclear and polemic, and this research area has still much to improve”.

Ngandu et al. [95] analyze 15 papers, conducting a literature review on IEEEXplore and Science Direct to understand the impact of gamification and its key elements to student interest in software engineering. Their main findings regard points and leaderboards, considered as the main elements used in this topic.

Chamorro-Atalaya et al. [96] study the impact of COVID-19 pandemics in the education shift and the opportunities carried by gamification in online teaching. Analyzed papers are gathered from Taylor & Francis, IEEEXplore, and Scopus. Software engineering and Computer Science result to be the main topics in which gamification is applied, and the application of gamification effectively generates motivation, commitment, and permanent participation of students.

Monteiro et al. [97] analyze papers coming from IEEEXplore and ACM Digital Library that report procedures for the evaluation of gamification. However, only three of the 64 studies actually propose evaluation models for gamification. The main finding is that “the evaluation of gamification requires a mix of subjective and objective inputs, and qualitative and quantitative data analysis approaches”.

Analysis of related works shows the need for developing an updated literature review, to analyze not only teaching effectiveness, but also the role of gamification in software engineering-based jobs. Our review also makes an analysis on how the effectiveness of gamification in SEET is measured. The comparison between the different reviews is summarized in Table 2. The columns in the table include the reference for the reviewed work, key terms used in this study to highlight its focus areas, whether this study followed a systematic review or mapping methodology, the primary setting or context where gamification was applied, whether this study involved or considered industrial applications or partnerships or their involvement in the SEET topic, and a summary of the practical findings regarding the effectiveness or impact of gamification as reported by the study.

Table 2. Comparison between related works and our review.

Reference	Keywords	Systematic Approach	Main Application Field	Industrial Applications	Findings
Dal Sasso et al. [92]	Games, Software Engineering, Context, Psychology, Computer Bugs, Collaboration, Systematics	No	Universities	No	No findings
Pedreira et al. [93]	Gamification, Software Engineering, Systematic Mapping	Yes	Universities	No	No findings
Barreto and França [94]	Motivation, Engagement, Gamification, Software Engineering	Mixed	Universities	No	Unclear practical results
Ngandu et al. [95]	Gamification, Software Engineering, Student Interest, Game Elements, Engagement, Motivation, Participation	Yes	Student engagement	No	No findings
Chamorro-Atalaya et al. [96]	Gamification, Engineering Education, Design, Success Factors, Motivation	Yes	Universities after COVID-19 pandemics	No	No findings
Barbosa Monteiro et al. [97]	Gamification, Systematic Mapping, Evaluation, Software Engineering, Education	Yes	Universities	Yes	Yes
Our proposal	Gamification, Software Engineering, Education, Learning, Literature Review	Yes	Universities and companies	Yes	Yes

6. Conclusions

The use of gamification in SEET is a contentious and highly debated topic among experts in teaching, while disagreements exist regarding the integration of gaming aspects into educational systems, numerous proposed projects and studies have demonstrated positive outcomes that support the efficacy of gamification.

The advantages of using game environments for teaching are numerous. Gamification allows students to actively construct their understanding of topics, learn at their own pace individually or collectively in spontaneous groups, and proceed on different paths at varying speeds according to their interests and abilities. It also promotes collaboration and encourages just-in-time learning, as opposed to general training.

However, this teaching method also has some psychological drawbacks, particularly concerning its potential negative effects on students. Students, with gamification, are free to fail and free to experiment, and while these freedoms aim to provide students with the ideal tools to build their own experiences, they can also lead to an overclock of responsibility. This can result in a stressful and unmanageable situation for some students.

Our review, based on the analysis of 68 papers, reveals a significant interest in using gamification in universities and secondary schools. Despite its recent development and the promising studies showcasing its effectiveness, there remains a tendency to rely on traditional learning methods, often overlooking the potential of gamification. Nonetheless, the growing need to integrate digital content to enhance learning has spurred an increased interest in this alternative method. It is crucial to recognize that not all students or users are motivated by gamification, and while the majority may find it effective, there will always be a percentage of students who prefer traditional teaching methods due to differing attitudes and learning times. Therefore, it is unrealistic to expect a single tool to be universally effective.

In conclusion, gamification in education should be viewed as a valuable support tool rather than a total replacement for traditional teaching methods. As highlighted in many of the analyzed papers, gamification is often used alongside classic, proven teaching methods and sometimes as an alternative support. Its potential to enhance learning experiences is significant, but it must be integrated thoughtfully to complement and not completely replace traditional educational approaches.