Integrating Augmented Reality, Gamification, and Serious Games in Computer Science Education

Abstract

This study aims to evaluate the impact of using augmented reality, gamification, and serious games in computer science education. The study presents the development process of an educational mobile application, describes an experiment that was conducted and involved 117 higher education students, and analyzes the results of a 49-item paper-based questionnaire. In total, 8 research questions were explored. The results of the study revealed that several educational benefits can be yielded when integrating such applications in teaching and learning activities and actively involving students in the design and development process. In particular, the application was assessed as an effective learning tool that could enrich and improve the educational process and create interactive, inclusive, and student-centered learning environments. Its use led mostly to positive effects and experiences while maintaining the negative ones to a minimum and most students expressed positive emotions. Students were able to learn in a more enjoyable and interesting manner, and their motivation, engagement, self-efficacy, and immersion were greatly increased. Students’ innate need for autonomy, competence, and relatedness was satisfactorily met and both their intrinsic and extrinsic learning motivations were triggered. They felt a sense of belonging and cultivated their social skills. The potential of the application to improve students’ knowledge acquisition and academic achievements was also observed. The application also enabled students to improve their computational thinking and critical thinking skills. Therefore, the potential of combining augmented reality, gamification, and serious games to enhance students’ cognitive and social–emotional development was highlighted.

1. Introduction

The integration of digital technologies in the educational process is an integral part for attaining high quality, equitable, and inclusive education, as well as meeting students’ educational needs which are key requirements to achieve under the sustainable development goal 4 (SDG-4) based on the 2030 sustainable development agenda set by the United Nations [1]. Additionally, the use of new technologies and virtual learning environments can lead to the creation of effective educational tools that can help meet the new educational needs and requirements, provide students with opportunities for meaningful and interactive learning experiences, and redefine teacher and student roles [2,3,4].

Digital technologies can support different fields of studies and computer science itself is not an exception. New technologies can facilitate the visualization and comprehension of complex concepts and improve learners’ engagement [5]. Furthermore, they can improve students’ motivation and learning outcomes when they are playfully integrated [6,7]. Hence, when used in a student-centered and gameful way, digital technologies can assist the teaching and learning of computer science in both K–12 [8] and higher education settings [9]. When adopting new technologies and approaches in educational settings, it is important to consider whether students’ innate need for autonomy, competence, and relatedness is satisfied [10]. Therefore, it is important to explore the self-determination theory (SDT) when integrating them [11].

Augmented reality is an interactive and flexible technology that is being used in educational settings to improve the learning experience and outcomes. Augmented reality combines the physical with the virtual environment by integrating and embedding virtual objects and information into the real world. In the immersive and interactive environment that is created, virtual and physical objects co-exist and users are able to interact with them and sense them through their senses without being isolated from the real environment [12,13,14,15,16,17]. Due to its flexible nature, augmented reality can be used in conjunction with other novel approaches and technologies. Gamification and serious games, which can be used together with augmented reality [14], are also being increasingly applied in the educational domain [18,19,20]. Gamification constitutes an active methodology that can enhance students’ motivation and engagement and involves the use of game elements in non-game-related contexts [21,22,23,24]. Serious games are also used in educational contexts to provide learning activities in a more playful, interactive, and engaging manner while increasing students’ motivation and involvement. Serious games refer to any game-based initiatives which, instead of focusing on pure entertainment, place emphasis on educational aspects and other primary purposes [25,26,27,28]. When combined with gamification and serious games, augmented reality has the potential to provide significant educational benefits. Additionally, through this combination and the use of artificial intelligence, inclusive learning can be supported and promoted as it enables ubiquitous and personalized learning experiences that take each learner’s unique traits and learning preferences into account [29].

However, the number of studies that explore the combination of gamification and serious games with augmented reality and examine its use in computer science education still remains small. Nonetheless, the results of existing studies suggest that their integration in educational settings is positively viewed and has the potential to yield positive learning outcomes [14,30,31,32]. Therefore, there is a clear need for more empirical studies to be conducted to better comprehend the impact that the combination of extended reality, serious games, and gamification can have in the educational and training domains and this is particularly true in the field of computer science [14].

Consequently, the aim of this study is to evaluate the impact of an educational augmented reality application that uses gamification elements and serious games in computer science education. The study goes over other related works and presents the method used, the development process, the experiment conducted, the research tools, and the analysis of the results. Additionally, it discusses the findings, makes comparisons with the literature, and, finally, it highlights the conclusions that arose and makes suggestions for future research directions. To guide the study, the following research questions (RQ) were set to be explored:

• RQ1: How often and on which devices do students mostly play games?
• RQ2: How did students evaluate the application in terms of usability and learnability?
• RQ3: What was the degree of competence, immersion, tension, flow, and challenge that students experienced when using the application?
• RQ4: Did the use of the application result in positive or negative effects on students and their learning experience?
• RQ5: What kind of motivation is mostly triggered when using the application as a learning tool?
• RQ6: Which emotions did students mostly feel when using the application as a learning tool?
• RQ7: To what degree does the application satisfy students’ innate need for autonomy, competence, and relatedness?
• RQ8: What were the educational benefits yielded when integrating the application?

2. Related Work

A recent systematic literature review conducted by Lampropoulos et al. [14] explored the use of augmented reality and gamification in education. Based on its findings, the field of computer science was one that had very few studies exploring the integration of augmented reality in combination with gamification and serious games.

Stefanidi et al. [33] combined augmented reality with gamification to create an application that will allow primary education students to better understand how intelligent environments work and are programmed. They followed a mixed method approach and focused on how students perceived the concept of intelligent environments through the use of observations, open-ended questions as well as pre-tests and post-tests. The application had game-like features and used points. Their results revealed that the overall learning experience was interesting and fun according to the students who better understood the idea of intelligent environments.

With the aim of creating a playful and enjoyable learning experience that would motivate and encourage students to review their acquired knowledge, Ortiz et al. [34] developed an augmented reality serious game that used points and quiz questions and focused on higher education students and the topic of distributed architecture. Using an ad hoc questionnaire, they quantitatively analyzed the students’ viewpoints. Based on their results, when using the application, students could better comprehend the subject taught, and they greatly valued its motivating aspects which encouraged them to revise the study material.

Alqahtani and Kavakli-Thorne [35] followed a quantitative approach using an ad hoc questionnaire to analyze higher education students’ viewpoints regarding the implementation of a gamified augmented reality application. In particular, the application used quiz questions and points, focused on the topic of cybersecurity, and aimed at motivating students to be more cautious and aware of cyberattacks. Students assessed the application as a useful educational tool that enabled them to learn about cybersecurity threats and how to stay secure and safe in online environments.

Aiming at understanding if students found the use of an augmented reality application that integrates gamification elements, such as points and levels, enjoyable, and useful as an educational tool to overcome learning problems, Stefanidi et al. [36] used a qualitative approach in their study. Specifically, they used observations to detect students’ interaction with the gamified augmented reality application they developed which focused on primary education students. Their results revealed that the application was positively assessed and made the overall learning experience more interesting and engaging.

Following a mixed method approach, Schez-Sobrino et al. [37] explored primary education students’ views regarding their intention to use a gamified augmented reality application and its perceived usefulness. More specifically, they used open-ended questions in combination with an ad hoc questionnaire that integrated questions from the Technology Acceptance Model (TAM) [38]. The application focused on increasing students’ knowledge acquisition and used achievements, badges, and game-like features. According to their results, the students showcased an increased interest in the learning tasks and were more motivated to participate.

In their study, Song et al. [39] explored higher education students’ engagement, task completion rates, and learning outcomes when using an educational application that combines augmented reality with gamification. They adopted a quantitative approach using data from students’ final grades as well as data that they gathered during their experiment. Their applications had specific tasks and game-like features. Based on their results, the specific application managed to increase students’ learning outcomes, engagement, and satisfaction and offered more personalized learning opportunities. Other studies, such as [40,41], have presented prototype applications and proposals regarding the design of their gamified augmented reality applications and their use to teach programming in educational settings.

From the above-mentioned studies, it can be inferred that the use of augmented reality and gamification can lead to positive learning outcomes. Nonetheless, most studies focus on perspectives of the overall experience using ad hoc questionnaires without assessing the application from a technical perspective or evaluating its gamefulness. Moreover, the studies only integrate a few gamification elements and, in some cases, these are not integrated in the context of the gamified experience but more as a means to assess students’ performance. Thus, although some gamification elements are integrated, they are not always incorporated in a cohesive way within the educational application. Computer science is a wide field of study and, consequently, studies focus on particular aspects of course material. Therefore, there is a clear need for more empirical studies to be conducted that assess not only the learning gains and perspectives of the educational community but also look into the technical aspects, students’ cognitive and social-emotional development, and the gamefulness of the application.

3. Materials and Methods

This section goes over the methods used in this study. It presents the development and characteristics of the mobile application, briefly goes over the research tools used, and describes the experiment carried out. In particular, the study followed a quantitative approach in which higher education students used and evaluated the educational application developed through a paper-based questionnaire.

3.1. Development Process

The target audience for the application was higher education students. As the application aimed to be accessible to most students through their own devices at any place and time, the application focused on mobile devices for both Android and iOS. This decision was based on observations that had been made prior to development as well as through discussions with the students.

To develop the mobile application and the serious games within it, Unity was used. Unity constitutes a real-time and cross-platform game engine and platform that enables the development of 2D, 3D, virtual reality, and augmented reality games and applications. Its core functions can be further extended through the use of extensions, packages, and plug-ins. One of the main packages used was the Vuforia Engine, which is a software development kit (SDK) for creating augmented reality applications within Unity. A particular fiducial marker was created as the basis for the augmented reality experiences, on top of which the game environment, virtual objects, and information were displayed.

The analysis, design, development, implementation, and evaluation (ADDIE) [42] model was adopted for this study. Initially, the learning needs and requirements of the students were observed during lecture and laboratory lessons of the courses taught in the Department of Information and Electronic Engineering at International Hellenic University. The results of their recent examinations were also examined. The specific courses can be grouped in programming, algorithms and data structures, artificial intelligence and machine learning, human–computer interaction, databases, web technologies, and operating systems. The material was specifically created based on the specifications and requirements of each course. For example, the material related to the operating systems course referred to file structures, file permissions, and regular expressions, among others. The material for the web technologies course referred to HyperText Markup Language (HTML), Cascading Style Sheets (CSS), and JavaScript. After determining the learning needs and the specific material, the design of the application began. Aiming at providing a student-centered learning experience, a cooperative inquiry [43] co-design approach was followed. In particular, a group of students provided ideas about and inputs into its design, the various stages, games, gamification elements, and user interface used, assessed the application throughout its development, and provided crucial feedback on the application, the learning experience, and its impact. The development of the application was carried out using the aforementioned tools. When the development reached a significant milestone (e.g., a game or level was fully completed), the group of students used and assessed the functionality of the application. Hence, these specific students were actively involved throughout the development process. It must be mentioned that these students did not take part in the final evaluation of the application presented in this study. After the development had been completed, the application was implemented within educational settings and then evaluated by the students. These two phases of the ADDIE model are analyzed in the following section.

As the application aimed at being implemented and covering several of the existing courses taught within this specific department, the application had different categories for each course, as explained above. Within each category, there were multiple levels of increasing difficulty and complexity in which students had to complete games, answer questions and quizzes, explore areas, interact with virtual objects and characters, view short educational videos, and solve puzzles. Gamification elements, such as tasks, points, scores, time, leaderboards, hints, feedback, and badges, were adopted to make the use of the application more engaging, motivating, and enjoyable [14].

As it was essential to provide students with a fully functional application, the data were stored in a Structured Query Language (SQL) database and students could view the leaderboards and assess their performance in comparison to others. Both specific to each course, leaderboards and a general leaderboard were available. This design approach enabled the use of learning analytics, since information, such as time to complete task, number of tries, mistakes made, help asked, performance, etc., was gathered which can lead to better learning outcomes and more personalized learning experiences [44]. It is worth noting that to offer ubiquitous learning opportunities, the application was also functional without an Internet connection, besides, of course, the connection to the SQL database and the retrieval and upload of data and information related to it and some of the longest educational videos used. To provide a better experience when using the application even offline, the fiducial marker database required was embedded in the application and the most recent version of the leaderboards was locally downloaded to the users’ device when using the application and viewing the leaderboards which could be recalled even if there was no Internet connection. Moreover, the use of only one fiducial marker led to a more flexible way of learning as specific material and books were not required and all the necessary information was augmented on top of it, including the study material when appropriate. Finally, it must be mentioned that only publicly available 3D models were used to develop the application so that students could easily find and reuse the models for their own applications.

3.2. Experiment

The application was developed to be used within the courses of the specific curriculum to enrich the educational process. However, before its adoption and implementation, it was essential to comprehend students’ viewpoints and make further adjustments and improvements based on their feedback. Therefore, the experiment that was carried out aimed at understanding students’ perspectives regarding the learnability and usability of the application, its use in educational settings, the educational benefits it can yield, the learning motivations triggered, the emotions felt, the affects experienced, and the overall serious game experience. Through this analysis, the applicability and suitability of the application to be implemented within the related courses were assessed and the areas that could be further improved and expanded were identified which, in turn, greatly enhanced the impact that the application can have.

In total, 117 higher education students of the Department of Information and Electronic Engineering at International Hellenic University participated in the experiment which was conducted in February 2023. The participants had an average age of 21.98 and the vast majority of them were in their third year of study. As the specific department offers a 5-year degree, it can be said that most students had enough experience in and knowledge of the subjects contained within the application and were capable of effectively evaluating the application and the overall learning experience. The sample consisted of 15 female (12.82%) and 102 male (87.18%) students which is representative of the student body of this department. All students voluntarily participated in the experiment and completed the questionnaire which was anonymous and confidential and did not pose any risks (physical or psychological) to the participants. Personal data were not collected at any stage and participants were informed about the aims and purpose of the study and were aware that the data would be used for research purposes.

Hence, the implementation of the application was carried out by eight groups of 20–23 students. Each group separately participated in the experiment and the same process was followed for each group. Students were able to download and install the application a week prior to the experiment as well as during it. Nonetheless, the fiducial marker required for the application to be fully functional was only given to them during the experiment so that their initial experiences, interactions, and thoughts could be assessed. Additionally, the instructor was the same in each group to minimize external factors influencing the results. Initially, a brief presentation was given regarding the application, its learning goals, and the whole process that was going to be followed. After the presentation had been completed and the students’ questions had been answered, the students could freely use and become familiar with the application and its controls. It is worth noting that the use of advanced human–computer interaction technologies is taught in the curriculum of the department, so few students from higher years were familiar with the use of augmented reality. After that, the learning tasks that had to be completed were presented to the students to carry out. The tasks involved students’ interactions with the application across different categories, levels, and in-game activities and events. After completing the specific tasks, students were free to spend more time using and further exploring the application and its contents. Finally, the students evaluated the application and provided their viewpoints through the use of a paper-based questionnaire. The whole process of the experiment adopted in this study is presented in Figure 1.

3.3. Research Tools

Besides the ad hoc questions used within the paper-based questionnaire which will be presented and explained in the result analysis section, two existing and validated questionnaires were also adopted. In particular, the System Usability Scale (SUS) [45] was used to assess the usability and learnability of the application. This questionnaire consists of 10 questions and used a 1–5 Likert scale from “Strongly disagree” to “Strongly agree”. Some of its questions (numbers 1, 3, 5, 7, and 9) involve positive elements of the use of the application while the rest (numbers 2, 4, 6, 8, and 10) refer to its drawbacks. Based on the formula provided in [45], an SUS score can be calculated. This score refers to a percentile ranking of the application in terms of learnability and usability. Due to the use of serious games, the second questionnaire adopted was the Game Experience Questionnaire (GEQ) [46]. More specifically, the questions related to the components of competence, immersion, flow, tension, challenge, negative affect, positive affect, negative experience, and positive experience were used. The GEQ uses a 0–4 Likert scale, but for the purposes of this study and to be in line with the other questions of the survey, a Likert scale of 1–5 was used. Consequently, the questionnaire used consisted of 49 closed-ended questions.

4. Result Analysis

This section goes over the results of this study which are based on higher education students’ perspectives and attitudes. Particularly, the separate components of the survey, which were distributed to the students, are presented using figures and tables, following a quantitative approach, and using a 1–5 Likert scale.

4.1. Students’ Gaming Habits

The average time students spend playing digital games every day and the devices they mostly use were explored. Based on the results presented in Figure 2, most students spend about two hours playing (30.77%), followed by less than 30 min (27.35%) while according to Figure 3, most students use their desktop to play digital games (47.79%), followed by mobile devices (29.41%). It is worth noting that students could select more than one device that they used on a daily basis. As a result, the total number of responses when analyzing the devices used was 136 instead of 117 which was the number of students that participated. It can be said that desktop devices were mostly used by the students for longer periods of play time when compared to mobile devices.

4.2. System Usability Scale (SUS)

Regarding the SUS questionnaire, the responses of the students are presented in Figure 4.

Table 1. Descriptive statistics of the SUS questionnaire.

Question	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree	Means	Standard Deviation
1. I think that I would like to use this system frequently.	0 (0.00%)	3 (2.56%)	12 (10.26%)	74 (63.25%)	28 (23.93%)	4.09	0.66
2. I found the system unnecessarily complex.	37 (31.62%)	73 (62.39%)	7 (5.98%)	0 (0.00%)	0 (0.00%)	1.74	0.56
3. I thought the system was easy to use.	0 (0.00%)	0 (0.00%)	7 (5.98%)	76 (64.96%)	34 (29.06%)	4.23	0.55
4. I think that I would need the support of a technical person to be able to use this system.	61 (52.14%)	53 (45.30%)	2 (1.71%)	1 (0.85%)	0 (0.00%)	1.51	0.58
5. I found that the various functions in this system were well integrated.	0 (0.00%)	0 (0.00%)	6 (5.13%)	59 (50.43%)	52 (44.44%)	4.39	0.59
6. I thought there was too much inconsistency in this system.	44 (37.61%)	62 (52.99%)	6 (5.13%)	5 (4.27%)	0 (0.00%)	1.76	0.74
7. I would imagine that most people would learn to use this system very quickly.	0 (0.00%)	2 (1.71%)	9 (7.69%)	79 (67.52%)	27 (23.08%)	4.12	0.6
8. I found the system very cumbersome to use.	59 (50.43%)	49 (41.88%)	9 (7.69%)	0 (0.00%)	0 (0.00%)	1.57	0.63
9. I felt very confident using the system.	0 (0.00%)	1 (0.85%)	22 (18.80%)	66 (56.41%)	28 (23.93%)	4.03	0.68
10. I needed to learn a lot of things before I could get going with this system.	39 (33.33%)	67 (57.26%)	9 (7.69%)	2 (1.71%)	0 (0.00%)	1.78	0.66

showcases the descriptive statistics of the responses to each question using frequency, percentage, means, standard deviation, mode, and median. Following the instructions provided in [45], the SUS score was calculated. The final SUS score for the specific application developed was 81.24, which placed it in the 90th–95th percentile with a grade A in terms of both usability and learnability. Students were mostly positive about using the application frequently (87.18%) and thought that it would be easy for others to learn to use the application very quickly (90.6%). Additionally, they found it easy to use themselves (94.02%) as its various functions were well integrated (94.87%) which led to them feeling very confident when using it (80.34%). On the other hand, students were mostly negative when asked if the application was unnecessarily complex (94.01%) and if they believed that there was too much inconsistency in the application (90.6%). When asked if they would require assistance from a technical person to use the application, the vast majority of students replied negatively (97.44%), and when asked if they had to learn many new things before being able to use the application, they also responded negatively (90.59%). Finally, when asked if they thought that the application was cumbersome to use, most students gave a negative response (92.31%).

Overall, the standard deviation of all questions was low, which indicates that the responses to each question were tightly clustered around each corresponding mean value. Additionally, based on the mode and median values, it can be inferred that most of the questions that students responded to positively regarding the learnability and usability of the application had a value of 4 and the ones to which they responded negatively, the values of mode and median were either 2 or 1. These outcomes in combination with the SUS score of the application demonstrate that the application was easy to learn and become familiar with and although several functions and elements were presented and used, its complexity level, consistency, and integration were appropriate rendering, thus, its usability high. Nonetheless, there remains room for adjustments and improvements.

4.3. Game Experience Questionnaire (GEQ) In-Game Module

Furthermore, the GEQ questionnaire was used to assess the competence, sensory and imaginative immersion, flow, tension, challenge, negative affect, positive affect as well as the overall positive and negative experience when using the application, playing the serious games, and carrying out the gamified learning activities. The GEQ in-game module and the questions related to the positive and negative experiences from the GEQ post-game module were adopted.

In Figure 5, the frequency and percentages of the responses related to the questions of the GEQ in-game module are presented while the related descriptive details are provided in

Table 2. Descriptive statistics of the GEQ in-game module.

Question	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree	Means	Standard Deviation
1. I was interested in the game story	1 (0.85%)	1 (0.85%)	2 (1.71%)	64 (54.70%)	49 (41.88%)	4.36	0.65
2. I felt successful	1 (0.85%)	9 (7.69%)	33 (28.21%)	55 (47.01%)	19 (16.24%)	3.7	0.86
3. I felt bored	39 (33.33%)	52 (44.44%)	19 (16.24%)	6 (5.13%)	1 (0.85%)	1.96	0.88
4. I found it impressive	2 (1.71%)	4 (3.42%)	19 (16.24%)	55 (47.01%)	37 (31.62%)	4.03	0.88
5. I forgot everything around me	16 (13.68%)	40 (34.19%)	31 (26.50%)	26 (22.22%)	4 (3.42%)	2.68	1.07
6. I felt frustrated	59 (50.43%)	48 (41.03%)	9 (7.69%)	0 (0.00%)	1 (0.85%)	1.6	0.71
7. I found it tiresome	53 (45.30%)	47 (40.17%)	12 (10.26%)	4 (3.42%)	1 (0.85%)	1.74	0.84
8. I felt irritable	63 (53.85%)	40 (34.19%)	9 (7.69%)	4 (3.42%)	1 (0.85%)	1.63	0.84
9. I felt skillful	4 (3.42%)	12 (10.26%)	23 (19.66%)	57 (48.72%)	21 (17.95%)	3.68	1
10. I felt completely absorbed	15 (12.82%)	28 (23.93%)	32 (27.35%)	28 (23.93%)	14 (11.97%)	2.98	1.22
11. I felt content	1 (0.85%)	5 (4.27%)	23 (19.66%)	61 (52.14%)	27 (23.08%)	3.92	0.82
12. I felt challenged	2 (1.71%)	14 (11.97%)	34 (29.06%)	47 (40.17%)	20 (17.09%)	3.59	0.97
13. I had to put a lot of effort into it	29 (24.79%)	62 (52.99%)	21 (17.95%)	3 (2.56%)	2 (1.71%)	2.03	0.83
14. I felt good	1 (0.85%)	1 (0.85%)	9 (7.69%)	67 (57.26%)	39 (33.33%)	4.21	0.69

. It is essential to mention how the components are calculated. Particularly, competence uses items 1 and 9, sensory and imaginative immersion uses items 1 and 4, flow uses items 5 and 10, tension uses items 6 and 8, challenge uses items 12 and 13, negative affect uses items 3 and 7, and positive affect uses items 11 and 14. The analysis of the components is displayed in

Table 3. Descriptive statistics of the GEQ in-game module components.

Component	First Item		Second Item		Total
	AVG	SD	AVG	SD	AVG	SD
Competence (GEQ items 1 and 9)	3.70	0.86	3.68	1.00	3.69	0.93
Sensory and imaginative immersion (GEQ items 1 and 4)	4.36	0.65	4.03	0.88	4.20	0.79
Flow (GEQ items 5 and 10)	2.68	1.07	2.98	1.22	2.83	1.16
Tension (GEQ items 6 and 8)	1.60	0.71	1.63	0.84	1.62	0.77
Challenge (GEQ items 12 and 13)	3.59	0.97	2.03	0.83	2.81	1.19
Negative affect (GEQ items 3 and 7)	1.96	0.88	1.74	0.84	1.85	0.87
Positive affect (GEQ items 11 and 14)	3.92	0.82	4.21	0.69	4.07	0.77

. When using the application, most students were interested in the game story (96.58%), found the application impressive (78.63%), and felt successful (63.25%), skillful (66.67%), content (75.21%), challenged (57.26%), and good (90.60%). On the other hand, most students replied negatively when asked if they had to put in a lot of effort (77.78%), if they found it tiresome (85.47%), and if they felt bored (77.78%), frustrated (91.45%), and irritable (88.03%). Once again, the standard deviation for each question was low which showcases that the responses to each question were tightly clustered around their corresponding mean value.

Moreover, the component of positive affect (4.01) was significantly higher than that of negative affect (1.85) which indicates the positive impact that the application had on students while simultaneously restricting the negative affect to the minimum and keeping their tension low (1.62). Of particular interest was the component of flow (2.83), with students having mixed opinions when asked if they forgot everything around them and if they felt completely absorbed. On the contrary, the sensory and imaginative immersion was high (4.20) which indicates that the application was capable of captivating students and keeping them engaged within an immersive augmented reality experience. Given that the application was an augmented reality one, these results were justified, as students not only focused on and were engaged in the application but also could observe and interact with their surrounding environment as well as communicate and collaborate with their peers. Hence, the significance of augmented reality to create mixed reality learning environments that combine the virtual and physical worlds was further highlighted. Additionally, the degree of competence (3.69) and challenge (2.81) demonstrated that the difficulty of the different levels and activities was appropriate for the targeted group.

4.4. Game Experience Questionnaire (GEQ) Post-Game Module

As far as the GEQ postgame module is concerned, the items related to students’ overall positive and negative gameplay experiences were adopted. More specifically, each component consists of 6 items. Positive experience is evaluated based on items 1, 4, 6, 7, 9, and 12 while negative experience is assessed based on items 2, 3, 5, 8, 10, and 11. The items, along with their related responses, are presented in Figure 6, and their descriptive statistics are showcased in

Table 4. Descriptive statistics of the GEQ post-game module. .

Question	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree	Means	Standard Deviation
1. I felt revived	1 (0.85%)	16 (13.68%)	49 (41.88%)	44 (37.61%)	7 (5.98%)	3.34	0.82
2. I felt bad	65 (55.56%)	49 (41.88%)	2 (1.71%)	0 (0.00%)	1 (0.85%)	1.49	0.62
3. I felt guilty	74 (63.25%)	42 (35.90%)	0 (0.00%)	0 (0.00%)	1 (0.85%)	1.39	0.59
4. I felt like a winner	4 (3.42%)	9 (7.69%)	31 (26.50%)	48 (41.03%)	25 (21.37%)	3.69	1
5. I found it a waste of time	60 (51.28%)	43 (36.75%)	11 (9.40%)	2 (1.71%)	1 (0.85%)	1.64	0.79
6. I felt energized	0 (0.00%)	9 (7.69%)	31 (26.50%)	61 (52.14%)	16 (13.68%)	3.72	0.8
7. I felt satisfied	1 (0.85%)	1 (0.85%)	17 (14.53%)	73 (62.39%)	25 (21.37%)	4.03	0.69
8. I felt that I could have done more useful things	28 (23.93%)	45 (38.46%)	25 (21.37%)	14 (11.97%)	5 (4.27%)	2.34	1.1
9. I felt powerful	6 (5.13%)	27 (23.08%)	46 (39.32%)	28 (23.93%)	10 (8.55%)	3.08	1.01
10. I felt regret	70 (59.83%)	40 (34.19%)	5 (4.27%)	2 (1.71%)	0 (0.00%)	1.48	0.66
11. I felt ashamed	82 (70.09%)	30 (25.64%)	4 (3.42%)	0 (0.00%)	1 (0.85%)	1.36	0.64
12. I felt proud	3 (2.56%)	13 (11.11%)	41 (35.04%)	48 (41.03%)	12 (10.26%)	3.45	0.91

. Based on their results, most students answered positively when they were asked if they felt revived (43.59%), like a winner (62.39%), energized (65.81%), satisfied (83.76%), and proud (51.28%) while they were neutral when asked if they felt powerful (39.32%). As a result, the overall game experience was positively evaluated (3.55) by students although the application was not meant to be used just for entertainment but for educational purposes too. Hence, some of its design elements focused more on improving learning activities and gains instead of providing a better game experience. In addition, the majority of students answered negatively when asked if they found using the application a waste of time (88.03%) or if they could have done more useful things with their time (62.39%). They also replied negatively when questioned if they felt guilty (99.15%), bad (97.44%), ashamed (95.73%), or regret (94.02%). As a result, the value of the negative gameplay experience component was really low (1.62) both in general and when compared to the positive one. The responses to each question were tightly clustered around their corresponding mean value as the standard deviation for each one of them was low.

4.5. Learning Motivations Promoted and Emotions Felt

To better understand the impact of the educational augmented reality application which integrated serious games and gamification elements, additional questions were set for students to respond to. Students regarded that the specific application promoted both intrinsic and extrinsic motivations (46.15%) and of the two, it mostly promoted and triggered intrinsic motivation (43.59%), which is particularly important for students to remain motivated and engaged, cultivate proper learning habits and behaviors, and pursue increased learning achievements (Figure 7). Following Plutchik’s wheel of emotions [47], the emotions mostly felt by students were also examined. Particularly, students mostly expressed joy (82.91%), surprise (67.52%), anticipation (36.75%), and trust (22.22%) while the negative emotions felt were very few. Therefore, based on the four pairs of emotions proposed by [47], the positive emotion was mostly felt in each pair. Thus, the impact that the application had on students and its ability to trigger positive emotions was demonstrated. These emotions can lead to a more enjoyable and satisfactory learning experience which, in turn, can result in better learning outcomes and increased motivation and engagement. These values are based on the responses of the 117 students who were involved and who could indicate more than one emotion that they felt. Hence, the results of each emotion represent the total number of students who felt the specific emotion (Figure 8).

4.6. Students’ Autonomy, Competence, and Relatedness—Self-Determination Theory (SDT)

Following the SDT [48], the potential of using gamification elements, serious games, and augmented reality to meet students’ innate needs for autonomy, competence, and relatedness was also explored. Specifically, Figure 9 presents students’ related responses while the descriptive statistics are showcased in

Table 5. Descriptive statistics.

Question	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree	Means	Standard Deviation
1. Were you able to act autonomously and have control of your decisions and actions?	0 (0.00%)	2 (1.71%)	4 (3.42%)	22 (18.80%)	89 (76.07%)	4.69	0.62
2. Were you competent in responding to the application requirements and challenges?	1 (0.85%)	4 (3.42%)	5 (4.27%)	24 (20.51%)	83 (70.94%)	4.57	0.8
3. Did you feel that you belonged to a group of people and related to them?	3 (2.56%)	8 (6.84%)	27 (23.08%)	49 (41.88%)	30 (25.64%)	3.81	0.98

. When students were asked if they could act autonomously and if they had control of their own decisions and actions within the application and games, the vast majority of them expressed a positive attitude (94.87%). Students were also positive (91.45%) about their skills, knowledge, and competence in responding to the requirements and overcoming the various challenges. It is worth noting that to both questions, the majority of students—76.07% for the first question and 70.94% for the second one—responded with “Strongly agree”. Furthermore, students also agreed (67.52%) that they felt like they belonged to a group of people that they could interact with, communicate with, and relate to. The use of augmented reality and the nature of games created the appropriate conditions for collaborative learning to occur in which students could actively engage and interact with their peers resulting in their feeling a sense of belonging. The responses to each question were clustered around their corresponding mean values.

4.7. Educational Benefits

The questionnaire also included questions about the educational benefits of the application. The perspectives and attitudes toward application use in educational settings were significantly positive, as can be seen in Figure 10 and

Table 6. Descriptive statistics of the educational benefits.

Question	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree	Means	Standard Deviation
1. Enhances the effectiveness of the educational process?	1 (0.85%)	1 (0.85%)	7 (5.98%)	63 (53.85%)	45 (38.46%)	4.28	0.69
2. Renders the learning process to be more enjoyable and intriguing?	1 (0.85%)	2 (1.71%)	8 (6.84%)	52 (44.44%)	54 (46.15%)	4.33	0.75
3. Creates student-centered environments?	2 (1.71%)	0 (0.00%)	10 (8.55%)	60 (51.28%)	45 (38.46%)	4.25	0.75
4. Helps me better comprehend the subjects taught	1 (0.85%)	2 (1.71%)	11 (9.40%)	50 (42.74%)	53 (45.30%)	4.3	0.78
5. Helps me improve my academic performance	2 (1.71%)	4 (3.42%)	14 (11.97%)	55 (47.01%)	42 (35.90%)	4.12	0.87
6. Increases computational thinking	1 (0.85%)	3 (2.56%)	9 (7.69%)	48 (41.03%)	56 (47.86%)	4.32	0.8

. The vast majority of students strongly agreed that the application could enhance the effectiveness of the educational process (92.31%) while simultaneously making it more intriguing and enjoyable (90.59%). Additionally, students agreed that the application created a student-centered learning environment (89.74%) that could improve their comprehension of the subjects taught (88.04%) and help them increase their academic performance and learning outcomes (82.91%). Finally, students positively evaluated the application as an educational tool that could significantly help them develop their computational thinking skills (88.89%) which are becoming more essential in the 21st century.

5. Discussion

The use of augmented reality in educational settings has been gaining in popularity as it can improve the overall educational process [49,50]. Augmented reality enables ubiquitous learning of high quality and interactivity and creates environments that support and promote inclusive education [51,52,53]. Several studies have highlighted the educational benefits that can be gained when adopting and integrating augmented reality in teaching and learning activities [31,54,55,56]. These positive learning outcomes and educational benefits can be further improved by combining augmented reality with gamification and serious games [14,28]. These findings are in line with the results of this study.

The combination of augmented reality with gamification and serious games can lead to several positive learning outcomes in education in general as well as in computer science education specifically. Previous studies demonstrated that through their use, useful educational tools [35] that create more interesting and enjoyable learning activities can be created [33,37]. These tools and applications are positively viewed by students and make the learning process more engaging and satisfying [36,39] which, in turn, improve students’ motivation and help them better understand the subject taught [34,37].

This study further confirms the results of previous studies but also expands across different dimensions in more detail. The participants were 117 higher education students from a 5-year degree department that specializes in information and electronic engineering. The paper-based questionnaire used consisted of 49 items and mostly adopted 1–5 Likert scales. Most students used their desktop or mobile devices to play games and either spend at least 2 h gaming or less than 30 min (RQ1). The devices used and games played as well as the daily activities of each individual, certainly affect these outcomes. Nonetheless, it can be said that the participants were all acquainted with playing digital games.

The majority of the students found the application easy to learn and use. This can be justified by the final SUS score, which was 81.24, as well as by the individual results of each question (RQ2). Hence, it can be inferred that despite the use of several technologies, functions, and modules when following a student-centered approach and taking students’ feedback into account throughout the design and development process, effective educational applications that are easily usable and learnable can be created. These results are in line with other recent studies that have also highlighted the positive impact that co-creating and co-designing effective learning experiences and educational applications with students can have in higher education [57,58,59,60].

Moreover, the results of the GEQ questionnaire were also positive. Students showcased a high level (4.20) of sensory and imaginative immersion which can lead to the conclusion that students were actively engaged and involved in the learning activities of the application. Students’ tension was low (1.62) which showcases that students were comfortable in using the application and carrying out the tasks. The level of difficulty and complexity in both the questions and activities was satisfactory as students’ competence (3.69) and challenge (2.81) were closer to neutral. The flow component also had a close to medium value (2.83). As the application focused on augmented reality experiences, students were able to communicate and interact with their peers and the surrounding environment while simultaneously being actively engaged in the learning activities of the application. These results highlight the potential of augmented reality to create collaborative learning environments that can lead to improvement in learning outcomes, academic performance, socio-emotional development, and social skills (RQ3). Several studies have also indicated the positive impact of collaborative learning on educational activities [61,62,63,64,65]. Students’ positive affect (4.01) was significantly higher than their negative affect (1.85) (RQ4). Thus, it can be inferred that the students’ socio-emotional effects of using the application were mostly positive. Additionally, the overall game experience was assessed as positive (3.55) and maintained the negative aspects to a minimum (1.62) (RQ4). Since the application mainly focused on learning activities and educational aspects and not entertainment, the results revealed that serious games not only yield positive learning outcomes but can also be a source of enjoyment and entertainment at the same time.

Moreover, based on the students’ viewpoints, the application triggered both their intrinsic and extrinsic motivations (46.15%) but mostly intrinsic motivations (43.59%) when comparing the two (RQ5). Several studies have commented upon and highlighted the importance of students’ motivation and its impact on the overall educational process and particularly within immersive learning environments [66,67,68,69,70,71]. Therefore, its suitability as an effective learning tool that enables students to be engaged and motivated, pursue increased academic achievements, and cultivate proper learning habits and behaviors was shown. When integrating educational applications, it is also important to understand students’ feelings while being engaged in learning activities. It is worth noting that the negative emotions that students experienced were minimal while the vast majority of students mostly expressed positive emotions, such as joy (82.91%), surprise (67.52%), anticipation (36.75%), and trust (22.22%), when using the application (RQ6). These outcomes showcase the significance of creating positive, interactive, and intriguing learning activities which, in turn, will lead to better learning outcomes as students remain motivated and engaged. The results also validate those of a previous study that focused on the public perspectives, sentiments, and attitudes regarding the use of augmented reality in education and demonstrated that the public mostly expresses positive emotions (e.g., anticipation, trust, and joy) and has a positive attitude toward the general and educational use of augmented reality [30].

Students’ innate need for autonomy, competence, and relatedness was also satisfactorily met as students’ responses to the related questions were significantly positive (RQ7). Therefore, it can be inferred that the application allowed students, who felt competent to respond to the requirements and overcome the challenges (91.45%), to act autonomously while having full control of their actions and decisions (94.87%) and simultaneously feeling a sense of belonging to a group of people with whom they related (67.52%).

Students also regarded that the application could enrich the teaching and learning processes and bring about educational benefits (RQ8). Specifically, the overwhelming majority of students strongly agreed that the application created a student-centered environment (89.74%) that improved the effectiveness of teaching and learning activities (92.31), made the learning process more enjoyable and fun (90.59%), and helped them understand the subjects and learning material better (88.04%) which, in turn, will lead to better learning outcomes and improved academic achievements (82.91%). As the focus was computer science education, it is also important to note that the application was also able to improve students’ computational thinking and critical thinking skills (88.89%) which are considered vital in the 21st century. All in all, the augmented reality application, which integrated gamification elements and serious games, was evaluated as an effective educational tool that can enrich the educational process, meet students’ educational needs, and yield several benefits in computer science education.

Although the overall results of the study were mostly positive, there were some limitations that should be mentioned. The participants were students of a computer science-focused department and were already familiar with using mobile applications. As such, their familiarity could have influenced their opinions regarding the usability and learnability of the application. Additionally, from a technical perspective, the focus on Android and iOS operating systems, as well as the use of mobile and tablet devices, to conduct the experiment can also be mentioned as additional technical limitations. Although these technical aspects have not influenced the results or findings of this study, there are certainly aspects that need to be improved in the future.

6. Conclusions

This study aimed to evaluate the impacts and benefits that the use of educational augmented reality can bring to computer science education when combined with gamification and serious games. Hence, it presented other related studies, went over the development process, described the experiment conducted, and showcased the research tools used. Moreover, it presented, analyzed, and discussed the results.

Based on the findings, it can be concluded that, when following a student-centered approach and actively involving students in the design and development process, effective educational tools can be created. The study also revealed that the developed application, which used augmented reality in combination with gamification elements and serious games, resulted in an easy-to-use and learn application that greatly improved students’ immersion, engagement, and motivation while providing them with adequate levels of challenge and complexity. Additionally, its use led mostly to positive effects and experiences and students expressed positive emotions (e.g., joy, surprise, anticipation, and trust) while maintaining the negative ones to a minimum. Following this approach, students’ innate need for autonomy, competence, and relatedness was also successfully met. The application yielded several educational benefits and was assessed as an effective educational means that can enrich and improve the educational process. Particularly, the application managed to create an interactive and inclusive learning environment that allowed students to learn in a more enjoyable and intriguing manner, increased their understanding of the subjects taught, improved their computer science self-efficacy, and triggered both their intrinsic and extrinsic learning motivations. As a result, the application has the potential to help students improve their academic achievements. Simultaneously, the application allowed students to feel a sense of belonging and cultivate their social skills. Elements of improved social–emotional development were also observed. Finally, through this approach, students’ critical thinking and computational thinking were improved.

Consequently, it can be inferred that the use of augmented reality, gamification, and serious games can positively affect education in general and computer science education specifically and meet students’ requirements. Their combination can result in the creation of inclusive and interactive learning environments that promote active learning, increase knowledge acquisition, academic achievements, and learning outcomes, encourage students’ active involvement, keep students motivated, allow them to cultivate both their cognitive and social-emotional skills, and enable them to improve their computational thinking, critical thinking, and social skills. Future work will focus on further improving the application and implementing it within each course using test and control groups as well as pre-tests and post-tests to evaluate its impact on academic performance, skills, and knowledge acquisition in different areas of computer science education.