Impact of Gamification on Students’ Learning Outcomes and Academic Performance: A Longitudinal Study Comparing Online, Traditional, and Gamified Learning

Abstract

This study aims to examine the influence of gamification in students’ learning outcomes and academic performance. A longitudinal study was conducted to compare students’ academic performance in online learning (2020–2021), traditional learning (2021–2022), and gamified learning (2022–2023). The longitudinal study lasted 3 years and a total of 1001 higher education students were involved. Three research questions were set to be explored and students’ viewpoints and experiences were also examined through a questionnaire of 20 questions. This study follows a quantitative research approach. The data refers to students’ academic performance, success rate, excellence rate, withdrawal rate, engagement, motivation, and perspectives. In the laboratory part of the course, gamified learning yielded better outcomes over online learning and traditional learning in success rate (39% and 13%), excellence rate (130% and 23%), average grade (24% and 11%), and retention rate (42% and 36%) respectively. In the theoretical part of the course, gamified learning resulted in better outcomes over online learning and traditional learning in success rate (19% and 14%), in excellence rate (125% and 79%), and in average grade (25% and 12%) respectively. In the overall course, gamified learning yielded better outcomes over online learning and traditional learning in success rate (14% and 14%), in excellence rate (122% and 70%), and in average grade (25% and 17%) respectively. The highest increase was observed in students’ excellence rate. Students highly regarded gamification as an effective educational approach that can increase their learning outcomes, engagement, productivity, and motivation and trigger both their both intrinsic and extrinsic motivation. The learning experience become more enjoyable and students’ basic needs in terms of autonomy, competence and sufficiency, and relatedness and sense of belonging were met. Traditional learning also resulted in better learning outcomes when compared to online learning. Gamification emerged as an effective learning approach which leads to improved learning outcomes and academic performance, learning motivation, engagement, and retention rate over online learning and traditional learning in both theoretical and applied course settings.

1. Introduction

In recent years, the educational domain has undergone several drastic changes that have affected all educational stakeholders. Specifically, teaching and learning activities had to transition to online learning during the COVID-19 pandemic and then revert to face-to-face learning [1]. However, during this period, educational stakeholders’ skills and knowledge have improved which led to the adoption and integration of new technologies, methods, and approaches in classrooms which, in turn, brought about new opportunities to transform and enrich the educational process [2,3].

In particular, besides all the negative implications that the COVID-19 pandemic brought about, it also highlighted weaknesses, barriers, and limitations of the educational system [4], although online learning had already started being more widely adopted and used [5]. To overcome the unprecedented circumstances that arose, teaching and learning had to rapidly transition to fully online environments [6,7]. Online education, also referred to as distance education, involves the use of digital technologies and devices to provide ubiquitous educational experiences and enables educational stakeholders to communicate and stay in touch even while in distance [8,9]. Hence, online education aims at creating effective learning environments that promote students’ flexibility, responsibility, and active participation without the direct and continuous supervision of teachers [6]. Despite the fact that some institutions were familiar with conducting well planned and organized online lessons, in its entirety, this form of education was characterized as emergency remote teaching and learning [10,11] due to the short-term and prompt changes to educational activities that involved different technological solutions as a means to continue the educational process across all educational levels [6,10].

Due to the potential that online learning brings to education [12,13,14], several studies have examined the impact of online learning before, during, and after the pandemic [15,16,17,18,19]. The results of these studies varied since there are multiple aspects that can influence students’ learning outcomes. However, all studies agreed that online learning can bring new opportunities in teaching and learning. Moreover, these educational innovations have brought about new pathways to explore toward achieving sustainability in the context of education as well as quality education which are integral aspects of the sustainable development goals set by the United Nations in the 2030 Agenda for Sustainable Development which strives to achieve prosperity and peace for the environment and man kind [20]. The role of technology and innovation is vital in achieving the sustainable development goals [21]. Hence, the efforts put into enriching and advancing education and into pursuing educational innovation by all stakeholders should continue even after the pandemic.

Moreover, recent systematic literature review studies [2,11,22,23] that examined the impact of the COVID-19 pandemic in education revealed that the education community managed to adapt their practices, learn new skills, and improve their knowledge. As a result, the post-COVID-19 education also disclosed several changes [24]. When the educational process transitioned back to traditional learning, educators tried to integrate their new practices and approaches in their face-to-face classrooms [25,26]. In the context of this study, traditional learning or face-to-face learning refers to the educational experiences in which both students and teachers are physically present in the same classroom, can directly engage in teaching and learning activities, and can communicate and interact in person [27,28].

Among the different approaches being adopted in education, gamification in particular is gaining ground [29,30]. Gamification is an educational approach which utilizes game mechanisms and elements and applies them in non-game related settings to actively engage and motivate students [31,32,33,34,35]. Recent studies [36,37,38,39] have highlighted the potentials and benefits that the adoption and integration of gamification can bring in various subjects and across all educational levels. Having acquired a new skill-set, educators are actively trying to find ways to effectively integrate gamification into their classes as it can transform and enrich the educational process.

In all learning environments, learning motivation constitutes an integral part of students’ learning experience, satisfaction, and performance [40]. Motivational theories, such as the Flow theory [41] and Self-determination theory [42], go in more detail on how specific attributes, such as autonomy, competence, relatedness, enjoyment, and engagement can influence learning motivation. Additionally, studies have further examined how motivation affects students’ performance in various contexts including traditional learning [43,44], online learning [45,46], as well as gamified learning [47,48,49]. The results of these studies reveal that learning motivation is a vital element in all learning environments as it contributes to effective learning and to achieving better learning outcomes. Hence, adopting approaches and techniques that increase students’ learning motivation is encouraged.

Furthermore, recent studies have explored how students’ learning outcomes compare between online learning and traditional learning before and during the pandemic or during and after the pandemic [50,51,52,53,54,55]. As the settings between the two environments differ and several factors can influence students’ learning performance, studies report mixed results related to the effectiveness of the methods when they are compared. Due to the potential benefits that the integration of gamification can bring in education, recent studies [35,56,57,58,59] have also examined how it affects students’ learning performance and motivation in computer science courses. Taking the complex nature of computer science courses into account, the ability of gamification to enrich the educational process further highlights its use as an effective learning approach. The role of gamification in online learning environments has also been examined by some recent studies which resulted in positive outcomes [60,61,62].

Based on the aforementioned, it is clear that several studies have been conducted throughout the years which looked into the effects of online learning, face-to-face learning, and gamified learning. However, these studies focus on examining a specific learning environment and do not directly make comparisons between different learning environments and how they can influence students’ learning outcomes. Additionally, to the best of our knowledge, there has not been any longitudinal study that compares students’ learning outcomes among these three learning environments. Therefore, there is a clear need to examine how different learning environments affect students’ academic performance. This is particularly true in more applied courses since they were the ones mostly affected by these changes.

To address this gap in the existing literature, this study aims to examine how gamification can impact students’ learning outcomes and academic performance in comparison to traditional learning and online learning. Specifically, the results and outcomes of a longitudinal study spanning 3 years from the academic year 2020-2021 to the academic year 2022–2023 are presented. The first part (2020–2021) involves teaching and learning in online learning environments during the COVID-19 pandemic, the second part (2021–2022) involves traditional learning, and the third part (2022–2023) involves gamified learning. To better understand the influence of gamification, the course selected to be used was both technical and theoretical in the context of computer science in higher education. The course involved both theoretical (lectures) and applied (laboratories) lessons, which were separately examined and evaluated.

The main research questions (RQ) set to be examined were:

1.

Which is the most effective approach among online learning, traditional learning, and gamified learning in terms of learning performance?

2.

How does the integration of gamification affect students’ learning performance?

3.

How do students perceive the use of gamification in education?

The main contributions of this study can be defined as:

• A longitudinal examination of three different approaches and their impact on students’ learning outcomes.
• The definition of a scoring algorithm to be used in gamified learning settings.
• The examination of students’ viewpoints and attitudes toward adopting and integrating gamification in education.

2. Materials and Methods

This section goes over the main materials and methods used during this study. Specifically, the details of the course which the study focused on are presented, and the examination process is explained. Additionally, the settings applied in each year and approach are detailed and the scoring algorithm created to be used in conjunction with gamification is presented.

2.1. Course Details

The course involves an introduction to the subject of “Operating Systems”, and its main objective is the understanding of the general principles of Operating Systems through the use and programming in UNIX. The course also aims at helping students examine and get to know an operating system that was created for programmers as well as to understand the philosophy that underlies an operating system “behind” graphical environments. During the course, students have the opportunity to comprehend and explore the use of a shell and how it interacts with the operating system. The course strives to help students understand how an operating system works and how they can utilize it. During the course, students have the opportunity to learn about the powerful tools a programmer has by using an operating system shell. Finally, students familiarize themselves with the philosophy of: I do complex actions by combining simple autonomous commands that communicate with each other. More information about the course contents can be found on the course web page [63].

2.2. Evaluation Process

The course consists of two parts, namely the theoretical and the laboratory parts, which take place on a weekly basis. The laboratory part of the course includes a series of laboratory exercise sets. The students try to solve the exercises at home after having attended the theory lectures. A laboratory class is conducted in which each exercise set is explained in more detail and additional examples and exercises are provided. Since the capacity of the laboratories is limited, students are divided into groups/classes of about twenty students. During these laboratory classes, students practice and have the opportunity to solve their questions and any unsolved exercises and delve into more difficult and complex exercises. Additionally, students are examined on the course material from the previous exercise sets. During the second laboratory meeting (fourth week), students are examined on Laboratory Test 1 (LT1) which covers the material of the first exercise sets. In total, there are four laboratory tests (LT). The tests are performed by using a web based application called OnLine Evaluation (OLE) which was developed specifically for this course.

The OLE application offers the ability to the instructor to define scripts that generate exercises. For example, one of the exercises is about the Unix file system and paths. The instructor creates a script that generates a random file system. Then, random files are selected, and the students are asked to type the paths for the aforementioned files. The paths can be checked automatically whether they are correct or wrong. The instructor has created a corresponding script that calculates the grade for each answer. The way the grade is calculated depends on the type of question. For example, in a question asking for a file path, the grade can be true or false, or alternatively, it can be analogous to the distance from the correct path. The grading is calculated automatically for the first three LTs. The students can see their grades during the LT, but they have a limited number of tries for each exercise (e.g., three to ten) depending on the nature and difficulty of each exercise. Finally, each LT has a time restriction.

The weight for each LT is based on the importance, complexity, and difficulty level. The weights used during our experiment were: 0.12 for LT1, 0.18 for LT2, 0.20 for LT3 and 0.50 for LT4. In comparison to LT3 and LT4, LT1 and LT2 include easier exercises that examine basic concepts. The first two tests are purposefully kept this way, to avoid disappointing the students with the level of difficulty and complexity. The third LT is of great difficulty and complexity, much higher in proportion to the previous two. This is done to show students that completing the course needs more effort and dedication. This is the reason why the weight percentage of LT3 was kept relatively low compared to its difficulty level. Finally, LT4 is the final laboratory test, which includes all the course material covered throughout the semester. As it combines concepts and requires knowledge of all the educational material taught, LT4 is the most important test to define students’ understanding. Because of the importance, difficulty, and complexity of LT4, the grading for LT4 is manually done by the professors.

After completing the laboratory part of the course, students are examined on the theoretical part. Their final grade is a weighted average of LT and theoretical examination (TE). The weights are $1/3$ for LT and $2/3$ for TE.

2.3. Experiment Participants

The experiment process is divided into three phases, namely online learning, traditional learning, and gamified learning. The course is taught during the second year of the undergraduate study program. The number of students in the course varies per year as it depends on the number of currently active students, on the number of admitted students at the department on the previous year, which is defined by the Ministry of Education, and on the available resources. Therefore, although there are differences in the number of students participating, measurements were taken, as explained further below, to ensure that the same settings were applied for each phase of the experiment.

Furthermore, to ensure that the outcomes of the study will directly reflect students’ learning outcomes and academic performance, only the students who actively participated in the course (e.g., those who took at least the first laboratory test) were counted and not the total number of students who initially declared that they would attend this course but chose not to in the end. More details regarding this fact are provided in the result analysis section. Based on the aforementioned, a total of 421 students actively participated during online learning, 331 students during traditional learning, and 249 students during gamified learning. Therefore, throughout the three years that the experiment lasted, a total of 1001 higher education students from the Department of Information and Electronic Engineering, International Hellenic University, Greece undertook this course and were involved in the experiment.

2.4. Scoring Algorithm

This subsection goes over the scoring algorithm which was defined and developed for the needs of the gamified learning. It should be mentioned that the score is different from the grade for each LT. The grade is computed based on students’ last submission, independently of the number of attempts they made. However, in a group, many students will have a grade of $1 (=10/10)$. Hence, for the gamification process, there was a need to distinguish between these cases. This was the reason why the number of tries/submissions was also taken into account. Therefore, given a set of laboratory tests $LT=\{L{T}_1,L{T}_2,...\}$. Each LT l consists of a set of questions $Q\left(l\right)$ (Equation (1)), and for each question, there is a weight $w_i$. The weights vector is $W\left(l\right)$ (Equation (2)). Additionally, for each question, the maximum number of tries is defined as $t_i$ while $T\left(l\right)$ (Equation (3)) is the set with the maximum number of tries for the LT l:

$$\begin{array}{cc}\hfill Q\left(l\right)& =\{q_1,q_2,\cdots ,q_n\}\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill W\left(l\right)& =\{w_1,w_2,\cdots ,w_n\},\sum _{\forall e\in LT\left(l\right)}{w}_l=1\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill T\left(l\right)& =\{t_1,t_2,\cdots ,t_n\}\hfill \end{array}$$

(3)

The set of students is defined as $S=\{s_1,s_2,\cdots .\}$. For each student s and for each question q, there is a grade $g(s,q)\in$ [0–1]. In addition, $g(s,q)=null$ if and only if the student did not try to answer the specific question at all.

The grade $g(s,l)$ of a student s for a laboratory test $l\in LT$ is computed as follows:

$$\begin{array}{ccc}\hfill g(s,l)& =\sum _{q\in Q\left(l\right)}g(s,q)\ast w_q\hfill & \hfill \mathrm{if}\exists g(s,q)!=null\end{array}$$

(4)

$$\begin{array}{ccc}\hfill g(s,l)& =null\hfill & \hfill ifg(s,q)=null\forall q\in Q\left(l\right)\end{array}$$

(5)

The grade of a student is null if they have not submitted any answers for the specific LT. This can be interpreted as their not taking part in the specific LT. Given the student’s s grade for a question q and the maximum number of possible tries $t_q$ for the specific question, a score $R(s,l)$ is defined as follows:

$$R(s,l)=\sum _{q\in Q\left(l\right)}\left(g(s,q)\ast w_q\ast \left((1-k)+k\ast \frac{t_q-t(s,q)}{t_q-1}\right)\right)$$

(6)

In Equation (6), k represents the tries factor. If k is set to 0, then $R(s,l)=g(s,l)$ will hold. For the purposes of this study, we set $k=0.5$. Having a k of $0.5$, the score R will be equal to g if and only if the student makes only one try per exercise. On the other hand, if the student used the maximum number of tries for all the exercises, their score R will be $0.5g$. Hence, R will have values between $[0\cdots 1]$, with the case of 1 to be almost impossible, since this corresponds to a student that got excellent ($g=1=10/10$) in all exercises with only one try for each question.

Furthermore, to incorporate the difficulty of each LT into the grading system, the $PR$ (percentile rank) for each student was defined. Given that $S=\{s_1,s_2,\cdots .\}$ is the set of students and $S\left(l\right)$ is the set of students that participated in LT l, we define $CF(s,l)$ in range $1\le CF(s,l)\le \left|S\right(l\left)\right|$ as the number of students b with $R(b,l)\le R(s,l)$ including student s. Based on $CF$, the $PR$ (percentile rank) is defined as:

$$PR(s,l)=\frac{CF(s,l)}{\left|S\right(l\left)\right|}$$

(7)

According to the aforementioned, it is obvious that $0<PR(s,e)\le 1$. $PR$ will be equal to 1 for students that scored the maximum R in a group for an LT.

Given the real score R and the percentile rank $PR$, the final game score $C(s,l)$ for the student s in the laboratory test l is defined as:

$$C(s,l)=\frac{PR(s,l)+R(s,l)}{2}$$

(8)

2.5. Gamification Elements

Following the findings of recent studies [64,65,66,67] that highlighted the most widely used gamification elements in educational settings and given the nature of the course and the environment, specific gamification elements were selected. In particular, this study involved the use of leaderboards, badges, scores, points, and nicknames. These elements were present in the course platform and the examination/evaluation system and played an active role during both theoretical and laboratory lessons. The rewards associated with the integration of gamification elements and students performance on the laboratory tests were both at every single class level and at a level including all classes as a whole. In this way, students were encouraged and motivated to perform better not only in comparison with their peer classmates but also with all peer students that undertook the course. The results announced to students indicated their grades based solely on their performance while their gamification ranking involved the elements specified in the scoring algorithm. The unified and shared among all classes leaderboard and the leaderboards for each class were available to all students. However, students’ randomly generated nicknames were used instead of their real ones. Each student could see their nicknames when they logged into the course platform. Based on their ranking, students received different badges which appeared on the leaderboards and on the course platform until the next examinations.

2.6. Main Variables

In the context of this longitudinal study, students’ learning outcomes, viewpoints, and experiences assessed following a quantitative approach and incorporating a descriptive data analysis. Specifically, using the questionnaire presented in the next subsection which consists of Likert-scale and closed-questions, the integration of gamification was evaluated focusing on students’ viewpoints, learning motivation, basic needs, potential benefits, and future use of gamification.

Furthermore, during the three phases of the study, data related to students’ grades, excellence rate, success rate, and withdrawal/retention rate was collected. This data was collected for the laboratory and theoretical parts of the course as well as of the overall course in all three learning environments examined, that is online, traditional, and gamified learning. The process of how students’ grades were calculated was previously explained. However, it is worth noting that grades above 8/10 are regarded as excellent and that bonus grades received through the gamified learning approach were not taken into account. Hence, excellence rate refers to the number of students whose grade in each environment was equal to or greater than eight (8) out of ten (10). Additionally, success rate refers to students’ meeting the requirements to pass the course (both laboratory and theoretical parts). However, students’ success rate of LT1 was also examined. Finally, students’ withdrawal/retention rate refers to students’ either actively participating in all examinations and parts of the course throughout the semester or withdrawing from it in the semester.

2.7. Students’ Viewpoints and Experiences Questionnaire

To examine students’ experience and viewpoints regarding the adoption and integration of gamification in education, an ad hoc questionnaire was developed. The questionnaire used 5-point Likert-scale (1 = Strongly disagree...5 = Strongly agree) questions as well as closed-questions and contained 20 questions about the following five categories:

1.

Category 1: Students’ viewpoints regarding the integration of gamification in education;

2.

Category 2: Students’ learning motivation;

3.

Category 3: Students’ basic needs;

4.

Category 4: Students’ viewpoints regarding the benefits of integrating gamification in the specific course;

5.

Category 5: Students’ perspectives regarding the future use of gamification in education.

The five categories, along with all their related questions and items, are presented below:

1.

Category 1 involved 9 items and students had to answer the following questions using a 1–5 Likert-scale: “Having experienced the application of gamification in education, do you believe that it can...”:

• increase the effectiveness of the educational process?
• improve academic performance and learning outcomes?
• promote motivation for learning?
• promote active participation and engagement?
• increase learning productivity?
• offer more enjoyable learning experiences?
• reinforce focus on the achievement of educational goals?
• create a competitive environment?
• cause negative feelings due to competition involved?

2.

Category 2 involved 1 item and students had to answer the question, “Having experienced the application of gamification in education, which motivations do you believe it mostly promoted?” selecting one of the following responses:

• No motivation
• Intrinsic motivation only
• Extrinsic motivation only
• Both intrinsic and extrinsic motivation

3.

Category 3 involved 3 items and students had to answer the following questions using a 1–5 Likert-scale: “Having experienced the application of gamification in education, do you believe that its use in educational settings can promote...”

• autonomy
• competence and sufficiency
• relatedness-a sense of belonging

4.

Category 4 involved 4 items and students had to answer the following questions using a 1–5 Likert-scale: “Having experienced the application of gamification in education, do you believe that its use in the educational process can...”

• offer educational benefits?
• render the educational process more interesting?
• promote learning motivation for success and intense effort through its reward systems?
• create learning environments which have students in their core?

5.

Category 5 involved 3 items and students had to answer the following questions using a 1–5 Likert-scale: “Having experienced the application of gamification in education, do you believe that gamification...”

• should be used again in this course?
• should be applied in other courses as well?
• can enrich higher education?

3. Research Design-Experiment Process

The initial goal was to create an educational platform for the course and an examination-evaluation system to assess its effectiveness and how it could have been further improved to offer lessons of higher quality. Hence, during the conceptualization phase (2019–2020), efforts were put to create educational material for both the theoretical and applied part of the course. It is worth mentioning that even prior to the pandemic, the course material was already available in online form to cope with distance education requirements; thus, even then the educational material had to be suitable for both traditional and online learning. Video lectures were also created to assist students throughout the course.

An interactive educational platform was developed for the specific course. On this platform, students had access to all the required material, specifications of the course, indicative exercises and examples, slides, and video lectures throughout the semester. The course-related material was only accessible to registered students. Furthermore, due to the technical and hands-on nature of the course, an examination-evaluation system was created. The scope of this system was to create semi-randomized questions at the same level of difficulty as well as providing students with feedback and grading in real-time. As it was planned to launch this experiment during the spring semester of the academic year 2020–2021, a pilot study, regarding the usability of the developed educational material, platform, and system, was conducted with students who had already taken the course. Students’ feedback was taken into account to make further adjustments.

However, the unprecedented circumstances that were caused by the COVID-19 pandemic led to a rapid transition to fully online learning. At this point, it was assessed that besides the several negative aspects, it could have been beneficial to assess the performance of the proposed approach as well as students’ learning outcomes in online learning environments to compare them with those deriving from different learning environments in the future. Hence, during the fall semester of 2020-2021, the prerequisites for the experiment presented in this study were set. The experiment was separated into three phases one for each different learning environment and approach. Phase 1 (2020–2021) involved teaching and learning in online learning environments, Phase 2 (2021–2022) involved teaching and learning in traditional face-to-face learning environments, and Phase 3 (2022–2022) involved teaching and learning in gamified learning environments. Hence, this longitudinal study lasted for three years. The detailed methodology is presented in Figure 1 and explained in the following subsections.

3.1. Phase 1 (2020–2021)—Online Learning

This phase involved the transition to online learning during the spring semester of 2020–2021. Specifically, further improvements and modifications to the course educational platform and the examination-evaluation system were made to address teaching and learning in remote online circumstances but without their core being changed. Appropriate measures were taken to ensure integrity during online examination. For example, besides the changes to the system, students were also required to share their screen during examinations. Students’ shared screens were only accessible to professors during the examination and not to other participants. This was made possible by having students undertake the examination in customized virtual environments. Moreover, theoretical lessons and laboratory lessons ran smoothly in online environments. Although it was possible to include many students in each virtual classroom, laboratory classes had a maximum of 22 students to achieve better learning outcomes. Therefore, multiple classes were created.

Given the examination specifications provided earlier, three laboratory tests took place on a biweekly basis starting from week 4. Hence, laboratory test 1 took place during week 4, laboratory test 2 was conducted in week 6, and laboratory test 3 took place during week 6. For these three laboratory tests, students’ responses were automatically corrected through the examination-evaluation system. Week 11 was used as a revision week for all classes. On week 12, after the revision, the final laboratory examination was carried out. Besides it being more difficult and requiring skills and knowledge from all sections of the course, this time students’ responses were evaluated by professors. The final theoretical examination took place during the institutional examination period (weeks 14–16) and was also evaluated by professors.

3.2. Phase 2 (2021–2022)—Traditional Learning

This phase involved the transition to face-to-face learning after the COVID-19 pandemic during the spring semester of 2021–2022. To address the change in the learning environment, slight adjustments and improvements were made on the course educational platform and examination-evaluation system, once again, without their core being changed. Both theoretical and laboratory lessons were taken in a face-to-face manner. Laboratory classes consisted of up to 22 students. The examination process was kept similar.

Specifically, three laboratory tests took place on a biweekly basis starting from week 4. Hence, laboratory test 1 took place during week 4, laboratory test 2 was conducted in week 6, and laboratory test 3 took place during week 6. For these three laboratory tests, students’ responses were automatically corrected through the examination-evaluation system. Week 11 was used as a revision week for all classes. On week 12, after the revision, the final laboratory examination was carried out. Besides it being more difficult and requiring skills and knowledge from all sections of the course, this time students’ responses were evaluated by professors. The final theoretical examination took place during the institutional examination period (weeks 14–16) and was also evaluated by professors.

3.3. Phase 3 (2022–2023)—Gamified Learning

This phase involved the transition to gamified learning during the spring semester of 2022–2023. To address the change in the learning approach, the course educational platform was further enriched to handle gamification elements, such as leaderboards, points, badges, nicknames, etc. Additionally, a scoring algorithm (Equation 8) was developed which took into account several aspects. Hence, the examination-evaluation system was modified to also account for the adoption of gamification elements and parameters which are used in the scoring algorithm. Both theoretical and laboratory lessons were taken in a face-to-face manner but, this time, gamification elements were also included. Laboratory classes, in which the use of gamification was easier to materialize, consisted of up to 22 students.

On week 2 it was announced to students that the gamification approach would be used and all the requirements, specifications, and rules were provided. The examination process was, once again, kept similar as the addition of gamification elements did not directly affect the conduct of each examination. Specifically, three laboratory tests took place on a biweekly basis starting from week 4. Hence, laboratory test 1 took place during week 4, laboratory test 2 was conducted in week 6, and laboratory test 3 took place during week 6. For these three laboratory tests, students’ responses were automatically corrected through the examination-evaluation system. A day after each examination was completed, the leaderboards which used nicknames instead of students’ real names were updated and students were informed. Week 11 was used as a revision week for all classes. On week 12, after the revision, the final laboratory examination was carried out. Besides it being more difficult and requiring skills and knowledge from all sections of the course, this time students’ responses were evaluated by professors.

The final theoretical examination took place during the institutional examination period (weeks 14–16) and was also evaluated by professors. After the completion of both final exams (theoretical and laboratory) and before announcing students’ final grades for each part as well as their final course grade, students were asked to complete a questionnaire regarding their experience and viewpoints about the integration of gamification in education.

Besides the presentation of the three phases, there are additional aspects that should be considered. Specifically, several actions were taken to ensure further credibility and validity of the results. The course material remained the same throughout the three years and so did the structure of the examinations. However, the questions of the examinations were different but of the same nature. Students of all three phases were aware of the whole process and structure of the course. Hence, all students participated in the course on an equal ground. Additionally, the two professors teaching the course remained the same and followed the same methods of teaching for all three phases. In this way, the influence of having different individuals teaching the course was eliminated. The core of the course educational platform and the examination-evaluation system remained the same with only minor adjustments and improvements being made to account for the change in the learning environment. By doing so, the platform and system themselves could not have affected the learning outcomes.

Moreover, the results presented refer only to students who took the course for the first time. This fact ensured that students who had previous experience with the course did not affect the outcome, as the performance of these students was not included in the examination or the results. The equipment and software used during the course were the same for the traditional and gamified learning. Hence, factors related to hardware and software aspects could also not influence the outcomes

Based on the aforementioned, the only thing that changed during the three phases was the learning environment and approach used. Due to this fact, the results of this study can be directly attributed to the effectiveness of online learning, traditional face-to-face learning, and gamified learning. Finally, it must be mentioned that all the results and grades displayed and analyzed are based on students’ exact performance and grades received in the examinations. Bonus grades received through the gamified learning approach were not considered as they would not realistically represent students’ outcomes and the impact of the gamification approach. Hence, the decision to only include students’ examination grades was taken to provide more objective results.

4. Result Analysis

4.1. Students’ Learning Achievements and Outcomes Comparison

This subsection goes over the result analysis of students’ learning outcomes and academic performance. Due to the nature of the data and it involving students’ academic performance, success rate, excellence rate, withdrawal rate, engagement, motivation, and perspectives, this study follows a quantitative approach. In

Table 1. Course Statistical information.

	Online Learning 2020–21		Traditional Learning 2021–22		Gamified Learning 2022–23
Course Registrations	510		399		357
Laboratory Registrations	505		395		301
LT1 Participants	421	100%	331	100%	249	100%
LT2 Participants	399	95%	313	95%	234	94%
LT3 Participants	373	89%	288	87%	222	89%
LT4 Participants	332	79%	264	80%	212	85%
Students withdrew from Lab	89	21%	67	20%	37	15%
Laboratory Success	211	50%	209	63%	178	71%
Theory Exam Participants	148		218		185
Theory Exam Success	116	78%	171	78%	166	90%
Course Success	116	78%	171	78%	166	90%
Course Success relative to LT1		28%		52%		67%
Laboratory Average Grade	4.2		4.7		5.3
Theory Average Grade	5.0		5.5		6.6
Course Average Grade	5.6		6.0		7.0

, statistical information is presented. The number of course registration is always the biggest number of all. There are a lot of students that just click the button of the online registration system since there are no drawbacks. Students are registered even if they do not intend to attend the course. The second line in Table 1 is the number of registrations in the laboratory part of the course. During the laboratory registration, students are registered to a class with fixed days and times within the week. However, even in the laboratory part of the course, a student can register and not attend without any consequences. The number of LT1 participants shows the real number of students that started to attend the laboratory part of the course. Therefore, the number of LT1 participants will be used as the baseline for the percentage calculation in the following cases.

In Greek universities, it is common for students to withdraw from a course during the semester, even though they initially intended to attend it. This situation is usually represented by a null or zero grade, as the withdrawal is not official but informal. This phenomenon is because students do not face any consequences, neither academic nor financial, when withdrawing from a course during the semester. According to Table 1, it is clear that students are withdrawing throughout the semester. Finally, about $80\%$ of students who started the laboratory part of the course completed it by participating in LT4 which is the final LT. In the last column, it is observed that the withdrawal percentage decreased to $15\%$ from about $20\%$ that was during the traditional and online learning periods.

After completing the laboratory part of the course successfully, students take the written examination on the theoretical part. Successful laboratory completion means that students’ average LT grade is greater than or equal to 4. It can be observed in Table 1 that the success rate of $50\%$ for the online learning period increased to $63\%$ during the traditional way of learning and finally increased to $71\%$ for gamified learning. In

Table 2. Laboratory grade distribution.

Method\Grade	1	2	3	4	5	6	7	8	9	10	Succ.	Excel.
Online Learning	21%	10%	18%	16%	15%	9%	4%	3%	3%	1%	51%	7%
Traditional Learning	20%	5%	11%	24%	10%	11%	5%	5%	5%	2%	63%	13%
Gamified Learning	13%	7%	9%	16%	18%	11%	10%	8%	4%	3%	71%	16%

, the detailed grade distribution is presented. It must be mentioned that all grades were rounded down to generate distinct values. Based on the graphical representation of the distributions, it is evident that there is a transposition of the curves to higher grades.

In Figure 2, Figure 3 and Figure 4, the (a) subfigures plot the distribution of grades for each LT and the (b) subfigures plot the cumulative distribution are presented. Axis y corresponds to the number of students and axis x to the grades rounded to the smaller integer. The distributions have similar shapes for all periods because it was made sure that LT in all three years will be different in terms of questions but similar in terms of difficulty and complexity.

• LT1 and LT2 have similar distributions for all periods. The difference is that in gamified learning the number of students graded by 10 in LT2 increased, although LT2 is slightly more difficult than LT1. This means that the students focused on and pursued to gain an excellence grade and to improve their knowledge and skills.
• In the cumulative distribution graphs, the curves distance on the final point (zero) indicates the withdrawal rate during the semester. This distance is decreasing from online learning to gamified learning. Therefore, it can be inferred that during gamified learning fewer students withdrew from the course.
• The majority of grades in LT1 during all periods are located on 9–10 area.
• The number of students graded by $0/10$ to $0.99/10$ in LT1 is analogous and indicates the number of withdrawals at LT4. Values for LT1 at 0 are 65%, 33%, and 26% for online, traditional, and gamified learning respectively. Withdrawal rate (LT1–LT4 participants) is 89%, 67%, and 37% for online, traditional, and gamified learning respectively (Table 1).
• There are a lot of students at 0/10–1/10 for LT4 during online learning (Figure 2. This number decreased during traditional learning (Figure 3) and became extremely low during gamified learning (Figure 4). This fact highlights the potential of gamification to promote students’ learning motivation, engagement, and retention.

In Figure 5, the final laboratory grade distribution is presented and in Table 2 the corresponding values are depicted. Based on the results, it is evident that gamification not only improved the overall success rate (up to $71\%$) but also contributed significantly to the improvement of the excellence rate, that is grades above $8/10$. The excellence rate is shown in the last column in Table 2.

Although the gamification approach was mostly applied only to the laboratory part of the course, it also helped students obtain a better understanding of the course contents. This is evident when looking at the results of Figure 6 in which the distribution of grades for only the theoretical part of the course is presented.

Table 3. Theory grade distribution.

Method\Grades	0	1	2	3	4	5	6	7	8	9	10	Succ.	Excel.
Online Learning	5%	9%	8%	0%	16%	19%	12%	10%	5%	7%	5%	78%	18%
Traditional Learning	2%	9%	10%	0%	9%	15%	18%	14%	8%	8%	7%	78%	23%
Gamified Learning	1%	2%	7%	0%	11%	9%	15%	14%	14%	14%	14%	90%	41%

presents the theory grade distribution in detail. Once again, an impressive improvement in the success rate (90% from 78%) is observed. The most significant outcome is the improvement in the excellence rate (grades $\ge 8/10$) which becomes two to three times higher than the traditional and online periods. This fact further proves the benefits yielded by gamification in terms of knowledge acquisition and concept understanding.

Figure 7 illustrates the distribution of the final course grades while

Table 4. Final grade distribution.

Method\Grade	0	1	2	3	4	5	6	7	8	9	10	Succ.	Excel.
Online Learning	0%	7%	12%	5%	0%	26%	18%	14%	12%	3%	3%	78%	18%
Traditional Learning	0%	4%	13%	2%	2%	19%	21%	14%	9%	10%	5%	78%	24%
Gamified Learning	0%	2%	6%	0%	0%	14%	16%	17%	19%	12%	9%	90%	41%

provides a more detailed view of the final course grade distribution. It can be observed that the success rate increased to 90% for gamified learning. Moreover, the excellence rate improved from $18\%$ to $24\%$ during traditional learning and increased to $41\%$ during gamified learning. A significant increase of $127\%$ was noticed when comparing gamified learning to online learning.

4.2. Students’ Experience and Viewpoint Analysis

After the completion of the laboratory and theoretical examinations, students were asked to complete an anonymous questionnaire which focused on their perspectives and experiences. Only students who participated in both parts of the course were allowed to complete the questionnaire and to ensure that, students had to first log in to the course educational platform using their institutional account as they did throughout the semester and select the link to the web-based questionnaire. Their logging in the system was regarded as their agreement to participate. The completion of the questionnaire was voluntary and participants could withdraw at any time. No information that could identify an individual was required. The only demographic information required was the participants’ gender and age which, on their own, are not enough to identify them. Hence, the anonymity of the participants was ensured.

Of the students who voluntarily participated, 185 answered all the questions of the questionnaire and their responses are examined and presented in this study. The 185 higher education students had an average age of 21.4 years old. The age is representative of the semester that students can take this course for the first time. Out of the participants, 158 were male students (85.4%) and 27 were female students (14.6%). The distribution is representative of the student population present at the specific department.

The ad-hoc questionnaire used focused on students’ experience and perspectives and consisted of 19 questions that used a 5-point Likert-scale (1: Strongly disagree...5: Strongly agree) and a closed-ended question. Following the reliability analysis, the 5-point Likert scale questions (19-items) resulted in a Cronbach’s Alpha value of 0.907. This fact highlights the reliability and internal consistency of the questions used. The questions were grouped into the following five categories:

4.2.1. Students’ Viewpoints Regarding the Integration of Gamification in Education

Having experienced the use of gamification throughout the semester, the vast majority of students highly regarded its potentials to enrich the educational process. Specifically, they assessed that when following a student-centered approach, gamification can increase the effectiveness of the educational process (80.54%), improve learning productivity (81.62%), and offer more enjoyable learning experiences (65.41%). Moreover, students considered gamification to be an educational approach that can promote motivation for learning (83.78%) as well as active participation and engagement (81.08%) while also offering a more enjoyable learning experience (65.41%). Additionally, students quoted that through the use of gamification, their focus on the achievement of educational goals was reinforced (75.68%). Students also considered that the use of gamification elements can result in the creation of a competitive environment (58.38%). However, this competitive environment is neither characterized by negative feelings nor causing them (47.57%). Instead, it allows students to stay more focused and motivated to perform better. Students’ related responses are presented in detail in Figure 8 and in

Table 5. Students’ viewpoints regarding the integration of gamification in education.

Having Experienced the Application of Gamification in Education, Do You Believe That It Can …	Strongly Disagree		Disagree		Neutral		Agree		Strongly Agree
	Freq.	Perc.	Freq.	Perc.	Freq.	Perc.	Freq.	Perc.	Freq.	Perc.
increase the effectiveness of the educational process?	1	0.54%	3	1.62%	32	17.30%	99	53.51%	50	27.03%
improve academic performance and learning outcomes?	1	0.54%	3	1.62%	27	14.59%	98	52.97%	56	30.27%
promote motivation for learning?	1	0.54%	5	2.70%	24	12.97%	88	47.57%	67	36.22%
promote active participation and engagement?	1	0.54%	5		29	15.68%	93	50.27%	57	30.81%
increase learning productivity?	1	0.54%	5		28	15.14%	102	55.14%	49	26.49%
offer more enjoyable learning experiences?	4	2.16%	12	6.49%	48	25.95%	72	38.92%	49	26.49%
reinforce focus on the achievement of educational goals?	1	0.54%	7	3.78%	37	20.00%	93	50.27%	47	25.41%
create a competitive environment?	6	3.24%	30	16.22%	41	22.16%	67	36.22%	41	22.16%
cause negative feelings due to competition involved?	24	12.97%	64	34.59%	44	23.78%	39	21.08%	14	7.57%

.

4.2.2. Students’ Learning Motivation

In the context of educational intervention, it is important to analyze learning motivation. Based on students’ responses, the integration of gamification in the educational process mostly promoted their intrinsic and extrinsic motivation (45.41%) followed by only promoting their intrinsic motivation (33.51%). Few students (15.68%) assessed that only their extrinsic motivation was promoted. Only 10 students quoted that this approach did not promote any learning motivation. Nonetheless, it is important to note that despite students being rewarded with extra credits depending on their performance, the vast majority of students assessed that gamification promoted intrinsic motivation more than extrinsic motivation. This fact highlights the potential of gamification to be used as an effective pedagogical approach. Figure 9 summarizes students’ responses concerning which learning motivation they felt was mostly promoted.

4.2.3. Students’ Basic Needs

Self-determination theory is strongly connected to the domain of education. Hence, when adopting new approaches or integrating new tools in teaching and learning, it is important to examine whether students basic needs, that is autonomy, competence, and relatedness, are met. Based on the results presented in Figure 10 and in

Table 6. Students’ basic needs.

Having Experienced the Application of Gamification in Education, Do You Believe That Its Use in Educational Settings Can Promote …	Strongly Disagree		Disagree		Neutral		Agree		Strongly Agree
	Freq.	Perc.	Freq.	Perc.	Freq.	Perc.	Freq.	Perc.	Freq.	Perc.
autonomy	2	1.08%	14	7.57%	77	41.62%	75	40.54%	17	9.19%
competence and sufficiency?	2	1.08%	4	2.16%	28	15.14%	109	58.92%	42	22.70%
relatedness—a sense of belonging	8	4.32%	26	14.05%	76	41.08%	61	32.97%	14	7.57%

, it can be inferred that students mostly expressed a positive or neutral opinion regarding the potential of gamification to promote autonomy and relatedness. However, the vast majority of students quoted that gamification can promote and improve their sufficiency and competence (81.62%). These results highlight the potential of gamification to meet students’ basic needs when appropriately integrated in the educational process.

4.2.4. Students’ Viewpoints Regarding the Benefits of Integrating Gamification in the Specific Course

Students were also asked about their opinions regarding the benefits that the integration of gamification brought about in the specific course. Based on the results presented in Figure 11 and in

Table 7. Students’ viewpoints regarding the benefits of integrating gamification into the specific course.

Having Experienced the Application of Gamification in Education, Do You Believe That Its Use in the Educational Process Can …	Strongly Disagree		Disagree		Neutral		Agree		Strongly Agree
	Freq.	Perc.	Freq.	Perc.	Freq.	Perc.	Freq.	Perc.	Freq.	Perc.
offer educational benefits?	1	0.54%	4	2.16%	39	21.08%	99	53.51%	42	22.70%
render the educational process more interesting?	2	1.08%	6	3.24%	39	21.08%	88	47.57%	50	27.03%
promote learning motivation for success and intense effort through its reward systems?	1	0.54%	7	3.78%	32	17.30%	98	52.97%	47	25.41%
create learning environments which have students in their core?	1	1.08%	5	4.86%	29	29.73%	93	48.65%	57	15.68%

, the vast majority of students (76.22%) assessed that gamification yielded several educational benefits while simultaneously rendering the educational process more interesting (74.59%). Moreover, they quoted that a student-centered environment was created (64.32%) and that this approach enabled them to be more motivated to achieve greater learning outcomes and to make intense efforts (78.38%).

4.2.5. Students’ Perspectives Regarding the Future Use of Gamification in Education

The last part of the questionnaire was related to students’ perspectives on the future use of gamification in education after having experienced a semester of gamified learning. The related results are presented in Figure 12 and in

Table 8. Students’ perspectives regarding the future use of gamification in education.

Having Experienced the Application of Gamification in Education, Do You Believe That Gamification …	Strongly Disagree		Disagree		Neutral		Agree		Strongly Agree
	Freq.	Perc.	Freq.	Perc.	Freq.	Perc.	Freq.	Perc.	Freq.	Perc.
should be used again in this course?	3	1.62%	6	3.24%	17	9.19%	78	42.16%	81	43.78%
should be applied in other courses as well?	8	4.32%	7	3.78%	18	9.73%	74	40.00%	78	42.16%
can enrich higher education?	3	1.62%	12	6.49%	30	16.22%	68	36.76%	72	38.92%

. Specifically, as shown from the rest of the results, students were fond of the integration of gamification and positively assessed it. This is also proved by the fact that the significant majority of students recommended the course should be taught in a gamified manner again in the future (85.95%) and suggested that other courses should also adopt similar approaches (82.16%). All in all, students highly valued gamification and regarded it as an approach that can significantly enrich higher education (75.68%).

5. Discussion

Students’ learning and the educational process in general can be evaluated from different perspectives and factors. Students’ learning outcomes and academic achievements are closely related to their educational success, ability to learn, pursuit of lifelong learning, and future career prospects [68,69]. Moreover, there are multiple factors that can influence students’ performance in academic settings [70,71,72]. Learning environments have proven to be one of the most significant factors that can impact students’ academic performance and learning outcomes [52,73,74,75].

Although there have been several studies that examine the impact of online learning [76,77,78,79], traditional learning [53,80], and gamified learning [81,82,83], to the best of our knowledge, there has not been any recent study that explores how these three environments influence students when they are the only differentiating factor. Given the significant changes that the educational domain has undergone in recent years and the existing gap in the literature, this experiment involved a longitudinal study to evaluate students’ learning outcomes and academic performance by comparing their performance during online learning, traditional learning, and gamified learning. The experiment lasted three years and a total of 1001 higher education students participated. Moreover, the study also involved the analysis of students’ viewpoints and experiences regarding the adoption and use of gamification in education. In total, three research questions were set to be explored.

5.1. Outcomes of the Longitudinal Experiment

Based on the results of the longitudinal experiment, it can be inferred that overall, gamified learning was the most effective approach in comparison to traditional learning and online learning. Specifically, regardless of the structure of the laboratory course and its increasing difficulty, better outcomes were observed in all laboratory tests when using the gamification approach followed by traditional learning. This resulted in students’ acquiring a higher laboratory average grade. The same stands true for the theoretical part of the course in which students outperformed during gamified learning. Additionally, it was evident from the distribution of the grades that as laboratory tests were becoming more difficult students that were more motivated and engaged performed better and acquired a better understanding of the concepts taught. This fact can be further justified by the increase in students’ learning outcomes and academic performance in the theoretical part of the course in which gamification elements were used to a lesser extent. The integration of gamification also improved students’ excellence grades and overall success rate while simultaneously decreased the course withdrawal rate.

These findings are in line with and further expand upon the results of other relevant studies which have also highlighted the positive impact that the use of gamification can have in education [35,58,59,79,84,85,86]. More specifically, while classifying and analyzing different gamification elements and gameful interventions in medical education, which is a field that combines theory with application, Krishnamurthy et al. [84] highlighted that gamification can be used to enrich the educational process while yielding advantages. In their longitudinal study, Putz et al. [85], highlighted that gamification can positively impact education and it is not restricted by students’ gender or age while gamification elements can be used to tailor learning experiences according to students’ demands and can increase students’ knowledge retention. In their meta-analysis, Kim et al. [79] presented the positive impact that the application of gamification has on promoting positive behavioral changes in education even in short-term courses. The studies of Gari et al. [58], Ibáñez et al. [59], Lampropoulos et al. [35], and Zhan [86] also revealed the positive impact that the use of gamification can bring in computer science education while also improving students’ learning experience, engagement, and outcomes.

As far as traditional learning is concerned, students performed better throughout the course in comparison to online learning but decreased academic performance was observed when comparing traditional with gamified learning. In the context of online learning, students demonstrated the lowest academic performance, had the fewer number of excellence grades, as well as showcased an increase in course withdrawals during the semester. This is true, when comparing online learning with both traditional learning and gamified learning. Existing studies have also highlighted the decrease in students’ performance during online learning in the pandemic when compared to students’ performance in traditional learning [87,88,89].

5.2. Outcomes of Students’ Viewpoints

As far as students’ viewpoints and experiences regarding the use of gamification are concerned, the vast majority of students had an overall positive attitude. This fact is in line with the findings of other related studies [81,90,91,92]. More specifically, in their mixed method research, Turan et al. [81] highlighted the positive view that students have regarding gamification and the benefits it can yield in learning achievements and cognitive load levels. Cheong et al. [90] also revealed that gamification is a suitable learning approach as it is viewed favorably by students who prefer to be engaged and actively participate in learning and indulge in social interactions. Moreover, Bicen & Kocakoyun [91] presented the positive perceptions that higher education students have toward gamification and its ability to increase their learning interest and their ambitions for success. In their study, Alabbasi et al. [92] highlighted that higher education students are particularly favorable to gamification as it promotes sophisticated learning, challenges them, increases their commitment, and improves their social interactions.

Based on the results of this study, it can be inferred that the integration of gamification in education can increase the effectiveness of the educational process, improve learning outcomes, and increase learning productivity. Moreover, gamified learning can promote engagement, active participation, and learning motivation and simultaneously create a more enjoyable learning experience and allow students to focus on achieving their educational goals. Due to the nature of gamification, a competitive environment can be created. However, this is not always a negative aspect as friendly and healthy competition can lead to increased motivation and academic performance. This fact was evident from the results of this longitudinal study.

Furthermore, it is important to note that the vast majority of students regarded that the use of gamification can trigger both their intrinsic and extrinsic motivation. This fact further highlights the capabilities of gamification to be used as an effective educational approach to increase students’ learning motivation and engagement. The adoption of gamification in teaching and learning can also meet students’ basic needs as students positively assessed its capabilities to improve their autonomy, competence and sufficiency, as well as relatedness and sense of belonging. In the context of the specific course, gamified learning was regarded by students as an effective educational approach that can offer educational benefits, promote learning motivation, increase students’ willingness to put more effort, and renders the educational process more interesting without hindering the overall teaching and learning activities. The academic benefits that the adoption and integration of gamification can yield cannot only be justified by students’ academic achievements which as showcased improved but it can also be proven by (i) students’ positive attitude and experience toward its use, (ii) the increment in their learning engagement and motivation, (iii) their suggesting that it can create student-centered learning environments, and (iv) stating that they would like this approach not only to be used again in this course but also be applied in other courses as well in the future.

5.3. Learning Outcomes from the Adoption of Gamification Learning

To provide a clearer representation of the influence that the different learning environments had on students’ learning outcomes and academic performance, the differences in success rate, excellence rate, average grade, and withdrawal rate are presented comparatively for each learning environment for the laboratory and theoretical part of the course as well as for the overall course in

Table 9. Comparison of the impact that different learning environments have on students’ learning outcomes and academic performance in the laboratory part of the course.

			Success Rate			Excellence Rate			Average Grade			Withdrawal Rate
			Onl.	Trad.	Gam.	Onl.	Trad.	Gam.	Onl.	Trad.	Gam.	Onl.	Trad.	Gam.
			51%	63%	71%	7%	13%	16%	3.57	3.98	4.42	21%	20%	15%
Success rate	Online	51%		−19%	−28%
	Traditional	63%	23%		−11%
	Gamified	71%	39%	13%
Excellence rate	Online	7%					−46%	−56%
	Traditional	13%				86%		−19%
	Gamified	16%				130%	23%
Average grade	Online	3.57								−10%	−19%
	Traditional	3.98							11%		−10%
	Gamified	4.42							24%	11%
Withdrawal rate	Online	21%											4%	42%
	Traditional	20%										−4%		36%
	Gamified	15%										−30%	−27%

,

Table 10. Comparison of the impact that different learning environments have on students’ learning outcomes and academic performance in the theoretical part of the course.

			Success Rate			Excellence Rate			Average Grade
			Online	Traditional	Gamified	Online	Traditional	Gamified	Online	Traditional	Gamified
			76%	78%	90%	18%	23%	41%	4.22	4.72	5.29
Success rate	Online	76%		−4%	−16%
	Traditional	78%	4%		−13%
	Gamified	90%	19%	14%
Excellence rate	Online	18%					−20%	−56%
	Traditional	23%				26%		−44%
	Gamified	41%				125%	79%
Average grade	Online	4.22								−11%	−20%
	Traditional	4.72							12%		−11%
	Gamified	5.29							25%	12%

and

Table 11. Comparison of the impact that different learning environments have on students’ learning outcomes and academic performance in the overall course.

			Success Rate			Success Rate of LT1			Excellence Rate			Average Grade
			Onl.	Trad.	Gam.	Onl.	Trad.	Gam.	Onl.	Trad.	Gam.	Onl.	Trad.	Gam.
			78%	78%	90%	28%	52%	67%	18%	24%	41%	5.56	5.95	6.94
Success rate	Online	78%		0%	−13%
	Traditional	78%	0%		−13%
	Gamified	90%	14%	14%
Success rate of LT1	Online	28%					−47%	−59%
	Traditional	52%				87%		−23%
	Gamified	67%				142%	29%
Excellence rate	Online	18%								−24%	−55%
	Traditional	24%							31%		−41%
	Gamified	41%							122%	70%
Average grade	Online	5.56											−7%	−20%
	Traditional	5.95										7%		−14%
	Gamified	6.94										25%	17%

, respectively. It is important to note that these results are not symmetric. For example, in Table 9, the success rate for gamified learning is $90\%$ and for online learning is $76\%$. This means that gamified learning yielded a $19\%$ improvement over online learning ($0.19=\frac{90-76}{76}$), but online learning is $16\%$ lower than gamified learning ($-0.16=\frac{76-90}{90}$).

Based on the results presented in Table 9, in the laboratory part of the course, gamified learning results in an overall increase of 39% in success rate in comparison to online learning and a 13% increase in comparison to traditional learning. In terms of excellence rate, gamified learning yielded a 130% increase in comparison to online learning and a 23% increase in comparison to traditional learning. When taking into account students’ average grade, the use of gamification improved the average grade by 24% when compared with online learning and 11% when compared with traditional learning. Gamified learning also resulted in fewer students withdrawing from the course. A total of 42% more students remained throughout the semester in comparison to online learning and 36% more when compared to traditional learning. Moreover, when comparing traditional learning to online learning, traditional learning yielded a 23% increase in success rate, an 86% increase in excellence rate, an 11% increase in average grade, and a 4% increase in withdrawal rate in comparison to online learning.

The learning outcomes were mostly similar in nature for the theoretical part of the course as it is presented in Table 10. Based on the results, gamified learning yielded a 19% increase in success rate in comparison to online learning and a 14% increase in success rate in comparison to traditional learning. The use of gamification improved the excellence rate by 125% in comparison to online learning and by 79% when compared to traditional learning. When taking into account students’ average grade, the use of gamification improved the average grade by 25% when compared with online learning and 12% when compared with traditional learning. Furthermore, traditional learning resulted in an increase of 4% over online learning in the success rate, an increase of 26% in the excellence rate, and an increase of 12% in the average grade.

As far as students’ performance in the overall course is concerned, Table 11 summarizes the related results. Based on the findings, it can be said that in the overall course the use of gamification resulted in an overall increase of 14% in success rate in comparison to both online learning and traditional learning. Gamified learning yielded a 142% increase in comparison to online learning and a 29% increase in comparison to traditional learning in success rate of LT1.

When taking into account students’ excellence rate, the use of gamification improved the excellence rate by 122% when compared with online learning and 70% when compared with traditional learning. Gamified learning also resulted in fewer students withdrawing from the course. Students’ average grade improved by 25% when using gamification in comparison to online learning and by 17% in comparison to traditional learning.

Moreover, when comparing traditional learning to online learning, traditional learning yielded an 87% increase in success rate of LT1, a 31% increase in excellence rate, a 7% increase in average grade. However, no differences were observed in terms of overall success rate between traditional learning and online learning. Therefore, it can be inferred that gamified learning yielded the best learning outcomes, followed by traditional learning and online learning which had the lowest learning outcomes in each of the course parts and in the overall course for each of the factors examined.

Based on the aforementioned, it can be concluded that among online learning, traditional learning, and gamified learning, gamified learning is the most effective approach as it resulted in increased educational benefits and students’ performance (RQ1). Regarding students’ learning outcomes, the use of gamification resulted in students’ achieving better academic performance, increasing their learning motivation and engagement, improving the success rate, excellence rate, and average grade, as well as reducing the withdrawal rate (RQ2). Students had an overall positive attitude toward the adoption and integration of gamification since they highly valued it as an effective educational approach that can yield several educational benefits and meet students’ basic needs and thus, should be used more frequently and in more courses in the future (RQ3).

6. Conclusions

Gamification is a promising approach which integrates game-related elements, mechanisms, and aspects into non-game related context. Gamification has the potential to enrich the educational process and improve learning outcomes. However, there is limited knowledge regarding how it affects students’ learning outcomes and achievements when compared to traditional learning and online learning. Additionally, although there are studies that examine the influence of the learning environment and approach on students’ performance, these studies focus mostly on examining a single learning environment. Therefore, this longitudinal study aimed at exploring how gamification impacts students’ learning outcomes and academic performance when compared with traditional learning and online learning. Specifically, the study lasted three years and a total of 1001 higher education students participated.

It is worth noting that the results presented in this study and the findings did not account for the bonus grades that students received through the use of gamification. The results and the outcomes presented are solely based on students’ performance in the examinations. Moreover, the complexity of the course and its having both a theoretical and laboratory part further highlight the capabilities of gamification.

Based on the findings of this study, students’ learning outcomes and academic performance as well as their learning motivation and engagement were higher during gamified learning, followed by traditional learning and online learning. Specifically, in the laboratory part of the course, gamified learning resulted in a 13% increase over traditional learning and 39% over online learning in success rate, a 23% increase over traditional learning and 130% over online learning in excellence rate, an 11% increase over traditional learning and 24% over online learning in average grade, and a 36% increase over traditional learning and 42% over online learning in students’ retention rate. In the theoretical part of the course, gamified learning resulted in a 14% increase over traditional learning and 19% over online learning in success rate, a 79% increase over traditional learning and 125% over online learning in excellence rate, and a 12% increase over traditional learning and 25% over online learning in average grade. In the overall course, gamified learning resulted in a 14% increase over traditional learning and 14% over online learning in success rate, a 29% increase over traditional learning and 142% over online learning in success rate of LT1, a 70% increase over traditional learning and 122% over online learning in excellence rate, and a 17% increase over traditional learning and 25% over online learning in average grade.

The findings regarding students’ viewpoints about the adoption and integration of gamification in education revealed that students had a positive attitude toward gamified learning and regarded its educational use highly. Specifically, students assessed that gamification can enrich the educational process, improve their learning outcomes, motivation, engagement, and productivity, and create more enjoyable, interactive, and student-centered environments. Based on the results, it can be inferred that the integration of gamification in education can increase the effectiveness of the educational process, improve learning outcomes, and increase learning productivity. Gamification was also proven to be an effective educational approach as it triggered students’ both intrinsic and extrinsic motivation and met their basic needs in terms of relatedness and sense of belonging, competence and sufficiency, and autonomy. The fact that students recommended gamification learning should be applied again in the future and in more courses further highlights its potentials and it being favored by students.

All in all, gamification emerged as an effective educational approach which can enrich the educational process, is positively assessed by students, and can result in increased learning outcomes and academic performance (e.g., excellence grade, average grade, success rate), learning motivation, engagement, and retention rate over traditional learning and online learning. Moreover, gamification led to students’ acquiring a better understanding of the course material and the concepts taught during the laboratory part of the course which was showcased during their performance in the theoretical part. Additionally, not only did students have a higher average grade and success rate, but their excellence rate was also significantly improved.

These findings further prove the potential and benefits when adopting and integrating gamification in education in both theoretical and applied courses. Therefore, based on the findings of this study, the adoption of gamification is generally encouraged; however, its use is becoming more evident in courses and subjects that are more complex, difficult, and require students’ active involvement and self-directed learning. Careful planning is required in order for gamification not to act as a distraction but rather as a factor that contributes to the creation of more engaging, enjoyable, and motivating learning experiences.

Nonetheless, there are some limitations that should be mentioned. Specifically, this study involved three approaches which were adopted and integrated in a single course. Additionally, the nature of the course having both lectures and applied lessons can be regarded as another limitation. Although achieving better learning outcomes in such a complex course further highlights the potential of gamification to enrich the educational process, further analysis regarding the integration of gamification in lecture-only or applied-only courses and how it affects learning outcomes is required. Hence, more studies should be conducted that analyze the adoption of gamification in different settings. To better understand the influence of gamification on teaching and learning approaches, future studies should examine its use in comparison with different learning environments, courses, areas of knowledge, and student population and examine its role in combination with other pedagogical approaches and technologies. Additionally, the competitive nature that might be developed in gamified learning environments and its influence on students should be further examined. Finally, there is a need to develop guidelines to effectively integrate gamification in classrooms and evaluation tools to assess its impact on more factors.