Game Elements towards More Sustainable Learning in Object-Oriented Programming Course

Abstract

Gamification of education is considered to have the capacity to increase students’ engagement in a learning process; to drive learning and skills acquisition; and creates changes for more sustainable behavior. Sustainable behavior is especially important in studying the initial courses at a university. Object-oriented programming (OOP) is such a course in the studies for a programmer’s diploma. We applied four elements of gamification: experience points, interactive content, local team, and global team. A local team is dedicated specifically to the OOP course, but all other game elements can be applied to any course. The course was implemented in a Moodle platform, where two additional plugins for experience points and interactive content were employed. Research using the implemented gamified course in the teaching process was carried out during the fall semester in 2020 and results compared to two previous non-gamified course years. A statistically insignificant increase in the mean grade was observed. Nevertheless, the gamified course resulted in a 7% decrease in the number of students who had to retake the OOP course compared to the year 2019. The gamified course also resulted in a drastic decrease in the number of students requiring a second attempt to pass the course’s exam. The obtained results demonstrate that the gamification of OOP course resulted in more sustainable behavior of the students. Based on the results of students’ questionnaires, we highly recommend using global team game element as it showed a positive effect on students’ engagement when the number of participating students is large.

1. Introduction

Engagement of students is important in teaching a course as it directly influences the students’ achievements. Students tend to lose interest while studying difficult topics, which usually involve a large amount of learning material [1]. Moreover, university educators are constantly under pressure from the authorities’ and students’ feedback after completing a course to improve its delivery. When all the usual means for increasing students’ engagement are exhausted, gamification [2] can come to the rescue and make the course more attractive, motivating, and engaging.

We analyzed the effect of applying gamification in a course for undergraduate students at the Kaunas University of Technology, Faculty of Informatics. The course is devoted to object-oriented programming (OOP) and is an introductory course given in the first semester of the first year. The objectives of the course are to form and develop object-oriented thinking; to strengthen the skills of algorithm design; and to increase the skills required in solving problems iteratively. After completing the course, students will be able to create or apply a known algorithm for a solution or task; apply the object-oriented principles for the creation of the programs; implement the constructed algorithm in C# programming language; and test a created program using initial testing skills.

Programming is one of the most difficult activities that humans have undertaken. The success stories of young people as told in the media make IT an extremely attractive profession. Partly as a result of this, around 650 students are admitted to the Informatics faculty every year. This does not necessarily guarantee that every new student has the motivation and ability required to become a programmer. In such a context, when all the other means of increasing students’ engagement were exhausted, we decided to apply game elements [3] in the teaching of OOP, since an essential goal of education is to motivate students to engage [4]. If students have no motivation to learn, there exists the possibility of motivating them extrinsically [5]. Moreover, Venter [6], after reviewing 21 papers presenting a gamification of programming courses, concluded that the gamification of higher education programming courses had a very positive impact on the student motivation, engagement, and programming knowledge.

Our gamified course was implemented in Moodle [7]—a popular platform commonly used by universities to support teaching as well as learning [8]. For enabling the required gamification elements, two Moodle plugins were deployed—H5P [9] and Level up [10]. These plugins were configured accordingly to the specifics of the OOP teaching course and modified to support the Lithuanian language that is used in the course.

OOP is not easy to learn, but it is essential to future programmers as it has been a dominant paradigm of programming for the last three decades. The objective of our research was to establish whether gamification can improve the ability of students to learn the skills of OOP. In order to determine whether the gamification of the OOP course succeeded in this objective, the following hypotheses were formed:

Hypothesis 1. (H1):

Gamification of the OOP course will have a positive influence on students’ grades.

Hypothesis 2. (H2):

Collaborative learning can enhance learning and increase students’ grades.

The research of using the implemented gamified course in the teaching process was carried out during the fall semester of 2020. Three-hundred-and-eighty-nine software engineering first-year students participated in this research.

2. Related Work

There exist many studies presenting the results of game element application for the gamification of the student learning process. In selecting relevant studies, attention was focused on journal publications, but appropriate conference papers were not excluded. This review of related works is presented chronologically in the order that they were written.

Barata et al. [11] presented a study in which a college master’s course—Multimedia Content Production—was gamified. Moodle was used as a platform for the gamification, and the gamified course lasted for two consecutive years. After the application of gamification in the first year, the final grades of the course demonstrated a decrease in comparison with the non-gamified course of the previous year. The course was then fine-tuned based on the student feedback, and the experiment was prolonged for another year. The feedback was mostly related to insufficient rewards and lack of student cooperation. After 2 years of gamification, the results showed significant improvements in terms of attention to reference materials, online participation, and proactivity. However, the increased student activity had little impact on the final grades. Moreover, the study had two serious limitations that necessitate caution when considering the results. These limitations are defined as: (1) the topics covered by the course having changed over the years, which may have affected the knowledge, motivation, and engagement of the students; and (2) the faculty staff having remained the same for the theoretical lectures but changed from 1 year to the next for lab classes. The communication skills of lab instructors is very important because they communicate with students face-to-face, and all students must take part in the lab classes.

Domínguez et al. [12] considered the gamification of the course Qualification for Users of ICT, in which students learn how to use common ICT tools. This includes basic use of the operating system, word processors, spreadsheets, presentation software, databases, web browsers, and email. This is usually an introductory course applied for non-computer-science students, and the exercises are downloaded from a Blackboard e-learning platform. The instructors of the course have created a Blackboard plugin to provide the same exercises in a gamified way, using both rewards and competition mechanisms. The course was delivered to two separate unrelated groups of students: control and experimental. The students of the control group studied the usual course, whereas the students of the experimental group had access to the gamified course. The final results showed that the students of the experimental group achieved higher scores in all of the categories that were related to the practical application of the concepts learned. The students of the control group, however, performed better on the written examination. One of the limitations of the course was the Blackboard plugin, with some students complaining that it was hard to use or did not work well. The authors concluded that the gamification of the course had the potential to increase student motivation, but it was difficult to achieve that effect and considerable effort was required in order to design and implement the course properly.

Filsecker and Hickey [13] investigated the effects of rewards on fifth graders’ learning in the educational game environment. Quest Atlantis was used as the game environment in this study, where the subject of learning was ecological science. The course was delivered to two separate groups of students: control and experimental. According to the results of this experiment, students that were presented with the opportunity to earn rewards showed deeper conceptual understanding than those that were not. However, the impact of rewards on the scores achieved in the test was not observed.

Ibanez et al. [14] presented a study that assessed the effect of gamification in C-programming language teaching within the boundaries of an operating systems undergraduate-level course. For this purpose, the authors developed the Q-Learning-G platform. The students’ goal was to earn 100-grade points in the gamified platform, and the authors collected and analyzed the data from systems’ logs, surveys, and pre- and post-tests. The results of the experiment demonstrated a moderate improvement in learning outcomes, with the authors concluding that collecting badges was the most successful mechanism for fostering engagement. Despite this, the study does, however, possesses the following limitations: (1) the sample was comprised of only 22 students; (2) the short-term knowledge retention of C-programming language was investigated only; and (3) the research did not have a control group, which is a necessity in achieving comparability in results [15]. Therefore, the conclusions drawn from this study must be viewed with caution.

Boticki et al. [16] presented a mobile learning system—SamEx—used by students to study natural processes in indoor and outdoor environments. The students generated data throughout a 1-year period, and the focus was on the media (pictures, video clips, and audio recordings) collected by the students, as this was the most widely used feature of the application. One of the more high-profile outputs of the research was an exploration of the emergence of individual and collaborative learning. The authors came to the conclusion that the design of collaborative tasks that have elements of individual accountability, and require interactions and social skills, are more likely to foster collaborative learning. A separate result of the research was the conclusion that the quantity and quality of contributions provided by students can predict the final assessment score. Thus, only self-directedness can be used as a predictor variable of the final score, meanwhile teacher-directedness remains insignificant. However, this research again lacked a control group and so possesses a similar limitation to Ibanez et al. [13].

Hanus and Fox [4] tested students using common game mechanics applied to a gamified Communication course, involving badges, leaderboards, and competition during a 16-week semester. The course was delivered to two separate groups of students: control and experimental. However, the study has a serious limitation in that it focused on gamification elements that can be applied by a traditional teacher in a classroom, whereas many game mechanics are best applied using computers and the virtual world. The results of the experiment showed that the students in the experimental group demonstrated less motivation and satisfaction over time than those in the control group. The final exam scores of students in the experimental group were lower than the final exam scores of students in the control group. The obtained results suggest that care is needed in the application of certain gamification mechanics to the educational context. Additionally, the authors suggested that future research should investigate the impact of the specific elements of gamification in order to create an ideal gamification system.

Çakıroğlu et al. [17] intended to reveal the effect of a gamified course on student engagement and demonstrated the relationship between engagement and academic performance. The study included the results from an Information and Communication Technology course in which participants were preservice primary school teachers. The course was delivered to two separate independent groups of participants—control and experimental—in the real classroom. However, many game mechanics are best applied using computers and the virtual world. An engagement scale, an activity evaluation rubric, and a gamification evaluation form were all used to collect the data. Clinical interviews were managed with only six selected students in order to address the relationships between the objects of the research. The results showed that using game elements provided a key positive impact on engagement. Additionally, the use of game elements positively affected academic achievement in a moderate way. The results obtained are of limited use because computers and the virtual world were not used for the application of game mechanics.

Mekler et al. [18] conducted an experiment that examined the four variables of a game in an image annotation task. These variables were as follows: points, leaderboard, levels, and plain condition without game elements. The goal of the experiment was to evaluate the effect of game elements in isolation on both intrinsic motivation and need satisfaction. The results obtained from the experiment showed that the game elements applied did not increase intrinsic motivation. Additionally, the findings obtained suggest that the game elements applied successfully functioned as extrinsic incentives for promoting performance quantity. This is a secondary result, but a very important one. However, the authors recognized that the nature of the image annotation task can have an influence on the outcome of the studies. Therefore, the results should be viewed with caution.

Sánchez-Martín et al. [19] investigated whether gamification promotes collaborative or competitive relationships across a group. A game-based methodology was used by prospective primary teachers to present the concepts of matter and energy. A Game-Index was introduced to promote collaborative efforts. This Game-Index corresponded to the minimum number of classmates that held a minimum number of points. The experiment showed that the majority of the students invested their game points for personal benefit. The authors therefore concluded that the Game-Index alone was not enough to intensify collaborative action.

Aldemir et al. [20] explored students’ overall perceptions about various game elements in a gamified technology within a teacher education course. A variety of game elements were used, including: challenge, narrative, leaderboard, reward, badge, teams, win-state, points, and constraints. Three distinct yet complementary tools were deployed in the gamified course: Edmodo, Blendspace, and Weebly. All game elements were thus explored comprehensively. Whilst some participants of the experiment criticized the team element for making them responsible for teammates’ activities, the authors advocated for the use of all aforementioned game elements. The authors therefore concluded that the design of the gamified course should be addressed as a multidisciplinary iterative process, and one that is built on interrelated fields such as psychology, education, and game studies.

Barna and Fodor [21] investigated the influence of several gamification elements on teaching Information Technology to non-computer science undergraduate students. Moodle was used as the platform for the gamification. The course employed the following gamification elements: rewards, instant feedback, alternative learning paths, feedback options, and social interaction platforms. The experiment was carried out twice—once in 2015 and once 2016—and in total more than 2500 students participated in the courses. The authors concluded that the gamification effect cannot be clearly measured since there was no control group for comparison.

Kyewski and Krämer [22] considered whether badges had a positive impact on students’ motivation and performance. Moodle was used as the platform for the gamification. The course was delivered to two separate groups of students: control and experimental, and the study was conducted during an online seminar over a period of one semester. The topic of the seminar was “Basic psychological mechanisms of computer-mediated communication: learning and teaching”. The results of the study showed that the badges did not have an influence on intrinsic motivation, however, certain limitations mean that they cannot be considered to be wholly valid. These limitations include: (1) the number of participants having diminished from 106 in the beginning to 27 at the end of the course; (2) the period in which badges were awarded was only the first 5 weeks of the course; and (3) the badges were not tied to the actual rewards. The authors therefore concluded that studies on the gamification of courses should, in future, be conducted more comprehensively.

Jurgelaitis et al. [23] analyzed the effect of applying gamification to undergraduate students studying a UML modeling course. Moodle was used as the platform for the gamification, and the gamified course was based on levels and the gradual unlocking of course content. The following gamification elements were used in the course: points, coins, items, badges, leaderboard, content locking, and trading. Analysis of students’ grades confirmed the hypothesis that grades can increase as a result of applying gamification. The analysis of the results confirmed expectations, as the student grade average of the gamified course was higher by 0.3 points in comparison with the previous year’s non-gamified course. The results of student questionnaire also confirmed the positive effect of gamification on student motivation. The authors did not discuss nor reveal which gamification elements had a positive effect on the success of gamification. These results have to be taken with caution as the experiment that was carried out had two serious limitations: (1) the participation in the gamified course was optional, as a result of which one-fifth of the students did not take part in the gamified course; (2) the gamified course was not included in the grading system of the main course.

Van Roy and Zaman [24] were aware of the contradictory findings regarding the effectiveness of game elements in educational courses. Their research goal was to complement the field by analyzing students’ motivation on the basis of Self-Determination Theory. The experiment took 15 weeks for 40 master’s students. Google+ Communities was used as a platform for gamification by implementing 12 weekly challenges, 71 unannounced badges, and group competition. Four surveys were administered to measure the evolution of students’ motivation. The results obtained indicated the individual nature of motivations, the need for longitudinal study, and the importance of the relevance of game elements. This research had two important limitations: (1) not all of the students participated in all four surveys; (2) a gamified platform was unfamiliar to many of the students.

Marín et al. [25] developed an open-source gamified platform—UDPiler—to teach C in a first-year programming course. UDPiler was developed using the MDA framework, and the gamified elements used were points, medals, a leaderboard, and keys to unlock specific contents. Two student groups were involved in the course: those using a non-gamified compiler and those using gamified platform. The results indicated that the students using the gamified platform obtained better grades. Moreover, evidence was provided to suggest a statistical significance in favor of using the gamified platform. The authors concluded that replication is needed to validate and support the findings that they have obtained.

Facey-Shaw et al. [26] investigated whether the badges can affect the intrinsic motivation in introductory programming students. The distinguishing feature of the study was its long period of application. The study reported the results of a baseline study without badges and four subsequent years of badge experiments. The obtained quantitative results suggested that the badges did not increase the intrinsic motivation of students. However, Facey-Shaw et al. [26] were cautious about the results obtained and declared that the findings are inconclusive.

García-Iruela et al. [27] tried to verify if there is an existing relationship between gamification and an increase in student activity that would imply a greater learning improvement. The study was applied to a database subject taught for the first year of computer engineering students. Many typical elements of gamification were used in the study. However, the experiment was carried out only during a month in the middle of the semester. The obtained results indicated no significant differences between the methodology or the activity with a learning improvement. García-Iruela et al. [27] called for a longer duration study of the application of the gamification.

Sanchez et al. [28] examined the impact of gamified quizzes on students’ learning. The examination of effects of gamification was carried out in an introductory psychology course during two consecutive semesters. The obtained results showed that the gamification had a positive effect only on the first test completed. This result supported the conclusion that gamification might work through a novelty effect, however, its impact might not be sustainable. The authors concluded that gamification might be a viable option for short-term assignments and called for longitudinal studies investigating the novelty effects of gamification.

Schlömmer et al. [29] explored the relationship between leaderboards, heart rate variability, and task performance. The leaderboards are tools that are used to exchange and compare the results within a community. Differently than other game elements such as badges, leaderboards allow an immediate comparison. However, they can be harmful for users with humble performance. Such users can feel declined compared to better performing persons and can encounter higher levels of stress. The experiment did not yield sufficient evidence to support the assumption that leaderboard positions increase stress and consequently negatively affect task performance.

Oliveira et al. [30] proposed a theoretical framework relating game elements to behavioral attitudes to promote sustainability and ensure quality of learning. The framework is associated with the PDCA (Plan, Do, Check, Act) stages. The framework was constructed based on the examination of 130 articles assessed as eligible to the study and indexed in the Web of Science database. The framework consists of the following seven steps: (1) user-centered and personalized experiences; (2) challenging and clear missions; (3) narrative and fantasy; (4) repetitive loops and freedom to fail; (5) competition and social engagement; (6) feedback and reflection; and (7) reward and social credibility. The proposed framework promotes changes towards more sustainable behavior of learners.

To summarize, the related work outlined in this section demonstrates that, despite widespread study on the use of game elements in the learning process, the results obtained are, in many cases, not cause for optimism. Especially, it is characteristic for the latest research works [26,27,28]. However, the authors [26,27,28] are cautious about their results obtained and call to continue to investigate whether the gamification can have a positive effect on learning outcomes. These are one of a few papers where gamification is applied for programming courses. Gamification is used for teaching C programming language (not object-oriented programming), or for a very introductory course, more algorithm-based, not even mentioning a particular programming language. Cooperation in a team is very important for the professional programmers, nevertheless, the team element has been addressed in very few research works. Moreover, Sanchez et al. [28] observed that the impact of gamification may not be sustainable. In addition, systematic literature review of gamification in education provided by Manzano-León et al. [31] in 2021 states the need to expand research on the achievements of students when learning with gamified techniques. So, our research work is a response to this call. Moreover, the gamification of the teaching of programming was presented in three research works only [14,25,26], two of them involve the C programming language. A wide variety of platforms were used for gamification, but Moodle dominates [11,17,21,22,23,26,27], and relatively few works [16,20] stressed the importance of collaborative learning. In order to drive for more sustainable behavior and ultimately to obtain a better academic performance, we are going to apply game elements for the teaching of OOP. Our platform for this gamification is Moodle. Collaborative work is very important for future professional programmers, and we will therefore devote a lot of attention to the collaborative process during these studies.

3. Methodology

We split a section of the methodology into two subsections. In the first subsection, we discuss the game elements that are chosen for the gamification of the OOP course. In the second subsection, we provide the details of the implementation of the chosen game elements in the OOP course.

3.1. Game Elements of the OOP Course

The Kaunas University of Technology follows a 16-week academic semester and a 4-week exam session. Students are divided into groups, each consisting of 25–27 students, by a computer-based program, and each group is treated as an administrative unit. The students are included in groups based on their academic performance, such that each group has students of varying academic performance, and the timetable of studies for students is usually constructed based on these students’ groups.

During the OOP course, every week students take one theoretical lecture and have 4 h of practice with computers, under the surveillance of the tutors. The tutor is working with 12 students at a time, and in total 20 tutors are involved in the OOP course. The students have to complete five laboratory assignments, with assignment completion following a timetable announced at the beginning of the semester. The topics for the laboratory assignments are as follows: collection, container, inheritance, text processing, and polymorphism. The topic of text processing falls out of the general sequence of the OOP course, however, the topic remains important for students in knowing the rules of text processing, and is therefore included. The topics of OOP are presented according to the increased complexity and sophistication of OOP concepts, and the programming language used for OOP is C#. For comparison purposes, it is possible to remember the programming languages used in 21 papers on gamification of teaching of programing reviewed in [6]. Eight of the studies did not specify the used programming language, at all. C, Java, and Python were used in three studies each. C++ was used in two studies, meanwhile JavaScript and SQL were used in one study each. If to take into account programming languages used in [14,25], it is possible to conclude that C is the most popular programming language to teach a gamified course. However, C is not an object-oriented programming language.

We adopted self-determination theory (SDT), which is the most frequently used psychological theory in gamification research [32], as the underlying theoretical framework. SDT declares that student’s sense of autonomy, competence, and relatedness are the three innate psychological needs supported by the internal factor of motivation. It highlights that satisfying students’ three innate psychological needs strengthens students’ intrinsic motivation. The more these needs are satisfied, the greater the intrinsic motivation that arises in gamified activities. The four gamification elements (Figure 1), which motivate the different students’ needs [33], were introduced into the OOP course. They are as follows: XP points, interactive content, local team, and global team. The chosen gamification approach is focused on the rewards for the accomplished tasks that are visible to other students and encourages students to collaborate in the team. The combination of competition and collaboration is expected to be effective for learning. These goals are achieved by game elements introduced in usual daily course tasks. The reasons for each game element are as follows:

Figure 1. Game elements of the OOP Course.

• XP points—getting the points for the accomplished tasks and competing with other students.
• Interactive content—provide content of learning in a more attractive way.
• Local team—small groups of students that work together towards a common goal.
• Global team—large groups of students that collaborate and provide help to each other.

Team element is important since the real life situation reflects the trend that programming is usually done in a team. The expectation is that team elements together with other popular game elements could improve learning and have a positive impact.

XP points motivate competence [33]. XP points, badges, levels, and leaderboards are normally implemented in digital games [34], and they are the most frequently used elements in the gamified courses [3,35]. The interactive contents motivate autonomy [33]. The interactive content is a tool enabling presentation of learning content in various attractive forms. The usage of this tool could foster the learning process. The “local team” motivates relatedness [33]. The “local team” is specific for the OOP course, and it is used to inspire close collaboration within the boundaries of the small groups of students. The importance of collaborative efforts within a small team of students during teaching of OOP is emphasized in [36], as well. The “global team” motivates relatedness [33]. The “global team” is used to encourage cooperation within the boundaries of a large group of students, and therefore this game element can be applied to any course. The following section presents a detailed application of our game elements.

The Moodle course included 16 lecture slides, several instruction files related to completing the course, several instruction files for completing laboratory assignments, and the laboratory assignments themselves. The slides of every lecture were followed by a file with 10 questions that stressed the most important aspects of the lecture. When using the plugin Level up! the opening of a specific file was rewarded with a specified number of experience points (XP reward), with the XP points being set according to the importance of the topic. During the semester, the students were able to watch the leaderboard only within the boundaries of their individual group. Only the four students above and four students below the current student could be seen. This restriction prevented the students from potentially using unfair means to take part in the race for XP points. At the end of the semester, the three most successful students were rewarded with badges and an additional prize: a proposed grade for the exam according to their achievements in the semester, as an alternative to taking the exam.

The H5P plugin was used to provide interactive content for the OOP course, which provides capabilities for up to 40 different ways of organizing the process of learning, however, in this instance we used only seven of them. They were as follows: multiple-choice, drag the words, fill in the blanks, drag and drop, mark the words, single choice set, and true/false questions. The H5P plugin enables a student to see the answer without completing the exercise—it is meant to facilitate the learning process rather than to test the student’s knowledge. This feature should help to attract students who prefer this type of learning process. The slides for every lecture (except the last two) were accompanied by this kind of interactive content meant to assist the students. In order to successfully defend their lab project, students had to pass a multiple-choice type test that constitutes 3 points of the total 10 available for the assignment. Our experience shows that obtaining a good score in this test is difficult, and therefore interactive content attached to the slides could help the students to prepare for the test.

Next, we introduced the idea of the local team. We arranged local teams of students for the completion of laboratory assignments, consisting of four members. The students were enrolled in the local team in a random fashion, with the four students sitting in a row in a computer class during laboratory assignments being enrolled in the team. The places of students in computer class during laboratory assignments are permanent. The main obligations of the students for the local team were as follows: (1) every student has to complete and defend their individual assignment according to the timetable; (2) after defense of the assignment, the student then shifts to the right within the boundaries of the team, their program, and the related documentation. The rule of shifting is not applied after the defense of the fourth laboratory assignment.

The idea of local team assignment is supported by several arguments. Firstly, programmers usually work in teams rather than alone, and business representatives consistently complain that university graduates are not ready to work as part of teams. Therefore, we foster an acquaintance with the issues involved in teamwork early in the first semester. Secondly, the context of student assignment is kept the same during the whole semester, as every assignment for students consists of four subtasks.

When a student moves to the next laboratory assignment, some details are added to the context of the assignment, the requirement for a more complex data structure is raised, and one of the subtasks is changed. During the change of laboratory assignment, the main focus is on increasing the complexity of data structures rather than algorithms. Thus, when a student gets a new laboratory assignment, they can see and check the algorithms of their teammate, and if the algorithms are worthwhile the student can apply them using new and more complex data structures. In such a way, the stronger students can share their knowledge with the weaker students, and different cultures and styles of programming are shared among students. Moreover, the students are obliged to use the code of the program, which was obtained from the teammate. Such an arrangement of assignments in the local team is designed to support cooperation among team members—an important factor in joining together efforts to achieve a positive result.

Finally, we presented the idea of the “global team”. The administration of the faculty arranged the students into groups, and we then employed such a group of students as a global team. Every student was given grades after the defense of the laboratory assignment. The grade average was calculated for the group of students, and then the leaderboard of groups was announced after the defense of every laboratory assignment. After the semester, a badge was given to every student of the best group. An additional prize, which is very desirable by the students, was allocated to every student in the best group. The additional prize was a proposed grade for the exam according to the semester’s achievements, instead of taking the exam. The implementation of the “global team” stimulates the communication of the students within the boundaries of the group, and can support cooperation amongst group members as well. It can motivate the giving of aid to the weaker students in completing the laboratory assignment, and can ultimately inspire every student to pay more attention to the completing of their individual assignment in a higher quality manner.

Naturally, a few students collaborate and discuss various exercises or problems and find solutions together. Four game-related features could create a better environment for collaboration and contribute to more sustainable behavior. It would facilitate interaction and support students in fulfilling their knowledge needs.

Especially, local and global team elements combined can address sustainable everyday learning and behavior. This approach could have good communicative and interactive qualities and enable students to use them effectively for a learning purpose. Students could integrate their solutions together and improve the everyday learning process, fostering the engagement of students and learning in a more sustainable way.

3.2. Implementation of Gamification in the Course

The gamified OOP course was implemented in a Moodle learning environment. In Figure 2, the view with plugins used and deployed in the Moodle environment is presented.

Figure 2. Gamified course implementation in the Moodle platform with additional plugins. “√” indicates completed part of the course and “×” indicates the remaining part.

A typical view of the first laboratory assignment in the gamified course is displayed in Box 1.

Box 1. Example of the first laboratory assignment.

Basketball. The players are invited to take part in the national team. The following information is available in the data file: name, surname, birth date, height in inches, position (center, forward, point guard, shooting guard), club, the label “invited” (true, false), and the label “captain” (true, false).

• Find the oldest player and display his name on the screen.
• Find all the forwards and display their names, surnames, and height in inches on the screen.
• Select all the players who are invited, and write their data into the file “Team.csv”.
• Form the list of clubs which delegated players to the national team, and write the data into the file “Clubs.csv”.

In order to prove whether the ideas concerning a local team presented in the previous section are correct, a typical view of the second laboratory assignment in the gamified course is represented in Box 2.

Box 2. Example of the second laboratory assignment.

Basketball. Information is available on the national basketball team of the previous and of the current years. The information is presented in two files, where the first line holds the year. In the following lines, the information on players is available as follows: name, surname, birth date, height in inches, position (center, forward, point guard, shooting guard), club, the label “invited” (true, false), and the label “captain” (true, false).

• Form the list of players that are present in both files. Display their data on the screen.
• Find all the different forwards and display their names, surnames, and height in inches on the screen.
• Select all the players who are invited, and write their data into the file “Team.csv”. Additionally, add the year of invitation to each player’s data.
• Form the list of clubs which delegated players to the national team, and write the data into the file “Clubs.csv”.

Students’ progress in the gamified course is indicated by the number of completed tests of the interactive content and experience points and represented in the leaderboard (Figure 3).

Figure 3. Fragment of the leaderboard in the gamified course.

The lead professor of the OOP course checked the leaderboard every day to keep note of the process of the accumulation of XP points. There was a suspicion that some students could try to collect XP points in an unfair way, as cheating is one of the most common pitfalls of gamification [37]. These concerns were proved valid, though only five students from several hundred were found collecting XP points in an unfair way during the semester. All of them were notified, and all of them recognized their culpability without hesitation. These students were given a single penalty, and the XP points that were collected unfairly were no longer considered in the competition. No public announcement of the penalty to these students was given.

A fragment of the leaderboard of a global team is represented in Table 1. It is possible to observe that the grade average is quite low, except for in the first laboratory assignment. There is a simple reason that explains this: a student has two equally legal attempts to defend a laboratory assignment, but here the first attempt was counted only. Students were encouraged not to postpone the defense of the assignment. The second attempt is the university bonus, and the students should take care not to misuse it. For the first laboratory assignment, there was no possibility to distinguish between these two attempts.

Table 1. Fragment of the leaderboard of a global team.

Leaderboard of Groups
	LD1	LD2	KD1	LD3	LD4	KD2	LD5	DA	Points
IFF-9/1	8.4	5.08	6.64	4.76	3.84	5.4	6.4	0	41
IFF-9/6	7.78	5.26	5.37	5.11	4.37	5	5.19	0	38
IFF-9/4	8.36	3.84	4.76	3.56	3.12	6.12	3.96	0	34

4. Research Results and Discussion

4.1. Methods

During the fall semester of 2020, students of the first-year undergraduate course Object-Oriented Programming were invited to participate in the gamified course. Three-hundred-and-eighty-nine students were enrolled in this gamified course, and after the semester three winners were selected based on the number of XP points accumulated. The second-placed student received only a badge, however, since the student had an uncompleted semester assignment and therefore was not eligible for an additional prize. This student will have to repeat the OOP course.

Statistics were collected for the use of interactive content (Figure 4), from which we can observe that more than a third of students (almost 140) completed no exercise using interactive content. A professor was working and preparing various forms of interactive content, however, a lot of students did not show any interest at all. The second-largest group of students are the ones that completed all of the exercises involving the interactive content. However, this group is less than half of the size of the group of uninterested students.

Figure 4. Statistics of students’ use of interactive content.

The “local team” game element gave rise to many discussions even amongst the tutors. After the semester, 9 tutors voted “for”, 10 tutors voted “against”, and 1 was undecided. The main objection of these tutors was that local teamwork caused more problems for students, and that the benefits of this method did not outweigh the disadvantages. The students also expressed a lot of complaints. These complaints mainly arose from better students, noting that they had more work, they received clumsy code, and they often had to rewrite it. On the positive side, more advanced students were comfortable sharing their code with less advanced students. The objective of the local team was to help the weaker students, reveal problems with teammate’s code, and demonstrate to students that working in a team requires additional effort. Disadvantages of using “local team” game element did not outweigh the advantages and therefore is not recommended for use in the following years for the first year students.

The last game element was the “global team”. Here, the students of the groups which were competing for the winner’s prize were very interested in this element. The winning group contained 25 students, and it is noteworthy that the winner of the XP points leaderboard also belonged to this group. The exam grade according to the semester results was proposed to 19 students, as 6 students did not complete the semester assignments and 2 left university during the semester. The suggested grades varied from 7 to 10, with all of the students accepting the suggested grades and therefore not sitting an exam. The game element of the “global team” was used for the other students in the spring semester as well. Here, the initial arrangement was to offer suggested grades only for those students who would be given 8 or more, but students’ demand to also offer suggested grades to those that would achieve a score below 8 proved impossible to resist. Therefore, the proposal of exam grades was carried out to all students that completed all of the semester assignments, and attention was not paid to the value of the grade.

4.2. Results

For the evaluation of the issues raised in the introduction with hypothesis H1, student’s grades were collected for analysis. Only the final grades of the course were collected and, in order to have a more reliable comparison, we used three data samples of students’ grades from the course Object-Oriented Programming.

Two samples (years 2018 and 2019) were non-gamified, and one sample from the year 2020 was gamified. It is necessary to note that the topics covered by the course did not change over these 3 years, and therefore all the material presented to the students was the same. The programming language, student exercises, and schedule remained unchanged. The third year students were all equally informed about game elements and reward system, additionally individual tutors had informed students during the practical courses. The only thing that did change was the students enrolled in the course. In Figure 5, we provide a diagram of students’ grades, presented as percentile values because the number of students can vary slightly from year to year.

Figure 5. Relative numbers of student grades.

The diagram (Figure 5) has columns that are labeled by text values underneath them, and the column “BR” shows the number of students that left the university during the semester each year. The OOP course is taught during the first semester, and always includes students that have chosen the study of programming incidentally. Gamification did not have an impact on this number, as it is almost stable across the years, and we can therefore suppose that we cannot influence these undecided students.

The label “NA” shows the number of students that did not complete the semester’s assignments, and will therefore have to repeat the OOP course. We are naturally very keen that this number should be as low as possible and can indeed observe that this number is lower during the gamified OOP course. This represents the first, albeit small, achievement. The next two achievements are: the increased number of 10s and the decreased number of 6s. If we calculate the mean grades for each of these 3 years, however, these achievements are quite low in value. The mean grades are as follows: 2018—7.39071, 2019—7.35836, 2020—7.43581; where the number of students having the value “BR” or “NA” was not taken into account as these labels are not grades. Therefore, if we discard the values of labels “BR” or “NA”, we obtain a near-normal distribution of grades, and can consequently evaluate the statistical significance of the mean grade using the Student’s t-test. The difference in grades between 2020 year (M = 7.44; SD = 1.22) and 2018 year (M = 7.39; SD = 1.07) was not significant (t (295) = 0.6362; p = 0.525). The difference in grades between 2020 year (M = 7.44; SD = 1.22) and 2019 year (M = 7.36; SD = 1.24) was not significant (t (295) = 1.0926; p = 0.275).

The results of the Students’ t-test show that the increase in the mean grade for the gamified course was statistically insignificant, and thus our first hypothesis was not fulfilled. However, during the exam session, we incidentally observed a very attractive result that we did not think to measure initially. The student has the opportunity to pass the exam three times during the session. The second time is provided by the university, and the third time is paid. Usually, we had around 70 students after the first exam trial during non-gamified course years. In the gamified course, however, we had only 30 students—a result with which we are very happy with. This resulted in us having much less work during the exam assessment of the second trial, consequently making all the investment into gamification of the OOP course worthwhile. As a result, we will continue the gamification of the OOP course for the next year.

4.3. Discussion

To discuss our results obtained in the general context of the gamification of studies, we have collected information from the papers reviewed in the related work section, added our achievements, and presented tabulated and summarized information in Table 2.

Table 2. Summarized information of the gamification field. “+” signs indicate that the particular feature is addressed in the corresponding paper.

	Programming	Bachelor Degree Students	Team Element	Moodle	Control and Experimental Groups	Comparison with Previous Year	Large Groups (>100 Students)	Students’ Questionnaire	Effect on Learning
Barata et al. [11]				+		+		+	+
Domínguez et al. [12]		+			+		+	+	+
Filsecker and Hickey [13]					+		+		+
Ibanez et al. [14]	+	+						+	+
Boticki et al. [16]			+				+		+
Hanus and Fox [4]		+		+	+			+	+
Cakiroglu et al. [17]					+				+
Mekler et al. [18]							+
Sánchez-Martín et al. [19]			+						+
Aldemir et al. [20]			+				+	+
Barna and Fodor [21]		+		+			+	+	+
Kyewski and Krämer [22]				+	+			+	0
Jurgelaitis et al. [23]		+		+		+	+	+	+
van Roy and Zaman [24]								+	0
Marín et al. [25]	+	+			+	+	+	+	+
Facey-Shaw et al. [26]	+	+		+		+	+	+	0
García-Iruela et al. [27]		+		+	+				0
Sanchez et al. [28]		+			+		+		0
Current paper	+	+	+	+		+	+	+	+

We selected features from the research field that are the most representative, both for our research and in general, and the features are arranged in increasing order of importance according to our view.

The first feature—“Programming”—defines the subject of study to which gamification has been applied, and is interesting primarily for our research. Ibanez et al. [14] and Marín et al. [25] gamified the teaching of C programming language, but our gamified subject is more general because it is OOP. Moreover, OOP represents the prevalent paradigm of programming, and therefore the success of gamification in our research would represent a method for teaching background subjects for programmers.

The second feature—“Bachelor’s degree students”—represents the students that undertook a gamified course. In 10 of the 19 reviewed cases, the students were at bachelor’s degree level. This underlines a general tendency and supports our view that gamification should firstly be oriented towards bachelor degree courses at the university level. At the master’s level, there is a much smaller number of students that study at the university. Master’s students are also more motivated than bachelor’s degree students because they make a more informed choice about their subject of study, and therefore there are fewer courses oriented towards background studies.

Next, we stress the importance of a team element. This element was used in 4 cases from the 16 reviewed. The use of team element raised some controversial issues [20]: some participants were happy with their teammates, whilst others did not want to be responsible for teammates. The team element is very important for future programmers, as the programmers usually work in teams to implement the complex tasks required of modern programmers. We even presented the team element in two varieties (local and global), and observed the most complaints regarding the use of the “local team” element in comparison with the other game elements applied. As we can see from the results that were reviewed [16,19,20], the team element is important not only in the learning of programming.

The most frequently used platform for gamification was Moodle [4,11,21,22,23,26,27], which was especially characteristic of the newer research. Moodle is a worldwide recognized platform for education, and students are therefore familiar with this platform, meaning that its use requires the least effort for gamification. Moreover, van Roy and Zaman [24] concluded that an unfamiliar platform could have a negative impact on the results. All other platforms in the works reviewed were different.

The next two lines—“Control and experimental groups” and “Comparison with the previous year”—demonstrate two views towards comparison of the gamification results. However, there is a large number of research works [14,16,18,19,20,21,24] that do not provide comparison at all. We prefer comparison with the results of the previous years, as this could represent an especially reliable result if the topics and tutors are the same. This was not true in the research of Barata et al. [10], but we speculate that the organization of the control group was artificial, because it is extremely improbable that such a group was organized in complete isolation given the prevalence of social networks.

We underline a feature of large groups, since gamification for small groups has a lesser need and is easier to implement: Large groups usually have a larger number of less motivated participants. In such a case, additional means of motivation can help—a view shared by a large part of the research works outlined above [4,12,13,14,20,21,23,25]. Moreover, the implementation of gamification is more difficult to support for large groups. Some other factors also come to play, for example the unwillingness of some tutors to encourage engagement with gamification.

The feature “Student questionnaire” shows the importance of feedback from the participants of the gamified course (Figure 6). Surprisingly, the student questionnaire was not organized for many research works [13,16,17,18,19,27,28]. By the end of the gamified semester, we carried out a formal questionnaire to gather student feedback about the use of our selected game elements. The students had to rate the use of the game elements using a five-point Likert scale (1—definitely not, 5—definitely yes). We got 251 answers out of 389 students. The students were unhappy with the game element of “local team” (2), since this element requires communication and sharing of knowledge with a participant who was unknown before, who may possess a different level of knowledge, or who may have a different view to the studies. However, in real-life situations we usually do not choose work fellows, and thus we have to accommodate ourselves to different characters and colleagues. Therefore, students will be better prepared for real-life situations if we use this game element for students’ training. A different situation arises with the “global team” element (5). The students were very interested in this game element, partly because the value of the winning prize was very high. It can be concluded that the results on the hypothesis H2 varied and while the “global team” was a highly motivating and involving factor, “the “local team” did not have the effect that we were expecting due to the following reasons: a different level of initial student knowledge in programming, and the course was run at the first semester of the student studies and the students were not yet engaged into collaboration.

Figure 6. Distribution of game element rating based on Likert scale.

The students’ opinion about the other two game elements (XP points (3) and interactive content (3)) was moderately positive. We can make the conclusion that when the value of the prize is high, the students show a greater interest. It gives us hope that such a passion for the prize motivates a zeal in learning, because the prize is won by those who achieve the better grades.

The last and most interesting feature is the effect that gamification has on learning. It must be noted that many authors declare a moderately positive effect on learning. We join this group, nevertheless we did not obtain a statistically significant increase in the grade average. Jurgelaitis et al. [23] were the most optimistic and declared an increase of grade average by 0.3 in comparison with the non-gamified course of the previous year. Mekler et al. [18] and Aldemir et al. [20] were cautious and concluded that the design of the gamified course should be based on the fields of psychology, education, and game studies. Kyewski and Krämer [22] and van Roy and Zaman [24] professed null results and proposed the longitudinal studies on the relevance of the game elements. Hanus and Fox [4] announced a negative impact of gamification on learning outcome. Almost all authors agree that care and thoroughness are needed in the design of the gamified course. Additionally, it is necessary to remember that the preparation of a gamified course incurs a time cost on the part of to the instructor [38]. We conclude that support from management staff and more comprehensive investigations are needed in the design of the gamified course.

5. Conclusions

Studies at the Informatics faculty are very popular, and many students express a wish to become professional programmers. As a result of this large quantity, not all newcomers are motivated. Moreover, studies at the Faculty of Informatics require great effort as a result of their difficulty. In order to increase motivation and ultimately to better academic performance, the OOP course of the first semester was gamified. The gamified course was implemented via a Moodle platform. As basic Moodle functionality was not sufficient for supporting required gamification elements; additional plugins were installed that allowed for four gamification elements to be introduced into the OOP course. These elements are as follows: XP points, interactive content, local team, and global team. XP points, levels, and leaderboards are normally implemented in digital games, and the interactive content was used to ease the learning process. Collaborative work is a very important skill for future professional programmers, and therefore a large amount of attention was devoted to the collaborative process during studies. The “local team” is specific for the OOP course and was used to inspire close collaboration within the boundaries of the small groups of students. The global team was used to encourage cooperation within the boundaries of a large group of students. This game element can be applied to any course.

The research showed that some students were collecting XP points just for fun, and thus the collection of XP points was not related to additional knowledge gain. Such students were disqualified. However, it is necessary to note that, to the joy of the professor, such students were few among the hundreds that took the course.

The interactive content did not attract students’ attention as was expected. This game element is directly related to the teaching methodology of the OOP course, as the defense of every laboratory assignment begins with a multiple-choice test. Many students are not very successful in this test, but the interactive content provides many forms of the test that can be studied in the preparation stage for the defense of the laboratory assignment. However, more than a third of the students (140) did not complete a single element of the interactive content successfully.

The “local team” element faced the most opposition among students and tutors alike. Half of the 20 tutors were against this game element, and their main argument was that students are not ready yet to be cooperative as the students coming in the first semester possess a different level of knowledge. This game element is very important for sharing knowledge among the students, and consequently the impact of this element is not easy to measure. The final academic performance was in this case the measure, as we are primarily interested in the final result.

The global team was the most favorably viewed game element. The interest, we suppose, was raised by the high value of the prize for the students of the winning global team—the ability to avoid an exam. The grade for the exam was assigned based on the students’ results over the semester. We recommend using global team game element for teaching programming as it contributes to an increased student engagement and interest already in the first academic year.

In order to have a more impartial result, a comparison of the academic performance was carried out with two previous years that were taught without gamification. The grade mean was calculated and compared, and although the gamified course showed an increase in the grade mean in comparison with the two previous non-gamified course years, this increase was not statistically significant. Nevertheless, we are happy with gamification because it resulted in two positive impacts. The key benefits were as follows:

• The number of students that will have to repeat the OOP course decreased by 2% in comparison with the 2018 year, and by 7% in comparison with the 2019 year.
• An unexpected byproduct was obtained during the exam session. Usually, we have around 60–70 students taking a second attempt to pass their exam. This year, however, we had only 30 students. This is an excellent result for exam organizers, and therefore gamification had an extremely positive effect on the learning of the OOP course.
• A global team clearly stood out as a game element contributing to an increased student engagement and interest already in the first academic year.
• Interactive content and XP points game elements had a positive impact on those students who chose to get involved in these gamified activities.

The obtained results demonstrate that the gamification of OOP course resulted in more sustainable behavior of the students. As a result, we will proceed with the gamified OOP course next year.

The limitation of our research work was that we had a large number of tutors, and not all of the tutors supported the gamification of the course. Therefore, not all of the tutors encouraged students to augment their knowledge using the elements of gamification. Subsequently, not all of the participants were participating under equal conditions. The “local team” was the only obligatory game element, and as a consequence it had the largest opposition.

For future research, we would note that gamification adds attractiveness to the course. Therefore, the gamification of the course is both beneficial and desirable. We suggest that an investigation might consider determining which game elements are universal and are recommended to any course, and which game elements are subject-specific and are recommended to the particular course of education. Additionally, we propose that the use of game elements in gamified courses should not be made mandatory for the students.