Gamification in Engineering Education: The Use of Classcraft Platform to Improve Motivation and Academic Performance

Abstract

Pedagogical innovation involving information and communications technology (ICT) may offer teachers the opportunity to create engaging learning environments in engineering courses. In this paper, we present a gamification teaching experience whose primary objective is to improve motivation, and we obtained results for students of a mathematics course during their first year at university. For this case study, we used Classcraft^®, which is a role-playing game supported by a digital platform and a mobile application that has been developed to answer teachers’ classroom management needs. We hypothesized that using this application as ICT could enhance learning and promote the development of the four “super skills” (or the Four C’s): critical thinking, communication, collaboration, and creativity. In order to explore the educational effectiveness of the methodology, a comparison between a gamification group of students and a control group was carried out. Our results showed that the mean mark obtained by the control group students was lower than that obtained by the gamification group students. In addition, the Nemenyi test showed that the Four C’s were improved thanks to the Classcraft^® activities and group project. Overall, course participants positively evaluated the use of the gamification platform.

1. Introduction

Today, a review of education quality is strongly required so that we can provide instructions that enable effective learning in the future [1]. In this sense, governments should encourage innovation in educational techniques, methodologies, and strategies in order to promote the competencies adapted to the emerging knowledge-based societies of the 21st century [2]. Critical thinking, communication, collaboration, and creativity were established as the most important skills for education in the 21st century by the Partnership for 21st Century Skills (also called P21) in 2002. Therefore, curricula should be changed to ensure that what students learn is important for them as individuals and members of modern society [3]. This matter is already being considered in mathematics [4].

As previously mentioned, critical thinking, communication, collaboration, and creativity (the Four C’s) are important aspects of modern teaching and education. The National Council for Excellence in Critical Thinking defined critical thinking as an intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from or generated by observation, experience, reflection, reasoning, or communication as a guide to belief and action [5]. In this sense, critical thinking involves clarity, accuracy, precision, consistency, relevance, sound evidence, reasoning, depth, breadth, and fairness. Another key skill is communication, which has not attracted the same amount of attention as critical thinking, creativity, and collaboration. This skill can be divided into mediated, digital, written, and oral communication. In order to assess whether students gain communication competences, educators should effectively teach how to communicate. In this sense, educators should focus on building a stronger and more empirically grounded framework for teaching by learning these skills. Collaboration is attracting more and more attention as an important educational outcome and a key educational skill; it is important for not only school but also for career and life success. This ability is critically important to effectively work with others and to facilitate team building and team-based work. Finally, creativity is well-known for being a key 21st century skill. Recently, it has been shown to be integral to a wide range of skills, including scientific thinking, designed thinking, and mathematics [6]. In 2010, more than 15,000 CEOs from 33 industries and 60 countries were interviewed for a study about leadership qualities. The results showed that the most important quality needed to meet the challenge of increasing complexity and uncertainty in the world was creativity [7]. According to Mihaly Csikszentmihalyi, “Most of the things that are interesting, important, and human are the results of creativity” [8].

The implementation of information and communications technology (ICT) in classrooms through mobile applications is intended to improve students’ learning, motivation, and participation [9,10,11]. To support the aforementioned skills through gamification, researchers have proposed the use of Classcraft^®, which is a web and mobile application that allows an educator to manage a role-playing game with students [12,13]. This application enables the creation of student teams, with an avatar assigned to every student. In this way, students can obtain experience points as rewards for positive achievements and can even be penalized with negative points in the case of faults. It is important to mention the ease of use of Classcraft^®, including for non-technical educators. The relevance of this use is supported by its success in some studies [5] and the importance of using new technologies to teach 21st century skills [7].

The main goal of this paper was to determine how the use of Classcraft^® could motivate students in a mathematics course during their first-year engineering degrees at Universidad Loyola Andalucía. For this purpose, we conducted a comparison of marks obtained by students who followed a traditional teaching methodology and by students who used the Classcraft^® application. Furthermore, the reliability of this method was assessed using different statistical methods.

2. Materials and Methods

2.1. Methodology

2.1.1. Participants

The participants of this study were students from Universidad Loyola Andalucía (Spain), which is a private university with sites at Seville and Córdoba. This study was carried out in the 2021/2022 academic year. Undergraduate and postgraduate programs can be studied in both Spanish and English at this university. Different degrees related to engineering, education, social sciences, law and economics, and business can be obtained.

The participants were the 38 students enrolled in the courses of Mathematics I and Mathematics II during their first-year engineering degrees. These degrees are degree in electromechanical engineering, engineering degree on industrial organization, degree in mechatronic and robotic engineering, and degree in computer engineering and virtual technology. From a descriptive point of view, 18.42% of the participants were women and 81.58% were men. The age range of the participants was from 19 to 21. The mean age was 20.12, and the coefficient of variation of the age was 0.0543.

2.1.2. Procedure

We intended to compare the marks obtained by the students who followed a traditional learning model (control group) with those obtained by students who used the innovative methodology (gamification group). To conduct this comparison, we designed five different tests to be solved by students at the end of each chapter of their study. In addition, students had to form groups and deal with some projects. All students (control group and gamification group) completed the same activities but in different forms. Control groups worked in class using paper and pen, while the gamification group used the Classcraft^® platform.

Firstly, students in the gamification group had to choose a character: warrior, mage, or healer.

To build motivation, students were able to obtain experience points if they showed different class behaviors. For example:

-

Asking an interesting question in class;

-

Correctly answering a question in class;

-

Correcting a mistake in class;

-

Showing positive attitudes in one of the Four C’s;

-

Helping another student with a class task;

-

Passing a class test;

-

Finishing a project before its deadline.

Students could also lose experience points if they demonstrated the following behaviors:

-

Arriving late to class;

-

Using a mobile phone or laptop during class;

-

Using inappropriate language;

-

Showing weakness in one of the Four C’s;

-

Failing a class test;

-

Finishing a project after its deadline.

As a consequence of these points, students were able to learn in-game powers, level up, and fall down in battle [14]. Their final marks were calculated by adding the score obtained in an exam (70%) and the points obtained by their behavior in class (30%).

Furthermore, the promotion of Four C-related behaviors in students was analyzed as follows. A real-world problem in the field of engineering regarding a vibration model was explained to the students, who then had to complete a research study and solve some problems related to this matter. They had to search for information about the problem, collect data of previous knowledge on the problem, and apply the techniques explained during the course in order to solve the problem. Moreover, they participated in the evaluation process of their classmates’ projects. Each group had to present its work, and another group acted as reviewers, describing the strengths and weaknesses of the project in order to develop their critical skills. Finally, each group also evaluated their own participation in the process. Related to communication skills, students had to write a technical report and explain their research and obtain results from their classmates. For the gamification group of students, all of these steps were conducted with the Classcraft^® platform. Concerning the communication skill, students had to collaboratively choose the name and the banner of their groups on the Classcraft^® platform such that the students of each group had different characters. Moreover, students had to distribute all potentially learnable powers and abilities to each member of each group. Consequently, students were encouraged to have a collaborative attitude. Regarding creativity, students were asked to design new real-life problems similar to the one that they had to solve. We also asked students to propose new characters and powers for the Classcraft^® platform. We submitted all their ideas to the official forum so that our students could receive some feedback on their initiatives. For the control group, the research study used to evaluate the students’ aptitude for the Four C’s was carried out as a traditional writing work.

2.1.3. Likert Scale

Different statistical tests were applied in order to study the influence of the Classcraft^® platform on the marks and motivation of our students. The Wilcoxon signed-rank test was carried out to evaluate the differences in the median mark, and the Friedman test and the Nemenyi test were conducted to assess the differences in the Four C’s scores. A level of significance of α = 0.05 was considered in all performed tests.

A satisfaction questionnaire was provided to the students in order to understand what they thought about this innovative methodology and the use of the Classcraft platform. They had to complete the questions by assigning a number to each of the ten statements where

-

1 means strongly disagree;

-

2 means somewhat disagree;

-

3 means neither agree nor disagree;

-

4 means somewhat agree;

-

5 means strongly agree.

3. Results and Discussion

Table 1 summarizes the basic descriptive statistics of the marks obtained by the students who followed the traditional learning methodology (control group) and those who learned using Classcraft^® (gamification group). It is important to note that the mean mark obtained by the gamification group was higher than that obtained by the control group, suggesting that the use of gamification enhanced the learning ability of the students. In addition, according to the Shapiro–Wilk test, the distribution of the marks of the gamification group followed a normal model but those obtained by the control group did not. These results were validated the Shapiro–Wilk p-values and the level of significance of α = 0.05. Moreover, considering the values of the standard deviation and the coefficient of variation, it seems that the data were more heterogeneous in the gamification group than in the control group. Considering these results, students who used Classcraft^® showed generally better learning capabilities than those who followed a traditional methodology.

Table 1. Descriptive statistics about the mark obtained by the students using a traditional methodology (control group) and using Classcraft^® (gamification group).

Statistic	Control Group	Gamification Group
Number of cases	38	38
Minimum	0.6	1.5
Maximum	9	10
Arithmetic Mean	4.69	5.63
Standard Deviation	2.60	2.06
Coefficient of Variation	0.56	0.37
Shapiro–Wilk Statistic	0.92	0.96
Shapiro–Wilk p-value	0.0098	0.2687

After this descriptive study, the Wilcoxon signed-rank test was applied to analyze the median difference of both samples. Considering that the sample size was 38, which was large enough for us to use normal approximation (Z-statistic), the results of this test are shown in Table 2.

Table 2. Results of the Wilcoxon signed-rank test.

Sum of positive ranks (W+)	133
Sum of negative ranks (W−)	608
Test statistic	103
Z-statistic	−3.4444

Since the significance level was α = 0.05, the critical value was 1.96, and the null hypothesis assuming that the medians of the two samples would be the same was rejected.

The average marks regarding the Four C’s obtained by students of the gamification and control groups are shown in Figure 1. All marks were higher than 2.5 for all students except for those regarding communication for the students of the control group; the maximum score was five. There was a substantial improvement in the results of the gamification group in comparison with those of the control group, proving that the use of gamification in a classroom can enhance behavior related to the Four C’s. The predominant improvement following the use of Classcraft^® was in creativity (from 2.45 to 3.42), followed by communication (from 2.96 to 3.90). It should be also highlighted that critical thinking seemed to be the best skill in both groups.

Figure 1. Average Four C’s marks: (1) critical thinking, (2) communication, (3) collaboration, and (4) creativity.

The significance of the results was analyzed by using the Friedman test and the Nemenyi statistical test. The Friedman test assesses whether the differences in a group of results are significant, while the Nemenyi test evaluates which comparable pairs have significant differences. A significant level of α = 0.05 was used. The results of the Friedman test proved that there were statistical differences among the obtained Four C-related marks. The results of the Nemenyi test showed that the students’ Four C-related skills were improved thanks to the activities and group project. The Nemenyi test also revealed that critical thinking was better promoted than other skills.

A satisfaction questionnaire was filled in by the students at the end of the course. Table 3 shows the average marks for every question in the survey. Generally, the Classcraft^® platform was very positively evaluated by students. They also highlighted that the use of this platform provoked an increase in their motivation. In addition, they liked working in groups, especially in comparison with the control students. Finally, students thought that Classcraft^® could be used in other courses. These results show that it is a good idea to implement this information and communications technology (ICT) in classrooms, regardless the subject.

Table 3. Average marks of the quantitative questions of the opinion questionnaire.

Question	Average Mark
	Control Group	Gamification Group
The use of Classcraft platform has increased my interest and motivation for the course.	-	4.5
My implication in the use of this tool has been appropriated.	-	4.12
I am satisfied with the results I have obtained after using Classcraft.	-	3.96
I think that this platform is attractive and advisable for teaching purposes.	-	4.02
The behaviors designed has been useful for my study and progress in the course.	-	3.18
The battles and sentences dealt with have helped me during the course.	-	3.97
The experience points have been configured properly.	-	4.21
I have enjoyed working in teams with my classmates.	3.18	4.38
I would like that other courses during my degree use this platform.	-	4.62

Finally, some answers to question 10 (Do you have any additional comments?) are shown.

• “I have enjoyed very much using the Classcraft platform, and it has helped to improve my marks in Mathematics. However, I think that the powers should be revised and configured in a better way”.
• “It was great to deal with Classcraft. It allowed me to improve my interest and motivation in Mathematics”.
• “Our teachers, by using Classcraft, have been able to get my attention and recover my interest in Mathematics”.
• “I have had some problems dealing with my groupmates. I think that teachers should design the groups in the future”.
• “I think that the game of Classcraft has been useful, and I got a better mark after using it”.
• “I wish that other teachers used Classcraft”.
• “We have had some disagreements working in groups to prepare the oral presentation”.

Students pointed out two areas that should be revised: some aspects of the configuration of the game and some communication issues when working in groups. In a future study, all recommendations provided by students in order to improve their motivation and results will be considered.

4. Conclusions

In this study, the use of Classcraft^® as an ICT to improve student education in two courses during first-year engineering degrees resulted in a measurable increase in knowledge. An analysis of the results illustrated that the gamification group’s students’ critical thinking, communication, collaboration, and creativity (the Four C’s) skills were improved following the use of the aforementioned platform in comparison with a control group. In addition, the platform was well-received by students. Therefore, Classcraft^® can be considered to be a promising new tool for the classroom implementation of active learning strategies, as well as an evaluation system in engineering degrees.