Influence of Gamification on the Commitment of the Students of a Programming Course: A Case Study

Abstract

In the realm of programming education, enhancing student commitment is pivotal for both academic success and practical application. Traditional methodologies often fall short in catering to the evolving needs of today’s learners. Addressing this gap, this paper introduces a web-based gamified tool designed to transform the pedagogical approach in programming courses. The study’s theoretical contribution lies in its exploration of gamification as a strategy to augment student commitment, bridging the divide between conventional teaching and contemporary learning needs. From a practical standpoint, empirical data encompassing performance metrics, dropout rates, and student feedback demonstrate the tool’s significant positive impact. Notably, students exhibited improved pass rates and reduced dropout tendencies, underscoring the tool’s practical efficacy. Furthermore, feedback highlighted the tool’s usability and the enhanced satisfaction derived from its engagement. Aligned with the existing literature, the findings advocate for the broader adoption of gamification in programming education. This research substantiates the potential of gamification as both a theoretical model and a practical strategy, enriching the discourse on modern programming education.

1. Introduction

In the evolving landscape of higher education, student commitment has become a paramount concern for educators and institutions alike. The digital age, characterized by rapid technological advancements and the proliferation of online learning platforms, has brought forth both challenges and opportunities in fostering student commitment. One such opportunity lies in learning with gamification, a pedagogical approach that integrates game mechanics into non-game contexts, aiming to enhance user engagement, motivation, and learning outcomes. As programming courses, often perceived as complex and challenging, grapple with issues of student commitment and dropout, the potential of gamification in this domain warrants thorough exploration.

While the broader educational sector grapples with engagement challenges, programming courses present their own unique set of hurdles. The abstract nature of programming concepts, coupled with the rigorous analytical thinking it demands, can often lead to feelings of being overwhelmed among students. Traditional teaching methodologies, though effective for some, might not cater to the diverse learning needs of the current generation. As educators seek to bridge this gap, the integration of modern pedagogical approaches becomes imperative. Among these, gamification emerges not just as a trend but as a potential cornerstone for reshaping the learning experience in programming courses, making it more interactive, engaging, and aligned with the preferences of today’s learners.

Recent studies have highlighted the alarming dropout rates and lack of motivation among engineering students, particularly in computer science disciplines [1], emphasizing the need for innovative educational strategies to improve student engagement and performance. Concurrently, research has also underscored the gender disparities in engineering career choices, pointing to a range of influencing factors such as interest, social influences, and outcome expectations [2]. The underlying reasons for such dropout rates encompass a spectrum of factors, from low academic performance and socio–economic challenges to lack of motivation and poor social support in the academic environment [3]. These challenges underscore the urgency for innovative educational interventions, such as gamification, to bolster student commitment and reduce the dropout rate in programming courses and beyond.

In the domain of software modeling, an area closely related to programming, the introduction of gamified platforms has shown promise for enhancing student engagement and comprehension. A study by Feichas and Seabra evaluated a gamified web platform tailored for studying the Unified Modeling Language (UML), a standard tool often integrated into programming and software engineering curricula [4]. Their findings revealed that students appreciated the platform’s usability and recognized its significant contribution to their learning process. Other studies oriented towards the analysis of students’ perceptions found that the use of gamified tools improves their motivation [5] and engagement [6].

Expanding the scope away from computer science and engineering disciplines, student dropout in higher education remains a global concern. A study from Central Java, Indonesia, further corroborates the complexity of this issue, emphasizing the diverse challenges students face across different educational settings [7]. Moreover, with rapid advancements in technology, data-driven models are emerging as potent tools to understand and predict student dropout risks. A recent study employed data mining and machine learning techniques to categorize university students based on their dropout risk, underscoring the potential of technology for addressing educational challenges [8].

Delving deeper into the psychoeducational aspects of student dropouts, Vidal et al. conducted a study at a Science, Technology, Engineering and Mathematics (STEM) higher-education institution that highlighted the nuanced interplay of emotional intelligence, learning goals, and causal attributions in influencing a student’s decision to persist or leave [9]. Such insights provide a fresh perspective on the dropout phenomenon, suggesting that beyond the commonly discussed socio–demographic and academic challenges, students’ emotional and cognitive frameworks play a pivotal role in their academic journey.

Further emphasizing the intricacies of student dropouts in higher education, a detailed analysis at the Faculty of Electrical and Computer Engineering of the University of Prishtina, Kosovo, from 2001 to 2015 revealed that the reasons for dropout were not predominantly influenced by socio–economic backgrounds or urban–rural divides. Instead, they were strongly tied to high school programs and academic performance in specific courses [10]. This underscores the importance of academic preparedness and the potential impact of high school curricula on university retention rates.

Shifting the lens to teacher training colleges, particularly in most under-resourced countries, the issue of student-teacher dropout emerges as a significant concern. A comprehensive study conducted by Singh and Alhulail on student-teachers in public teacher training colleges of Ethiopia, an under-resourced country, underscored the composite and relative impacts of various personal, academic, and socio–economic predictive variables on student-teacher dropout [11]. The research identified academic performance and aspirations as paramount predictors, while factors traditionally believed to influence dropout, such as cultural values and financing education, were found to be less impactful.

This paper delves into gamification as a potential solution to some of the challenges faced in programming courses. By presenting a case study, it seeks to understand the influence of gamification on the commitment of students in a programming course. The case study research methodology has been widely applied in many fields. In [12], Yin provides a comprehensive overview of case study research methodology, including its applications in educational settings. The research also offers guidance for designing and conducting case studies. On the other hand, Stake’s book [13] emphasizes the importance of case study research as a qualitative approach for understanding complex phenomena in real-life contexts. The primary focus of the presented case study is to ascertain whether the use of a gamified tool can enhance students’ commitment to the subject. Additionally, the study aims to gather insights into students’ perceptions and experiences regarding the usability, effectiveness, and overall satisfaction with the web-based gamified tool. Through a comprehensive analysis, the research endeavors to shed light on the potential of gamification as a transformative pedagogical approach in the context of programming education, offering valuable insights for educators and institutions in the field of computer science education.

Hypothesis

The integration of gamification in educational settings, particularly in programming courses, has been a topic of growing interest in recent years [14]. The potential benefits of gamification, such as increased student engagement, motivation, and improved learning outcomes, have been highlighted in various studies [15,16,17,18,19]. However, the specific influence of gamification on student commitment in programming courses remains an area that warrants further exploration. In this context, the present research aims to investigate the impact of a web-based gamified tool on the commitment of students to a programming course.

The central hypothesis of this study is grounded in the belief that the introduction of gamification elements into a programming course can significantly enhance students’ commitment and overall learning experience. By incorporating game-like mechanics and dynamics, such as rewards, challenges, and leaderboards, it is anticipated that students will exhibit increased motivation, engagement, and persistence in their coursework. Furthermore, the study seeks to understand students’ perceptions and experiences with the gamified tool, particularly in terms of its usability, effectiveness, and overall satisfaction.

To provide a comprehensive understanding of the influence of gamification on student commitment in a programming course, the following research questions have been formulated:

• What is the tool’s influence on the student’s commitment to the subject?
• What are the students’ perceptions and experiences regarding the usability and the effectiveness of the web-based gamified tool?
• What are the students’ perceptions and experiences regarding their satisfaction with the web-based gamified tool?

The subsequent sections of this paper are organized in the following manner. Section 2 provides a summary of the web-based gamification tool and its integration into the programming course. This section also outlines the quantitative evaluation methodology employed, the participant demographics, and the survey conducted to gauge students’ experiences and perceptions. Section 3 presents a comprehensive analysis of the results derived from the study. Finally, Section 4 discusses the implications of the findings, compares the results with prior studies, and addresses the study’s hypotheses and limitations. The conclusions drawn from the research are presented, along with suggestions for potential avenues for future work.

2. Materials and Methods

The core methodology of this research is centered around a novel web-based application that seamlessly integrates automated programming assessment features with gamification concepts. This tool, detailed by the same authors in foundational research [20], was meticulously designed to offer students in a programming course a dynamic and engaging learning environment. The gamified tool incorporates various game design elements, such as points, achievements, badges, leaderboards, and a store feature, all tailored to foster heightened engagement and motivation among students.

To discern the impact of the gamified tool on students’ commitment and learning experiences, a quantitative evaluation methodology was employed. The primary participants of this study were students enrolled in a programming course who interacted with the tool throughout its duration. Their performance metrics, commitment levels, and feedback were meticulously analyzed to derive meaningful insights.

A survey akin to the one utilized in the original study was conducted to quantitatively gauge the students’ experiences and perceptions. This survey aimed to glean insights into the effects of gamification and automated assessments on students’ commitment, perceptions, usability, effectiveness, and satisfaction within the programming course context.

The study was conducted at a public university in Spain, with 215 students participating from three distinct computer engineering degree programs, but having the same previous subjects about programming. Stratified random sampling [21,22] was employed to ensure a representative sample for the case study, with students divided into strata corresponding to their specific degree programs. This approach provided a comprehensive perspective on the effects of the gamified learning environment across diverse educational paths.

The participants were undergraduate students enrolled in an object-oriented programming course. This course was gamified using the aforementioned web-based tool, which incorporated motivational factors with the aim of enhancing their commitment and overall learning experience. The participants could achieve an extra 0.4 points through gamification out of a total of 10 points for the subject’s final mark as extra credit obtained by reaching certain levels or milestones of programming [20].

To rigorously assess the impact of the gamified tool on student performance, the study employed a comparative approach, dividing the participants into two distinct groups: an experimental group and a control group. The experimental group consisted of the 69 students who achieved the extra 0.4 points, while the control group comprised the 146 students who did not secure these additional points. This distinction allowed for a clear comparison of performance metrics, commitment levels, and overall experiences between the two groups.

The performance metrics were categorized into three primary outcomes: Pass, Drop out, and Fail. For each outcome, percentages were calculated for both the experimental and control groups, providing a comprehensive view of the influence of gamification on student performance. This comparative approach ensured a robust evaluation of the gamified tool’s effectiveness, with the differences in performance between the experimental and control groups serving as a testament to the tool’s impact.

3. Results

The subsequent section delineates the quantitative findings obtained from the students’ questionnaires. These findings provide insights into students’ commitment, perceptions, usability, effectiveness, and satisfaction with the gamification tool used in the programming course. Additionally, the impact of gamification on overall student performance, specifically focusing on pass rates, dropout rates, and failure rates, is examined in a dedicated subsection.

3.1. Questionnaire Results

In this subsection, the survey data are analyzed with the aim of elucidating student commitment, perceptions, usability, effectiveness, and satisfaction with the gamification tool in the programming course. The students were asked to indicate their degree of agreement regarding the following statements:

• The tool is easy to use.
• I found the tests to be very useful.
• I find the points and achievements system to be appropriate.
• Using the tool has been a positive experience.
• I felt happy when I earned points.
• I felt satisfaction when the code passed the tests.
• I would like similar tools to be used in other subjects.
• I did not like that my results were public and could be compared to the rest of the students.

Figure 1, Figure 2, Figure 3, Figure 4, Figure 5, Figure 6, Figure 7 and Figure 8 quantify the students’ levels of agreement or disagreement with a range of statements pertinent to these aspects.

In Figure 1, the ease of use of the gamification tool is evaluated. It is observed that a majority of respondents (56.64%) completely agree that it was easy to use. Additionally, 34.51% agreed with this statement. A smaller percentage (7.07%) maintained a neutral stance. Conversely, both the disagree and completely disagree categories were minimal, each with 0.89%.

Regarding Figure 2, which focuses on the utility of unit tests within the gamified exercises, a significant majority of students found them to be valuable. Specifically, 34.51% of the respondents completely agreed that the tests were very useful, and a further 54.81% concurred with this sentiment. It should be noted that these unit tests include both standard and concealed versions, with the concealed tests not revealing the input and expected output to the learners.

In Figure 3, the appropriateness of the points and achievements system implemented in the gamification tool is assessed. A total of 79.65% of the students, comprised of 44.25% who completely agreed and 35.40% who agreed, found the system to be suitable for the course context. Notably, 16.81% of the participants remained neutral on this subject, while both the disagree and completely disagree categories each accounted for 1.77% of the responses.

Assessing the overall experience with the gamification tool, as depicted in Figure 4, a substantial majority of students reported positive feedback. Cumulatively, 59.29% completely agreed and 36.28% agreed, which results in 95.57% of respondents indicating that using the tool had a beneficial experience.

Examining the emotions associated with earning points, as highlighted in Figure 5, an overwhelming combined total of 85.94% of the respondents conveyed feelings of happiness. Delving deeper into these data, 42.48% of the students completely agreed and an additional 43.36% agreed with the sentiment of joy upon earning points in the gamified tool. This underscores the motivating power of the points system in gamification to foster positive emotions among participants.

Delving into the feelings of accomplishment when code successfully passed the tests, as shown in Figure 6, a combined total of 97.34% of the respondents expressed satisfaction. Specifically, 71.68% of the students completely agreed, while an additional 25.66% agreed with this sentiment. This illustrates the significant impact that successful test completion, both standard and concealed, had on student satisfaction within the gamified context.

Regarding Figure 7, which explores the students’ inclination towards the adoption of similar gamified tools in other subjects, an impressive consensus emerges. Specifically, a significant 73.45% of respondents completely agreed with the statement, and an additional 23.90% also agreed. This leads to a cumulative 97.35% of the students showcasing a strong preference for the integration of gamified tools in diverse academic contexts, reflecting the widespread appreciation of the gamification approach in the programming course.

In Figure 8, the student’s comfort level regarding the public display of their results and subsequent comparisons with peers is examined. A total of 15.70% of respondents, made up of 11.27% who completely agree and 4.43% who agreed, expressed discomfort with their results being publicly displayed. Significantly, 36.28% of the participants remained neutral on this matter. Meanwhile, 20.35% disagreed, and a further 25.67% completely disagreed with the sentiment of discomfort related to the public leaderboard. The majority of the responses lean towards a preference for the public display of results in the gamification tool’s dual leaderboard system, which features a top 10 leaderboard and a personal one for those not within the top rankings.

3.2. Performance Impact of Gamification

In Figure 9, the influence of the extra 0.4 points acquired via gamification on student performance is analyzed. Three distinct outcomes are presented: Pass, Drop out, and Fail. For each outcome, two percentages are shown: one reflecting the proportion of students who secured the extra points and another denoting the percentage who did not.

The data for those who passed the programming course offers a stark contrast. Among the students who achieved the 0.4 extra points, 82.61% passed. Conversely, of the students who did not secure the additional points, only 35.48% managed to pass the programming course. When evaluating dropout rates, 10.14% of the students who earned the extra points dropped out, in comparison to a significantly higher 48.39% from the group that did not. As for failure rates, 7.25% of the students with extra points failed the programming course, whereas this percentage slightly increased to 16.13% for those without the additional points.

4. Discussion and Conclusions

This section discusses the implications of the findings in relation to the research questions, compares the results with prior studies, and addresses the study’s hypotheses and limitations. The aim is to contextualize the results within the broader framework of gamification research, highlighting the unique contributions and insights of this investigation.

Regarding the first research question, “What is the tool’s influence on the student’s commitment to the subject?”, the data derived from the performance impact of gamification provide a compelling answer. The stark contrast in the pass rates between students who achieved the extra 0.4 points through gamification and those who did not is indicative of the tool’s positive influence on student commitment. Specifically, 82.61% of students who scored the additional points successfully passed the programming course, compared to a significantly lower pass rate of 35.48% among students who did not achieve the extra points. This substantial difference underscores the effectiveness of the gamified tool in enhancing commitment to the subject.

Furthermore, the dropout rates provide additional insights into the tool’s impact on student commitment. A total of 10.14% of students who earned the extra points opted to drop out of the course, whereas a considerably higher percentage of 48.39% from the group that did not achieve the extra points chose to drop out. These data suggest that the gamified tool not only motivates students to excel in their studies but also significantly reduces the likelihood of them abandoning the course.

The failure rates further corroborate the tool’s positive influence. Only 7.25% of students who secured the extra points failed the programming course, while this percentage slightly increased to 16.13% for those who did not achieve the additional points. The reduced failure rate among students who engaged with the gamified tool is a testament to its efficacy in fostering a deeper commitment to the subject matter.

In corroborating the positive influence of gamification on student commitment, our findings align with multiple studies in the field. Moliner-Heredia and Abellán-Nebot observed a significant increase in both course attendance (from 38% to 66%) and pass rates (from 22% to 34%) in manufacturing engineering courses after introducing gamification tools like Kahoot quizzes and Moodle activities [23]. Similarly, Nesterowicz et al. reported that gamification led to a remarkable rise in Massive Open Online Course (MOOC) completion rates, with 13.7% of participants completing gamified MOOCs compared to a mere 1.7% for non-gamified ones [24]. These studies, in conjunction with our own, underscore the profound potential of gamification for not only enhancing student commitment but also for motivating them to excel academically.

Regarding the second research question, “What are the students’ perceptions and experiences regarding the usability and the effectiveness of the web-based gamified tool?”, the survey results provide valuable insights into the students’ experiences and perceptions of the tool’s usability and effectiveness.

A significant majority of students found the gamification tool user-friendly. Specifically, 56.64% of respondents completely agreed that the tool was easy to use, and an additional 34.51% concurred with this sentiment. This combined total of 91.15% indicates that the tool’s design and interface are intuitive and accessible, allowing students to engage with the gamified elements without facing usability barriers. The minimal percentages of disagreement (0.89% each for disagree and completely disagree) further underscore the tool’s user-centric design.

Turning to the utility of unit tests within the gamified exercises, the results suggest that students found these tests valuable for their learning experience. A combined total of 89.38% of students (34.51% completely agreed and 54.87% agreed) perceived the tests, both standard and concealed, as beneficial. The concealed tests, which did not reveal the input and expected output to learners, likely added an element of challenge and intrigue, prompting students to think critically and problem solve.

Feedback on the points and achievements system further highlights the tool’s effectiveness in the course context. A substantial 79.65% of students (44.25% completely agreed and 35.40% agreed) found the system appropriate and motivating. The points and achievements system, which serves as a tangible measure of progress and accomplishment, appears to resonate with students, encouraging them to actively participate and strive for excellence.

Assessing the overall experience with the gamification tool reveals overwhelmingly positive feedback. Nearly all students, with 95.57% in agreement, indicated that their engagement with the tool had been beneficial. This high level of satisfaction suggests that the gamified elements not only enhanced the learning experience but also fostered a sense of achievement and motivation among students.

Our findings on the positive perceptions and experiences regarding the usability and effectiveness of the gamified tool resonate with other studies in the field. For instance, the effectiveness of game-based learning tools, such as Kahoot, in enhancing motivation and engagement in online tertiary education classrooms during the pandemic was also observed in a study conducted in Hong Kong [5]. Furthermore, the motivational impact of competitive behavior and the value of immediate feedback in a MOOC setting were highlighted in other research [25]. These insights emphasize the pivotal role of gamification in shaping positive student experiences across diverse educational contexts.

Regarding the third research question, “What are the students’ perceptions and experiences regarding the satisfaction with the web-based gamified tool?”, the survey data also shed light on the students’ overall satisfaction and experiences.

The emotions tied to earning points in the gamified system are overwhelmingly positive. Specifically, 85.84% of the respondents, comprising 42.48% who completely agreed and 43.36% who agreed, expressed feelings of happiness upon accumulating points. This positive emotional response underscores the effectiveness of the points system in evoking joy and satisfaction, further enhancing the students’ engagement with the tool.

When examining the sense of accomplishment students feel when their code successfully passes the tests, the feedback is notably positive. A vast majority, 97.34% of the respondents, consisting of 71.68% who completely agreed and 25.66% who agreed, conveyed satisfaction upon passing the coding tests. This sentiment emphasizes the importance of immediate feedback in the learning process. Successfully passing the tests, especially the concealed ones, provides students with a tangible measure of their progress, fostering a sense of achievement and reinforcing their learning.

The inclination towards the adoption of similar gamified tools in other subjects also provides insights into the students’ satisfaction levels. An impressive 97.35% of students, with 73.45% completely agreeing and 23.90% agreeing, expressed a preference for the integration of gamified tools in other academic contexts. This strong inclination reflects the students’ appreciation of the gamification approach, suggesting that they found it not only effective in the programming course but also see its potential benefits in other subjects.

However, when assessing students’ comfort levels regarding the public display of their results and comparisons with peers, the feedback is more varied. While a minority of 15.70% expressed discomfort with their results being publicly displayed, a significant 45.92% (20.35% disagreed and 25.67% completely disagreed) showcased a preference for the public display of results. This indicates that while some students may feel apprehensive about public comparisons, a large segment finds value in the competitive aspect of the leaderboard system, viewing it as a motivational tool.

Building on the findings from the second research question, the overall satisfaction levels with the gamification tool, as previously highlighted, further reinforce the students’ positive perceptions and experiences. It is noteworthy to reiterate that an overwhelming 95.57% of respondents found their engagement with the gamified tool to be beneficial. These data, which were instrumental in addressing the tool’s usability and effectiveness, also provide a compelling answer to the third research question. The high levels of satisfaction underscore the tool’s capacity to not only enhance the learning experience but also to foster a profound sense of satisfaction among students. The repeated emphasis on this particular data point across different research questions underscores its significance for understanding the multifaceted impact of the gamification tool on students’ experiences.

Diving deeper into the nuances of gamification and its impact on student satisfaction, there exists a spectrum of perspectives. One research study suggests that while gamification can initially boost student engagement, its long-term effects on satisfaction might be mixed, with some students eventually perceiving gamified activities as mere coursework [6]. On the other hand, a study by Charles Baah and Subramaniam revealed that students were notably satisfied with gamified courses, emphasizing the fulfillment and enjoyment derived from their achievements during the learning process [26]. These contrasting findings highlight the intricate dynamics of gamification in education, suggesting that while gamified tools can foster positive learning experiences, their implementation requires thoughtful consideration to ensure sustained benefits for students.

In conclusion, this study has provided valuable insights into the effects of a web-based gamified tool on students’ commitment, perceptions, and experiences in a programming course. The data collected and analyzed from the questionnaire have demonstrated the significant positive impact of the gamified tool on various aspects of the learning experience, from enhancing student commitment to fostering a profound sense of satisfaction. The overwhelmingly positive responses from students underscore the potential of gamification as a powerful pedagogical tool in modern education.

Future work could focus on expanding the scope of the study by implementing the gamified tool across multiple universities. This would provide a more comprehensive dataset, allowing for more in-depth analysis and potentially revealing patterns or trends specific to different educational settings or cultures. Additionally, exploring the long-term effects of gamification on student satisfaction and commitment, beyond the initial phases of enthusiasm, would be a valuable avenue of research. This would help educators and researchers understand the sustainability of gamified tools for maintaining student interest and commitment over extended periods. Another potential area of exploration could be the integration of more advanced analytics and feedback mechanisms within the gamified tool, offering students more personalized learning experiences and further enhancing its effectiveness.