- freely available
Informatics 2019, 6(3), 32; https://doi.org/10.3390/informatics6030032
- What are the gamification elements mostly used in online learning environments?
- What are the effects of these game elements on learners’ behaviour in online learning environments?
- What factors need to be considered for designing effective gamification of online learning environments, specifically MOOCs?
2. Related Works
3. Methods and Data Collection
4. Data Analysis and Results
4.1. Q1: What Are the Game Elements Most Used in Online Learning Environments?
- Feedback can be defined as information delivered to users related to their progress, achievements, issues, or other aspects of their activities. Feedback can take several forms and can be delivered asdirect or indirect information. Sometimes, a clue (information on how to solve a quest) can also be considered feedback. This game element can be used in combination with other game elements, for instance with leaderboards and badges. We tracked six studies related to feedback [61,66,67,70,71,77].
- Agent, not to be confused with avatar or profile, is a virtual character by the system (not by the user). In the literature, we have found two studies that implemented agent .
- Task, which is generally connected with the previous one, but it has been considered by different authors .
- Virtual Currency is a type of reward in the form of virtual money; it has been investigated by .
- Personalising features; refers more to features typical of games in which the player can personalise the look and the outfit of the avatar/character; these were discussed in one study .
- Mission, which is a type of challenge, generally connected with task and time limit, and it has been investigated by .
- Replayability is the possibility given to users to re-do an action if, at the first attempt, he/she did not succeed. It was investigated by one study .
- Goal Indicators can be combined with several game elements such as levels and missions, and it has been examined by .
- Competition, usually within teams and or player vs. player; this has been presented in one study .
- Win State very typical of the game world, but it has been considered only by one recent study .
4.2. Q2: What Are the Effects of These Game Elements on Learners’ Behaviour in Online Learning Environments?
- PerformanceGrant and Betts  examined the influence of badges on (1,295,620) users’ performance (activity in web forums); the positive impact of badges on users’ performance were investigated by Hakulinen et al. ’s study, who investigated badges, within an online learning environment (involving 281 students) and their impact on the carefulness of the performance (looking at significant differences in terms of time spent per submission and number of attempts per exercise), highlighting the fact that badges can affect students’ behaviour in terms of time needed to perform and precision in the performance increasing students’ carefulness. “Students in the treatment group spent more time per submission on average” (p. 23), showing a higher level of carefulness compared to their colleagues in the control group . In a similar scenario, an online learning course (involving 150 students), Kyewski and Krämer  investigated the effects of badges on learners’ performance, and, in contrast to the studies previously detailed, recorded a negative effect of badges on learners’ performance.Leaderboards were the focus of studies by Landers and Landers  and Bernik et al. , both implemented in an online learning scenario. The first (involving a total of 86 participants) aimed to study the effect of the game element leaderboard on learners’ academic performance, looking at the amount of time spent on a task (consisting of a wiki project). As a result, users in the treatment group (with leaderboard) increased their time-on-task performance . The second, Bernik et al.  (involving a total of 55 students), looked at performance comparing the achievements of the control and treatment groups. Students in the treatment group had much better achievements, when comparing the average results in the post-tests performed by the two groups .The single game element narrative was studied by Armstrong and Landers , who tested how users would perform and react to gamified training, finding that participants in the gamified condition reacted better to the training than their colleagues in the control (non-gamified) condition .Several studies have investigated the effects of gamification on users’ performance by combining different game elements. Four game elements: challenges, leaderboards, trophies and medals were implemented by Domínguez et al.  within an e-learning platform. To study the effects of their plugin on students’ learning performance, they compared the scores achieved in several activities by participants in both control (non-gamified) and treatment (gamified) groups. The data analysis showed that students in the treatment group performed better than those in the control group in the initial activity, and in the majority of the assignments. Negative effects of the gamification approach were instead reported for the final examination, where students in the treatment group registered a significantly lower score . Tsay et al.  also opted for four game elements: immediate feedback; badges; leaderboard and time limits. By comparing the scores of the two groups (control-non-gamified and treatment-gamified), the authors highlighted that students in the gamified condition performed better (higher score) than their colleagues in the control group. Finding that their performance was mediated by their level of engagement, the students in the gamified condition were more engaged and therefore had better performance compared to their colleagues in the control condition .The study of De-Marcos et al.  aimed to determine which of these five different conditions: plain, educational game, gamification, social networking and social, was the most effective on learning performance (75 undergraduate students were involved). Participants, distributed into the different conditions, were asked to perform five different tests (word processing, spreadsheets, presentation, databases and final examination). In the comparative analysis, it was demonstrated that all the experimental conditions outperformed the control group in three of these tests (word processing, spreadsheets and presentation). The authors concluded that gamification impacts on learning performance when it is combined with social approaches . Similar conclusions were drawn by Krause et al. , who showed that social gamification increases learning success (see details of this study in the engagement section).Long and Aleven  studied the effect of a gamified Intelligent Tutoring System (ITS) on students’ learning and enjoyment (involving 267 students). The game elements selected were: rewards (Rwd) for good performance; and replay (RePl), the chance to complete a task again (in this case a problem already solved). The four different conditions were analysed combining the two game elements (RePl + Rwd; No-RePl + Rwd; RePl + No-Rwd; No-RePl + No-Rwd) and the most effective combination for students’ learning was replay without rewards (RePl + No-Rwd) . Students under the condition with both replay and rewards activated, performed significantly worse than their colleagues exposed to the other three conditions of the study. No significant effects were found when comparing the two groups in terms of enjoyment .Two studies recorded the effects of gamification on performance in terms of goal commitments. The first by Landers et al.  reported on a quasi-experiment (involving 339 university students), to better understand how leaderboards can enhance user performance and influence goal commitment, or vice versa . Participants were randomly assigned to one of five conditions (do-your-best, easy goal, difficult goal, impossible goal and leaderboard). With their study, Landers et al.  demonstrated that the leaderboard condition was the most influential and that the goal-setting theory is suitable to understand and explain leaderboard effects on users’ performance: “commitment moderates the success of leaderboards as goal-setting theory would predict” (, p. 5). In accordance with the goal-setting theory , goal commitment and performance are directly related to each other, “goals are only effective if people are committed to them, and performance is maximised when individuals are committed to difficult, specific goals” (, p. 5). The results showed that users in the leaderboard condition performed better than those in the other conditions. Participants in the leaderboard condition were “likely to target the top or near-top goals presented on that leaderboard, even without specific instructions to target those goals” (, p. 6). The second study, conducted by Hakulinen and Auvinen , focused more on goal orientation. The authors studied the relationship between the achievement of badges and goal orientations, they hypothesised “that students with different goal orientation profiles respond differently to badges” (, p. 9). Their study was conducted in an online environment (involving 278 students) divided into a control (non-gamified) and a treatment (gamified) group. They “found no statistically significant differences in the behaviour of the different goal orientation groups regarding badges. However, their attitudes towards the badges varied” (, p. 16).
- MotivationThe effect of rewards in the form of badges on learner motivation was studied by Kyewski and Krämer . They first measured the participants’ level of the intrinsic motivation before the intervention. Subsequently, they checked: (1) if the intrinsic motivation level changed in relation to participants earning the badge; (2) if the level of intrinsic motivation changed compared to the beginning (higher intrinsic motivation for those who had a low level at the beginning and vice-versa, lower for those who declared a high level of intrinsic motivation). No effects of badges were found on students’ intrinsic motivation. These results contradicted a study conducted in similar conditions (participants from the university level, working in an online learning environment) by Hakulinen et al. , reporting that the majority of participants declared being motivated by badges and concluding that “achievement badges seem to be a promising method to motivate students” (, p. 18). The positive effect of badges on learners’ motivation was confirmed by Gooch et al. , who conducted their study in Moodle but with a small group of children with a learning disability (dyslexia). Based on their findings, badges significantly improved students’ motivation .Dyslexic students (40 were involved) were also the target audience of Saputra ’s study, who adopted the following game elements: narrative, goal indicators, levels, scores, feedback and badges, highlighting that these game elements produce positive effects on students’ engagement, enjoyment and motivation in the short term. A longitudinal study instead was conducted by Hanus and Fox  to assess the effects of badges and leaderboards on students’ motivation, as well as on social comparison, effort, satisfaction, learner empowerment, and academic performance. Two classes (gamified and not-gamified) were set up and the results showed that, over time, students in the gamified course presented less motivation, satisfaction and empowerment compared to those involved in the non-gamified class .The study of Utomo and Santoso  was implemented in an online learning environment to test the effects of a ‘pedagogical agent’ on students’ motivation and behaviour towards learning. The agent activity was enhanced with the adaptation of some game elements, such as progress bar, badges, scores and feedback. Based on students’ evaluation, the authors concluded that personalised feedback in real time boosted learners’ motivation “toward active learning behaviour” (, p. 7). Badges, as well as points and leaderboards, were used also by Huang and Hew , who gamified an SPSS (Statistical Package for Social Sciences) course in ‘Moodle’, showing that badges and leaderboards motivated most of the learners. Points engaged students and stimulated them to undertake challenging tasks and extracurricular learning, while “learners in the control group did not attempt any challenge” . The same game elements were implemented in LOPUPA (Learning On Projects of United Promotion for Academia), a gamified platform proposed by Kuo and Chuang  implemented with the purpose of engaging and motivating online academic dissemination. Based on the data collected via the platform (as well as google analytics), and the feedback given by participants by filling in a questionnaire, the authors showed increased engagement and motivation of the participants that used the platform . Similar conclusions were reached by Domínguez et al.  who, in their study, also investigated the effect of their gamification design, described above, on students’ motivation, stating that “gamification in e-learning platforms seems to have the potential to increase student motivation, but it is not trivial to achieve that effect, and a big effort is required in the design and implementation of the experience for it to be fully motivating for participants” (p. 391).
- EngagementKyewski and Krämer  found that earning badges in an online learning environment did not increase students’ level of activity, which was used as a measure of engagement. Furthermore, data “revealed that students in the no-badge condition who did not earn badges were even more active than students in the gamification conditions” (, p. 32). This contradicts previous research findings, such as those reported by Sitra et al. , suggesting positive effects of badges on students’ engagement, even though they tested it within a different population (small number of students with special needs), using an LMS system-Moodle similar to Kyewski and Krämer . Badges as well as narrative, score/ranking, levels, and quests are covered in the study of . They aimed at investigating whether these game elements affect learners’ motivation and engagement on a peer assessment platform. By comparing the results of control and treatment groups, they found that the number of elaborated essays as well as the number of corrected essays were higher for participants in the treatment group. Therefore, the data indicated that gamification stimulated students to use the platform more, thus enhancing engagement .The following three studies underlined how the ‘social factor’ is important for engaging students. Firstly, Krause et al.  set up three conditions in ‘Moodle’ (plain–no game elements, game–with game elements, social-with social game elements). In particular, the game condition consisted of implementing, in an online environment (Moodle), game elements such as: avatars, badges, points, leaderboards and time limits. The social-game condition included all the game elements listed above and in addition integrated social game elements encouraging competition among students, also in remote mode, via pre-recorded actions, mainly by using the leaderboard and mechanism of social comparison. The purpose was to understand the differences (if any) between gamification and social gamification on students’ retention and learning success. The data showed that learners in the social game condition were more engaged compared with their colleagues in the plain condition, and the social game elements enhanced gamification effects on retention and success . Secondly, De-Marcos et al.  set up a quasi-experimental design with five conditions (control group, non-gamified, gamified, social non-use, social use) with the purpose of understanding the effects of ‘social networking’ and ‘gamification’ on students’ academic achievement, engagement and attitude in an undergraduate course . The game elements implemented were: badges, challenges, leaderboards, levels, trophies and forum. The data revealed that both gamification and social networking were perceived positively by students, therefore an improvement of participants’ attitude towards these approaches was recorded, while no significant statistical differences were found in students’ academic achievements or engagement . Thirdly, Mazarakis  conducted a study on using feedback mechanisms, testing four types of feedback to increase user engagement, i.e., participation in the course wiki: (1) gratitude feedback: this expressed thankfulness without giving any further information to the users, (2) historical: this gave information about users’ activities and contributions, (3) ‘relative ranking’: showing relative user rank, and (4) ‘social ranking feedback’: this aimed at providing information about the points and activated competition via ranking comparison. The results showed that providing feedback mechanisms can enhance participation. Among the feedback mechanisms tested, the one enabling social comparison (called ‘social ranking feedback’) was the most effective feedback . More focused on the attitude towards gamification is the study of Aldemir et al. , included in this dimension because it deals also with the aim of increasing students’ perception of others within an online educational scenario. The study investigated in particular the participants’ attitude towards leaderboards, challenges, narratives, teams, badges (and rewards in general) win-states, points, and ‘constraints’, providing generally positive results and “insights about game elements integrated into a gamified course in both online and face-to-face sessions” (, p. 251).
- Attitude towards gamificationThe following authors: Bernik et al. , Hakulinen et al. , Aldemir et al. , whose work is already presented above, have also all studied the effects of their gamification designs on the attitude towards gamification, reporting all positive responses of the participants in this matter. In the same line is the study of , who in an online learning scenario set up a quasi-experiment with the purpose of studying the effect of social networking and gamification on student academic achievement, participation and attitude towards gamification. The participants were split into three conditions: gamification (114); social network (185) and control (75). In the gamified condition, the following game elements were displayed: badges, challenges, leaderboards, forum, levels and trophies. A survey was completed by both the experimental groups to assess learners’ attitudes towards gamification and social networks. The results showed a positive effect on users’ attitude towards gamification; no significant effects were found on performance improvement and neither on written examinations. Regarding academic achievement, it was better for the social network participants .
- CollaborationKnutas et al.  studied the effects of a gamified online discussion system on users’ collaborative behaviour and communication (involving 249 students). The system was used during an existing university course ‘introduction to programming’ for fourteen weeks. The major aim of the gamified online discussion system was to facilitate contributions for effective discussions. Therefore, to enhance peers’ contribution and communication, users were enabled to ‘like’ or ‘unlike’ the comments of their peers. In addition, a reward feedback system was implemented for the users who were contributing. To determine the effects of the described gamified online discussion system on users’ collaborative behaviour and communication, several tools were used, such as surveys, interviews and Social Network Analysis (SNA). From the data analysis, it was possible to conclude that the gamified online discussion system increased student collaboration and course communication efficiency by reducing response time to students’ questions. In particular, the survey showed that skilled students liked the gamification features (consisting of the discussion system and rewards) and Knutas et al.  concluded that the game elements stimulated users to contribute more by giving more answers and proposing more questions. Part of this dimension is also in the Aldemir et al.  study, described above, who point out that, in using the team game element, the balance of "the team skills" is important, enabling the right level of competition and collaboration within the team.
- Social awarenessTwo studies were dedicated to the “social awareness” that gamification can generate: Aldemir et al. , Christy and Fox . In particular,  set up a study on leaderboards, aimed at investigating whether interaction with a leaderboard (with male vs. female leaders) produced effects in the social comparison condition or led to stereotype threats and effects on academic performance. Their data showed that “leaderboards appear to have inspired social comparison processes” (p. 74) more than ‘stereotype threats’ . The second study by  adopted “teams”, another game element that can enable participants, in an online learning course, to be aware of the others and thus facilitate community building. Using the authors’ words: “Community-building process is affected by the interaction and relationship between teammates, implying that good communication facilitates community building. Another implication is that, the fewer people in a team, the easier it is to communicate; therefore, the teams should be small community building” (, p. 250). Based on the qualitative data collected by Aldemir et al. , participants reacted positively to this game element, with positive effects on learning achievement.
- Effects of badges/rewards are observed on motivation, attitude toward gamification use, and performance in terms of time management, engagement, emotional states, and enjoyment. Effects of badges may vary according to gender and personality, and, if perceived as controlling and restrictive, they may negatively affect motivation and engagement. They can be used to set clear goals or to stimulate social comparison, both variables that have a positive effect on performance (in the training field, specifically).
- Effects of leaderboards have been found on attitude toward gamification use, learning performance, performance in general, engagement, enjoyment, and goal commitment, by engaging students in difficult tasks. Effects generated by leaderboards vary according to personality. Just like badges, leaderboards can enable social comparison that can positively influence performance. More specifically, the game element leaderboards provides information about points, scored by users, which activates social comparison and competition among them and achieve, as effects, higher participation and engagement, in particular at the cognitive level .
- Effects of point/score/ranking have been reported related to motivation, attitude toward gamification use, learning performance, performance in general, engagement, enjoyment and emotional states. Effects of point/score/ranking may vary according to personality and gender. They can foster social comparison and thus also influence performance. Furthermore, the game element points/scores/ranking enhanced the level of users’ engagement so much that they were stimulated to undertake challenging tasks and target “top or near-top goals” in terms of difficulties .
6. Future Work
7. Limitations of This Study
Conflicts of Interest
|MOOCs||Massive Open Online Courses|
|MMORPGs||Massive Multiplayer Online Role Playing Games|
|MMOGs||Massive Multiplayer Online Games|
|Bernik et al. ||Pre-post survey||55 (28 + 27)||e-learning|
|De-Marcos et al. ||Experimental||75||e-learning|
|De-Marcos et al. ||Quasi- exp||318 (62 + 111 + 146)||e-learning|
|Domínguez et al. ||Quasi- exp||173 (62 + 111)||e-learning|
|Borras-Gene et al. ||Mixed Methods||3866||MiriadaX|
|Gooch et al. ||Pre-post tests||22 (10 + 10 + 2)||ClassDojo|
|Armstrong and Landers ||Experimental (two courses compared)||273||e-learning|
|Hakulinen et al. ||Experimental||281||online learning|
|Christy and Fox ||Experimental||76||online|
|Aldemir et al. ||Empirical (observation, interviews, documents)||118||e-learning|
|Huang and Hew ||Quasi- exp||40 (21 + 19)||online learning|
|Knutas et al. ||Empirical (SNA, survey, interviews)||249||online learning|
|Krause et al. ||Experimental||206 (71 + 67 + 68)||Moodle|
|Kuo and Chuang ||Empirical (surveys, web analytics)||73||LOPUPA|
|Kyewski and Krämer ||Experimental||126||Moodle|
|Long and Aleven ||Experimental||190||online learning|
|Utomo and Santoso ||Empirical (questionnaire and focus groups)||31||Moodle|
|Tsay et al. ||Empirical (two versions of the course compared)||136||Moodle|
|Pedro et al. ||Empirical (questionnaire)||16||E-Game|
|Saputra ||Empirical (observation and questionnaire)||40||e-learning|
|Sitra et al. ||Case Study||5||Moodle|
|Tenorio et al. ||Experimental||32||MeuTutor|
|Mazarakis ||Experimental||436||online learning|
|Landers et al. ||Experimental||240||online learning|
|Landers and Landers ||Experimental||109||online learning|
|Codish and Ravid ||Empirical (questionnaire)||102 (58 + 44)||online learning|
|Grant and Betts ||Experimental||1,295,620||Stack Overflow|
|Game Elements||No. of Studies||References|
|Likes (social features)||4||[47,53,58,61]|
|Performance||Performance and time management||[51,53,55,57,58,62,64,65,70,75,78,81]|
|Performance and goal commitment||[74,75,81]|
|Attitude towards gamification||[47,51,54,58,72]|
|Social awareness||Community building|||
|Areas Impacted—Game Elements||Engagement||Engagement (Retention)||Enjoyment||Performance (Goal Commitment)||Attitude towards Gamification||Motivation||Collaboration (Communication)||Community Building||Social Comparison|
|Likes (social features)||[53,61]||[47,58]|
- Kalz, M.; Specht, M. If MOOCS Are the Answer, Did We Ask the Right Questions? Implications for the Design of Large-Scale Online-Courses; Maastricht School of Management: Maastricht, The Netherlands, 2013. [Google Scholar]
- Kizilcec, R.F.; Piech, C.; Schneider, E. Deconstructing disengagement: analyzing learner subpopulations in massive open online courses. In Proceedings of the Third International Conference on Learning Analytics and Knowledge, Leuven, Belgium, 8–13 April 2013. [Google Scholar]
- Pappano, L. The Year of the MOOC. The New York Times, 14 November 2012. [Google Scholar]
- Deterding, S.; Dixon, D.; Khaled, R.; Nacke, L. From game design elements to gamefulness: Defining “gamification”. In Proceedings of the 15th International Academic MindTrek Conference on Envisioning Future Media Environments (MindTrek ’11), Tampere, Finland, 28–30 September 2011; ACM: New York, NY, USA, 2011; pp. 9–15. [Google Scholar] [CrossRef]
- Reich, J.; Ruipérez-Valiente, J.A. The MOOC pivot. Science 2019, 363, 130–131. [Google Scholar] [CrossRef] [PubMed]
- Antonaci, A.; Klemke, R.; Stracke, C.M.; Specht, M. Gamification in MOOCs to enhance users’ goal achievement. In Proceedings of the 2017 IEEE Global Engineering Education Conference (EDUCON), Athens, Greece, 25–28 April 2017; pp. 1654–1662. [Google Scholar] [CrossRef]
- Henderikx, M.A.; Kreijns, K.; Kalz, M. Refining success and dropout in massive open online courses based on the intention–behavior gap. Distance Educ. 2017, 38, 353–368. [Google Scholar] [CrossRef]
- Reich, J. MOOC completion and retention in the context of student intent. EDUCAUSE Review Online, 8 December 2014. [Google Scholar]
- Kihl, M.; Aurelius, A.; Lagerstedt, C. Analysis of World of Warcraft traffic patterns and user behavior. In Proceedings of the International Congress on Ultra Modern Telecommunications and Control Systems, Moscow, Russia, 18–20 October 2010; pp. 218–223. [Google Scholar] [CrossRef]
- Drachsler, H.; Kalz, M. The MOOC and learning analytics innovation cycle (MOLAC): A reflective summary of ongoing research and its challenges. J. Comput. Assist. Learn. 2016, 32, 281–290. [Google Scholar] [CrossRef]
- Croft, N.; Dalton, A.; Grant, M. Overcoming isolation in distance learning: Building a learning community through time and space. J. Educ. Built Environ. 2010, 5, 27–64. [Google Scholar] [CrossRef]
- Kasurinen, J.; Knutas, A. Publication trends in gamification: A systematic mapping study. Comput. Sci. Rev. 2018, 27, 33–44. [Google Scholar] [CrossRef]
- Rodrigues, L.F.; Oliveira, A.; Costa, C.J. Playing seriously—How gamification and social cues influence bank customers to use gamified e-business applications. Comput. Hum. Behav. 2016, 63, 392–407. [Google Scholar] [CrossRef]
- Hamari, J. Do badges increase user activity? A field experiment on the effects of gamification. Comput. Hum. Behav. 2015, 1–10. [Google Scholar] [CrossRef]
- Marques, R.; Gregório, J.; Pinheiro, F.; Póvoa, P.; Da Silva, M.M.; Lapão, L.V. How can information systems provide support to nurses’ hand hygiene performance? Using gamification and indoor location to improve hand hygiene awareness and reduce hospital infections. BMC Med. Inform. Decis. Mak. 2017, 17, 1–16. [Google Scholar] [CrossRef]
- Ryan, J.; Edney, S.; Maher, C. Engagement, compliance and retention with a gamified online social networking physical activity intervention. Transl. Behav. Med. 2017, 7, 702–708. [Google Scholar] [CrossRef]
- Sardi, L.; Idri, A.; Fernández-Alemán, J.L. A systematic review of gamification in e-Health. J. Biomed. Inform. 2017, 71, 31–48. [Google Scholar] [CrossRef]
- Kim, K.; Ahn, S.J.G. The Role of Gamification in Enhancing Intrinsic Motivation to Use a Loyalty Program. J. Interact. Mark. 2017, 40, 41–51. [Google Scholar] [CrossRef]
- Yang, Y.; Asaad, Y.; Dwivedi, Y. Examining the impact of gamification on intention of engagement and brand attitude in the marketing context. Comput. Hum. Behav. 2017, 73, 459–469. [Google Scholar] [CrossRef]
- Mekler, E.D.; Brühlmann, F.; Opwis, K.; Tuch, A.N. Do points, levels and leaderboards harm intrinsic motivation? In Proceedings of the First, International Conference on Gameful Design, Research, and Applications (Gamification ’13), Toronto, ON, Canada, 2–4 October 2013; pp. 63–73. [Google Scholar] [CrossRef]
- Mekler, E.D.; Brühlmann, F.; Tuch, A.N.; Opwis, K. Towards understanding the effects of individual gamification elements on intrinsic motivation and performance. Comput. Hum. Behav. 2015, 1–10. [Google Scholar] [CrossRef]
- Dicheva, D.; Dichev, C.; Agre, G.; Angelova, G. Gamification in Education: A Systematic Mapping Study. Educ. Technol. Soc. 2015, 18, 75–88. [Google Scholar] [CrossRef]
- De Sousa Borges, S.; Durelli, V.H.S.; Reis, H.M.; Isotani, S. A Systematic Mapping on Gamification Applied to Education. In Proceedings of the 29th Annual ACM Symposium on Applied Computing (SAC ’14), Gyeongju, Korea, 24–28 March 2014; ACM: New York, NY, USA, 2014; pp. 216–222. [Google Scholar] [CrossRef]
- Caponetto, I.; Earp, J.; Ott, M. Gamification and Education: A Literature Review. In Proceedings of the European Conference on Games Based Learning, Berlin, Germany, 9–10 October 2014; Volume 1, pp. 50–57. [Google Scholar]
- Nah, F.F.H.; Zeng, Q.; Telaprolu, V.R.; Ayyappa, A.P.; Eschenbrenner, B. Gamification of Education: A Review of Literature. In International Conference on HCI in Business; Nah, F.F., Ed.; Springer: Cham, Switzerland, 2014; pp. 401–409. [Google Scholar] [CrossRef]
- Mora, A.; Riera, D.; Gonzalez, C.; Arnedo-Moreno, J. A Literature Review of Gamification Design Frameworks. In Proceedings of the 7th International Conference on Games and Virtual Worlds for Serious Applications (VS-Games), Skövde, Sweden, 16–18 September 2015; pp. 1–8. [Google Scholar] [CrossRef]
- Çakıroğlu, Ü.; Başıbüyük, B.; Güler, M.; Atabay, M.; Yılmaz Memiş, B. Gamifying an ICT course: Influences on engagement and academic performance. Comput. Hum. Behav. 2017, 69, 98–107. [Google Scholar] [CrossRef]
- De Almeida Souza, M.R.; Constantino, K.F.; Veado, L.F.; Figueiredo, E.M.L. Gamification in Software Engineering Education: An Empirical Study. In Proceedings of the IEEE 30th Conference on Software Engineering Education and Training (CSEE&T), Savannah, Georgia, 7–9 November 2017; pp. 276–284. [Google Scholar] [CrossRef]
- Monterrat, B.; Lavoué, É.; George, S. Adaptation of Gaming Features for Motivating Learners. Simul. Gaming 2017, 48, 625–656. [Google Scholar] [CrossRef]
- Smith, T. Gamified Modules for an Introductory Statistics Course and Their Impact on Attitudes and Learning. Simul. Gaming 2017, 48, 832–854. [Google Scholar] [CrossRef]
- Hamari, J.; Koivisto, J.; Sarsa, H. Does gamification work?—A literature review of empirical studies on gamification. In Proceedings of the Annual Hawaii International Conference on System Sciences, Waikoloa, HI, USA, 6–9 January 2014; pp. 3025–3034. [Google Scholar] [CrossRef]
- Lister, M.C. Gamification: The effect on student motivation and performance at the post-secondary level. Issues Trends Educ. Technol. 2015, 3, 1–22. [Google Scholar] [CrossRef]
- Looyestyn, J.; Kernot, J.; Boshoff, K.; Ryan, J.; Edney, S.; Maher, C. Does gamification increase engagement with online programs? A systematic review. PLoS ONE 2017, 12, e0173403. [Google Scholar] [CrossRef]
- Lumsden, J.; Edwards, E.A.; Lawrence, N.S.; Coyle, D.; Munafò, M.R. Gamification of Cognitive Assessment and Cognitive Training: A Systematic Review of Applications and Efficacy. JMIR Serious Games 2016, 4, 1–14. [Google Scholar] [CrossRef]
- Khalil, M.; Wong, J.; Ebner, M.; de Koning, B.; Ebner, M.; Paas, F. Gamification in MOOCs: A Review of the State of the Art. In Proceedings of the IEEE Global Engineering Education Conference, Santa Cruz de Tenerife, Spain, 18–20 April 2018; pp. 1635–1644. [Google Scholar] [CrossRef]
- Ortega-Arranz, A.; Muñoz-Cristóbal, J.A.; Martínez-Monés, A.; Bote-Lorenzo, M.L.; Asensio-Pérez, J.I. How gamification is being implemented in MOOCs? A systematic literature review. In European Conference on Technology Enhanced Learning; Lavoué, É., Drachsler, H., Verbert, K., Broisin, J., Pérez-Sanagustín, M., Eds.; Springer: Tallinn, Estonia, 2017; Volume 10474, pp. 441–447. [Google Scholar] [CrossRef]
- Bakar, N.F.A.; Yusof, A.F.; Iahad, N.A.; Ahmad, N. The Implementation of Gamification in Massive Open Online Courses (MOOC) Platform. In International Conference on User Science and Engineering; Springer: Singapore, 2018; pp. 183–193. [Google Scholar]
- Gené, O.B.; Mart, M.; Blanco, Á.F. Gamification in MOOC: Challenges, Opportunities and Proposals for Advancing MOOC Model. In Proceedings of the 2nd International Conference on Technological Ecosystems for Enhancing Multiculturality, Salamanca, Spain, 1–3 October 2014; ACM: Salamanca, Spain, 2014; pp. 215–220. [Google Scholar] [CrossRef]
- Saraguro-Bravo, R.A.; Jara-Roa, D.I.; Agila-Palacios, M. Techno-Instructional Application in an MOOC Designed with Gamification Techniques. In Proceedings of the Third International Conference on eDemocracy & eGovernment (ICEDEG), Quito, Ecuador, 30 March–1 April 2016; pp. 176–179. [Google Scholar] [CrossRef]
- Mesquita, M.A.A.; Toda, A.M.; Brancher, J.D. BrasilEduca—An Open-Source MOOC platform for Portuguese speakers with gamification concepts. IEEE Front. Educ. Conf. 2014, 446–449. [Google Scholar] [CrossRef]
- Vaibhav, A.; Gupta, P. Gamification of MOOCs for increasing user engagement. In Proceedings of the IEEE International Conference on MOOCs, Innovation and Technology in Education (IEEE MITE 2014), Patiala, India, 19–20 December 2014; pp. 290–295. [Google Scholar] [CrossRef]
- Antonaci, A.; Klemke, R.; Kreijns, K.; Specht, M. Get Gamification of MOOC right! How to Embed the Individual and Social Aspects of MOOCs in Gamification Design. Int. J. Serious Games 2018, 5. [Google Scholar] [CrossRef]
- Antonaci, A.; Peter, D.; Klemke, R.; Bruysten, T.; Christian, M.; Specht, M. gMOOCs—Flow and Persuasion to Gamify MOOCs. In International Conference on Games and Learning Alliance; Lecture Notes in Computer Science (LNCS) Series; Springer: Cham, Switzerland, 2017; Volume 10653, pp. 126–136. [Google Scholar] [CrossRef]
- Antonaci, A.; Klemke, R.; Stracke, C.M.; Specht, M. Identifying game elements suitable for MOOCs. In European Conference on Technology Enhanced Learning; Springer: Cham, Switzerland, 2017; pp. 355–360. [Google Scholar]
- Antonaci, A.; Klemke, R.; Stracke, C.M.; Specht, M. Towards Implementing Gamification in MOOCs. In International Conference on Games and Learning Alliance; Lecture Notes in Computer Science (LNCS) Series; Springer: Cham, Switzerland, 2017; Volume 10653, pp. 115–125. [Google Scholar] [CrossRef]
- Chang, J.W.; Wei, H.Y. Exploring engaging gamification mechanics in massive online open courses. Educ. Technol. Soc. 2016, 19, 177–203. [Google Scholar]
- Borras-Gene, O.; Martinez-Nuñez, M.; Fidalgo-Blanco, Á. New Challenges for the Motivation and Learning in Engineering Education Using Gamification in MOOC. Int. J. Eng. Educ. 2016, 32, 501–512. [Google Scholar]
- Binti Mohd Nor Hisham, F.; Sulaiman, S. Adapting Gamification Approach in Massive Open Online Courses to Improve User Engagement. In UTM Computing Proceedings Innovation in Computing Technology and Applications; UTM: Skudai, Malaysia, 2017; Volume 2, pp. 1–6. [Google Scholar]
- Navío-Marco, J.; Solórzano-García, M. Student’s social e-reputation (“karma”) as motivational factor in MOOC learning. Interact. Learn. Environ. 2019, 1–15. [Google Scholar] [CrossRef]
- Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G.; PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med. 2009, 6, e1000097. [Google Scholar] [CrossRef]
- Bernik, A.; Bubaš, G.; Radoševi, D. A Pilot Study of the Influence of Gamification on the Effectiveness of an e-Learning Course. In Central European Conference on Information and Intelligent Systems; Faculty of Organization and Informatics Varazdin: Varazdin, Croatia, 2015; pp. 73–79. [Google Scholar]
- Codish, D.; Ravid, G. Personality Based Gamification—Educational Gamification for Extroverts and Introverts. In CHAIS ’14—Conference for the Study of Innovation and Learning Technologies: Learning in the Technological Era; Eshet-Alkalai, Y., Caspi, A., Geri, N., Kalman, Y., Silber-Varod, V., Yair, Y., Eds.; The Open University of Israel: Raanana, Israel, 2014; pp. 36–44. [Google Scholar]
- De-Marcos, L.; Garciá-López, E.; Garciá-Cabot, A.; Medina-Merodio, J.A.; Domínguez, A.; Martínez-Herraíz, J.J.; Diez-Folledo, T. Social network analysis of a gamified e-learning course: Small-world phenomenon and network metrics as predictors of academic performance. Comput. Hum. Behav. 2016, 60, 312–321. [Google Scholar] [CrossRef]
- De-Marcos, L.; Domínguez, A.; Saenz-De-Navarrete, J.; Pagés, C. An empirical study comparing gamification and social networking on e-learning. Comput. Educ. 2014, 75, 82–91. [Google Scholar] [CrossRef]
- Domínguez, A.; Saenz-De-Navarrete, J.; De-Marcos, L.; Fernández-Sanz, L.; Pagés, C.; Martínez-Herráiz, J.J. Gamifying learning experiences: Practical implications and outcomes. Comput. Educ. 2013, 63, 380–392. [Google Scholar] [CrossRef]
- Gooch, D.; Vasalou, A.; Benton, L.; Khaled, R. Using Gamification to Motivate Students with Dyslexia. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’16), San Jose, CA, USA, 7–12 May 2016; ACM: New York, NY, USA, 2016; pp. 969–980. [Google Scholar] [CrossRef]
- Grant, S.; Betts, B. Encouraging user behaviour with achievements: An empirical study. In Proceedings of the IEEE International Working Conference on Mining Software Repositories, San Francisco, CA, USA, 18–19 May 2013; pp. 65–68. [Google Scholar] [CrossRef]
- Hakulinen, L.; Auvinen, T.; Korhonen, A. The effect of achievement badges on students’ behavior: An empirical study in a university- level computer science course. Int. J. Emerg. Technol. Learn. 2015, 10, 18–30. [Google Scholar] [CrossRef]
- Huang, B.; Hew, K.F. Do points, badges and leaderboard increase learning and activity: A quasi-experiment on the effects of gamification. In Proceedings of the 23rd International Conference on Computers in Education, Hangzhou, China, 30 November–4 December 2015; Society for Computer in Education: Hangzhou, China, 2015; pp. 275–280. [Google Scholar]
- Hanus, M.D.; Fox, J. Assessing the effects of gamification in the classroom: A longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance. Comput. Educ. 2015, 80, 152–161. [Google Scholar] [CrossRef]
- Knutas, A.; Ikonen, J.; Nikula, U.; Porras, J. Increasing collaborative communications in a programming course with gamification. In Proceedings of the 15th International Conference on Computer Systems and Technologies (CompSysTech ’14), Ruse, Bulgaria, 27–28 June 2014; Volume 883, pp. 370–377. [Google Scholar] [CrossRef]
- Krause, M.; Mogalle, M.; Pohl, H.; Williams, J.J. A Playful Game Changer: Fostering Student Retention in Online Education with Social Gamification. In Proceedings of the 2nd ACM Conference on [email protected] ([email protected]’15), Vancouver, BC, Canada, 14–18 March 2015; pp. 95–102. [Google Scholar] [CrossRef]
- Kuo, M.S.; Chuang, T.Y. How gamification motivates visits and engagement for online academic dissemination—An empirical study. Comput. Hum. Behav. 2016, 55, 16–27. [Google Scholar] [CrossRef]
- Kyewski, E.; Krämer, N.C. To gamify or not to gamify? An experimental field study of the influence of badges on motivation, activity, and performance in an online learning course. Comput. Educ. 2018, 118, 25–37. [Google Scholar] [CrossRef]
- Long, Y.; Aleven, V. Gamification of Joint Student / System Control over Problem Selection in a Linear Equation Tutor. In International Conference on Intelligent Tutoring Systems; Trausan-Matu, S., Boyer, K.E., Crosby, M., Panourgia, K., Eds.; Lecture Notes in Computer Science (LNCS) Series; Springer: New York, NY, USA, 2014; pp. 378–387. [Google Scholar] [CrossRef]
- Pedro, L.Z.; Lopes, A.M.Z.; Prates, B.G.; Vassileva, J.; Isotani, S. Does gamification work for boys and girls? In Proceedings of the 30th Annual ACM Symposium on Applied Computing—SAC ’15, Salamanca, Spain, 1–17 April 2015; ACM: Salamanca, Spain, 2015; pp. 214–219. [Google Scholar] [CrossRef]
- Saputra, M.R.U. LexiPal: Design, Implementation and Evaluation of Gamification on Learning Application for Dyslexia. Int. J. Comput. Appl. 2015, 131, 37–43. [Google Scholar] [CrossRef]
- Sitra, O.; Katsigiannakis, V.; Karagiannidis, C.; Mavropoulou, S. The effect of badges on the engagement of students with special educational needs: A case study. Educ. Inf. Technol. 2017, 22, 3037–3046. [Google Scholar] [CrossRef]
- Tenorio, T.; Bittencourt, I.I.; Isotani, S.; Pedro, A.; Ospina, P. A gamified peer assessment model for online learning environments in a competitive context. Comput. Hum. Behav. 2016, 64, 247–263. [Google Scholar] [CrossRef]
- Tsay, C.H.H.; Kofinas, A.; Luo, J. Enhancing student learning experience with technology-mediated gamification: An empirical study. Comput. Educ. 2018, 121, 1–17. [Google Scholar] [CrossRef]
- Utomo, A.Y.; Santoso, H.B. Development of gamification-enriched pedagogical agent for e-Learning system based on community of inquiry. In Proceedings of the International HCI and UX Conference in Indonesia on (CHIuXiD ’15), Bandung, Indonesia, 8–10 April 2015; pp. 1–9. [Google Scholar] [CrossRef]
- Aldemir, T.; Celik, B.; Kaplan, G. A qualitative investigation of student perceptions of game elements in a gamified course. Comput. Hum. Behav. 2018, 78, 235–254. [Google Scholar] [CrossRef]
- Christy, K.R.; Fox, J. Leaderboards in a virtual classroom: A test of stereotype threat and social comparison explanations for women’s math performance. Comput. Educ. 2014, 78, 66–77. [Google Scholar] [CrossRef]
- Landers, R.N.; Bauer, K.N.; Callan, R.C. Gamification of task performance with leaderboards: A goal setting experiment. Comput. Hum. Behav. 2015, 1–8. [Google Scholar] [CrossRef]
- Landers, R.N.; Landers, A.K. An Empirical Test of the Theory of Gamified Learning: The Effect of Leaderboards on Time-on-Task and Academic Performance. Simul. Gaming 2014, 45, 769–785. [Google Scholar] [CrossRef]
- Björk, S.; Holopainen, J. Patterns in Game Design; Charles River Media, Inc.: Needham, MA, USA, 2005. [Google Scholar]
- Mazarakis, A. Using Gamification for Technology Enhanced Learning: The Case of Feedback Mechanisms. Bull. IEEE Tech. Comm. Learn. Technol. 2015, 17, 6–9. [Google Scholar]
- Armstrong, M.B.; Landers, R.N. An Evaluation of Gamified Training: Using Narrative to Improve Reactions and Learning. Simul. Gaming 2017, 48, 513–538. [Google Scholar] [CrossRef]
- De-Marcos, L.; Garcia-Lopez, E.; Garcia-Cabot, A. On the effectiveness of game-like and social approaches in learning: Comparing educational gaming, gamification & social networking. Comput. Educ. 2016, 95, 99–113. [Google Scholar] [CrossRef]
- Locke, E.A.; Latham, G.P. New Directions in Goal-Setting Theory. Curr. Dir. Psychol. Sci. 2006, 15, 265–268. [Google Scholar] [CrossRef]
- Hakulinen, L.; Auvinen, T. The effect of gamification on students with different achievement goal orientations. In Proceedings of the International Conference on Teaching and Learning in Computing and Engineering (LATICE 2014), Kuching, Malaysia, 11–13 April 2014; pp. 9–16. [Google Scholar] [CrossRef]
- Ryan, R.M.; Deci, E.L. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am. Psychol. 2000, 55, 68–78. [Google Scholar] [CrossRef]
- Nacke, L.E.; Deterding, S. The maturing of gamification research. Comput. Hum. Behav. 2017, 71, 450–454. [Google Scholar] [CrossRef]
- Tseng, F.C.; Huang, H.C.; Teng, C.I. How Do Online Game Communities Retain Gamers? Social Presence and Social Capital Perspectives. J. Comput.-Mediat. Commun. 2015, 20, 601–614. [Google Scholar] [CrossRef]
- Gollwitzer, P.M. Implementation intentions. Am. Psychol. 1999, 54, 493–503. [Google Scholar] [CrossRef]
- Gollwitzer, P.M.; Sheeran, P. Implementation intentions and goal achievement: A meta-analysis of effects and processes. Adv. Exp. Soc. Psychol. 2006, 38, 69–119. [Google Scholar] [CrossRef]
|Criteria||Inclusion Criteria||Exclusion Criteria|
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).