Topic Evolution in the Research on Educational Gamification

Abstract

The research on educational gamification spans many topics of interest. As the total volume of research in this area has greatly increased in the last 10 years, it is interesting to see how the interest in the respective topics has changed over the same period. In this paper, we answer this question by means of keyword analysis performed on 7572 unique keywords extracted from 2203 papers. The obtained results reveal (1) the high popularity of keywords that are non-obviously relevant to gamification, (2) vast disproportions in the volume of research dedicated to different aspects of the same research sub-area, and (3) differing patterns of popularity among the most frequent keywords, as well as (4) keywords introduced and abandoned in recent years. The presented findings bear a number of implications for the future of research on educational gamification.

1. Introduction

Educational gamification strives to increase learners’ motivation and engagement by incorporating game design elements into educational environments [1]. There is an ongoing debate about the extent and in which educational contexts it delivers what it is expected to [1]. The empirical data indicate that, while not always, more often than not, it does; the most extensive-to-date survey on educational-gamification outcomes reported positive results for 71.4% of the 91 analyzed studies [2].

While educational gamification builds upon the success of gamification in business environments, making use of many techniques first developed for business purposes (primarily, employee and customer engagement [3]), as the interests of teachers and students are much more aligned than those of employers and employees, or vendors and customers, the educational context finds itself free from some of its known drawbacks–in particular, there is no risk of exploitation often associated with gamification in the workplace [4,5,6].

Once a niche topic, the research on gamification in education amassed in recent years into a large body of knowledge, amounting to 105,000 items reported by Google Scholar for the search terms “education gamification” [7], thus, creating a large opportunity for secondary research. This opportunity did not go unnoticed, and a number of reviews on the research on educational gamification have been published in the last five years. These include systematic literature reviews [1,2,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32], meta-analyses [33,34], systematic mappings [35,36,37], and bibliometric surveys [38,39,40,41,42,43,44,45]. None of them, however, focused on how the popularity of the covered topics changed over time.

The gap described above is addressed in this paper, which pursues the aim of understanding the evolution of the topics chosen in the research on educational gamification. The presented study is based on keyword analysis, a capable tool for tracking the evolution of a research area. The potential of this approach stems from the fact that keywords are deliberately selected by authors to correctly express the subject matter of papers ([46] and works cited therein). Keyword analysis has been widely implemented for providing an overview of the development of various research areas, including such diverse fields as animal behavior [47], epidemiology [48], psychiatry [49], management information systems [50], technology forecasting [51], and interdisciplinary research [52]. It has also been successfully applied in the fields of education [53] and educational technology [54,55,56], game-based learning [57,58], and gamification in general [40,59].

The following research questions have been stated:

RQ1.

How did the yearly number of publications on educational gamification change over the analyzed period?

RQ2.

What were the keywords most frequently used throughout the analyzed period?

RQ3.

How did the set of the most frequently used keywords change over the subsequent years of the analyzed period?

RQ4.

Which pairs of keywords were most often used together?

RQ5.

How did the popularity of respective keywords change over the subsequent years of the analyzed period?

RQ6.

What have been the keywords that have become popular most recently and the keywords that lost their popularity in recent years?

The research questions listed above are answered in the six respective subsections of Section 3. Before that, in Section 2, the applied research method and used data sources are described. The final Section 4 concludes the main findings and presents their implications for future research within the area of educational gamification.

2. Materials and Methods

As the publications on educational gamification are not limited to a few venues or journals, rather than focusing on a preselected set of these, we decided to query bibliographic databases covering thousands of potentially relevant sources. Three providers of such databases have been chosen, two of which because of their overall extensive coverage of the scientific literature (Scopus [60] and Web of Science [61]) and one because of its thematic focus on education (ERIC [62]).

Considering the abundance of publications on educational gamification, in a strive to avoid false positives more than false negatives in the search results, we followed the example of [45] and used just two search keywords: “education” and “gamification”.

Unless specified below, the standard query options have been used.

As only incomplete data are available for the year 2022, and the earliest indexed publications were published in 2011, the search period has been limited to 10 publication years: 2012–2021.

In both Scopus and Web of Science, the allowed language has been set to English, and the search was performed on Title, Author Keywords, and Abstract fields. The Index Terms field containing keywords generated by the respective database provider was not included in the search, as doing so resulted in a high number of false positives.

From Scopus, the allowed document types have been limited to “Conference Paper” (1915 items), “Article” (1103 items), and “Book Chapter” (142 items), giving a total of 3160 papers found.

From Web of Science, the allowed document types have been limited to “Proceeding Paper” (1518 items), “Article” (701 items), and “Book Chapter” (21 items); a total of 2203 papers has been found (some papers were assigned to more than one document type simultaneously).

From ERIC, the “Peer reviewed only” option has been set. The query results comprised 227 publications.

The three publication data files were then combined with the duplicates removed. The resulting dataset contained 4324 items, including 3138 that were found in Scopus, 1105 that were found in Web of Science (excluding those also found in Scopus), and 81 that were found in ERIC (excluding those also found in Scopus or Web of Science).

The Author Keywords field for the publications found in Scopus or Web of Science and the Manual Tags field for the publications found in ERIC were then processed to obtain a list of keywords. The first processing step involved converting all keywords and titles into lowercase, in order to allow for case-insensitive matching.

Next, the identified keywords were looked for in each publication’s title, and matching publications were marked as relevant to a given keyword even if it was not specified in its own Author Keywords or Manual Tags field. Note that, in the case of keywords containing other keywords, only the longest matching keyword found in the title was considered (e.g., if a title contained “mobile learning”, only the “mobile learning” keyword has been assigned to the work, and not “learning”).

Next, the keywords used as the search terms (and their variant forms) were removed, as all analyzed works were required to use them to be included in the set, so there was no point in considering them for further analysis. The list of removed keywords comprised the following: “gamification”, “education”, “gamification in education”, “educational gamification”, and “gamifying”.

Then, the counts for those terms having different spelling variants or forms were aggregated. In total, 56 such alternative forms were identified, including, e.g., “game” and “games”; “Technology Acceptance Model”, “TAM”, and “Technology Acceptance Model (TAM)”; or “game-based learning”, “game based learning”, “games-based learning”, and “GBL”. Finally, the uppercase letters were restored to the proper nouns and acronyms identified in the keywords.

After the above-described pruning, 7572 unique keywords were identified, which occurred 27,272 times in total during the analyzed period. There were 21 keywords which occurred at least 100 times, 58 keywords which occurred at least 50 times, and 158 keywords which occurred at least 25 times during the analyzed period. Looking at the other end of the list, there were 4677 keywords which were encountered only once over the whole analyzed period, 1053 keywords which were encountered only twice, and 448 keywords which were encountered 3 times each. The large number of single-instance keywords is clearly an indicator of a very wide thematic span of the analyzed dataset; among such keywords, there were, e.g., “3d digital storybook”, “Ada”, “business process simulation”, “certification systems”, “digital citizenship education”, “egyptology”, “Filipino sign language”, “global warming”, “honeymoon effect”, “information ecology”, “joyful learning”, “Kolb cycle”, “language fluency”, “Mandarin as second language”, “nutritional knowledge”, “on-the-hour effect”, “passive haptics”, “quality education”, “recurrent neural networks”, “scale development”, “team formation”, “user research”, “virtual library”, “WordNet”, “xMOOC”, “young workers”, and “zero-cost implementation”.

All bibliographic data processing, unless specified otherwise, was performed using dedicated scripts written in Python.

3. Results and Discussion

3.1. The Evolution of the Yearly Number of Publications

Before we proceed to the keyword analysis, Figure 1 shows the quantitative development of the educational gamification field during the last 10 years.

As can be observed in Figure 1, the field was characterized by a very fast consistent growth from its beginnings until the year 2019. There is a small drop in 2020 (which could be possibly linked to the general decrease in research efforts due to the COVID-19 pandemic and the turmoil it caused in higher education institutions [63]) and an even smaller rebound in 2021 (possibly indicating that the educational gamification research is slowly getting back on its pre-COVID-19 level).

3.2. The Overview of Educational Gamification Research Topics

In order to provide an overview of the topics appearing in educational gamification research, a word cloud has been drawn based on the list of the most frequent keywords. The WordClouds.com tool has been used for this purpose [64]. The resulting word cloud is shown in Figure 2.

Based on the observations in Figure 2, the following groups of keywords used in the research on educational gamification could be identified:

• Keywords denoting the purpose of applying gamification (keywords such as “motivation”, “engagement”, “learning”, “teaching”, or “training”);
• Keywords denoting the means, using which type of gamification is being applied (keywords such as “game”, “serious games”, “game-based learning”, or “educational games”);
• Keywords denoting the specific area or the subject of applying the gamification (keywords such as “higher education”, “e-learning”, “blended learning”, “mobile learning”, “MOOC”, “course”, “university”, “children”, or “engineering education”);
• Keywords denoting the technological aspect of applying gamification (from general keywords such as ”technology” and ”educational technology”, across keywords indicating the kind of used technology, such as “augmented reality” or “virtual reality” to the names of specific solutions, the only visible example of which is “Kahoot!”);
• Keywords denoting the character of the contribution of particular research (keywords such as “design”, “model”, “development”, “evaluation”, “case study”, “analysis”, or “review”).

Note that some keywords can be assigned to more than one group, as they can be used for different meanings, e.g., “application” could mean either software (i.e., member of group 4) or the act of applying some method to solve a given problem (i.e., member of group 5).

The identified groups of keywords allow for the indication of the main directions of research on educational gamification:

• Verifying the effectiveness of gamification in education, with regard to known purposes of applying gamification, as well as finding new purposes of applying it;
• Proposing and analyzing the means using which gamification can be applied in educational domain;
• Identifying the specifics of applying gamification to specific educational areas or subjects;
• Adapting or developing new technologies supporting educational gamification;
• Designing, developing, and validating new models of educational gamification, as well as reporting case studies and analyzing prior research results relevant to educational gamification.

3.3. The Most Frequently Used Keywords

Throughout the entire analyzed period, the top 10 keywords were as follows: “higher education” (used 383 times), “game-based learning” (363), “motivation” (353), “serious games” (314), “game” (298), “learning” (273), “e-learning” (224), “teaching” (204), “engagement” (186), and “use” (149). These 10 keywords together were used 2747 times, which constitutes about 10% of all counted uses. The top 25 keywords (covered in Section 3.5) were used 4393 times, which constitutes about 16% of all counted uses.

Table 1. The most frequently used keywords for each year covered in the analysis.

Year	1st Most Frequent	2nd Most Frequent	3rd Most Frequent	4th Most Frequent	5th Most Frequent
2012	e-learning, game-based learning, motivation (3 each)			educational games, serious games (2 each)
2013	e-learning (13)	motivation, learning, game-based learning (12 each)			serious games (11)
2014	motivation (19)	game, game-based learning (14 each)		educational games, serious games (12 each)
2015	serious games (35)	game (28)	e-learning (24)	motivation (22)	higher education (21)
2016	game-based learning (42)	serious games (41)	e-learning (34)	game (32)	motivation (30)
2017	learning (56)	higher education, motivation (43 each)		game-based learning (39)	serious games (34)
2018	motivation (73)	higher education (57)	game-based learning (46)	game (43)	serious games (42)
2019	higher education (61)	game-based learning (60)	motivation (57)	game (51)	serious games (50)
2020	higher education (69)	game-based learning (56)	game (48)	motivation (46)	teaching (42)
2021	higher education (91)	game-based learning (70)	serious games (50)	game (49)	motivation (48)

lists the 5 most frequent keywords for each year covered in the analysis.

As can be observed, with the growing number of papers published each year, the number of appearances of the top keywords grew as well; having three uses was enough to become the most frequent keyword in the area of educational gamification in 2012, whereas it required over 90 uses in 2021.

Only one keyword appeared in the top five list throughout the entire analyzed period: “motivation”. As motivation is mentioned in the presented definition of educational gamification [1], this could be expected; what was not expected is the much lower popularity of the second term mentioned in the definition: “engagement”, which, although included among the ten most frequent keywords, did not make it into the top five list in any year of the analyzed period. This indicates that engagement (or the rising thereof) is a gamification purpose that is tackled more rarely than motivation (or the building thereof) by educational gamification researchers, despite the fact that, in education, there are multiple sources of motivation [65], whereas the lack of engagement is often a serious issue, especially in formal education [66].

Interestingly, the two keywords which were missing from the top five (only once, in 2015 and 2020, respectively) were “game-based learning” and “serious games”. Together with “game” (having six appearances in Table 1), these keywords have a non-obvious relationship to gamification, which is usually portrayed as a concept different to fully-fledged games [67]. After screening the publications using these keywords, it was found that, while some of them are actually devoted to both serious games and strictly defined gamification, far more of them simply denote the application of games in education as gamification, clearly indicating that not all researchers are aware of the distinction between the two concepts or agree with it. This point of view can be easily understood if we consider gamification as a process of making non-game activities resemble games, with the most extensive form of this process consisting in turning non-game activities into fully-fledged games.

“Higher education” also made it into the top five keywords six times, but was the keyword which took the top spot most often (three times). No other keyword denoting a level of education made it into the top five for any year during the analyzed period. For a comparison, during the whole period, “higher education” was used 383 times as a keyword, whereas “primary education” was used only 39 times and “secondary education” only 33 times. Such a huge difference suggests that educational gamification researchers are most interested in implementing gamification where they themselves teach, and this is most often in higher-education institutions. This is in contrast to the evolution of students’ interest in games, which was reported to peak at the middle-school age [68].

While “learning” appears twice in Table 1 (both times in the top spot in the respective years), “teaching” appears only once (and in the fifth spot), and “training” did not qualify for the top five list during any year. This indicates that the educational gamification literature is much more focused on the student’s perspective than the teacher’s.

Only one keyword denoting a mode of education made it to the top five keyword list: “e-learning” (in four years: 2012, 2013, 2015, and 2016). It was used 224 times during the whole analyzed period; for comparison, in total, only five uses of “traditional learning” and three uses of “face-to-face learning” were found, and not a single instance of “brick-and-mortar learning”, whereas “blended learning” was used 71 times. This clearly indicates that, despite gamification being used in offline contexts, it is most often applied to computer-mediated instruction and learning.

3.4. Keyword Co-Occurrence

Figure 3 presents 40 pairs of keywords with the highest measured value for the Pearson correlation coefficient (the beige line between two keywords indicates such a pair). While most of these pairs consist of two obviously-related concepts (e.g., “educational games” and “educational technology”, “engagement” and “motivation”, “augmented reality” and “virtual reality”, “active learning” and “flipped classroom”, “innovation” and “technology”, or “game” and “serious games”), some connections bring some new insights, as is stated in the following.

• The links between “design” and “evaluation” as well as “development” and “evaluation” suggests that proposed designs and developed solutions were often evaluated;
• The link between "Kahoot!” and “use” suggests that the papers on Kahoot! most often merely reported its use;
• The link between “augmented reality” and “mobile learning” suggests that the former technology was often applied to the latter form of learning;
• The link between “training” and “virtual reality” suggests that the latter technology was often used for training (rather than, e.g., primary education).

3.5. The Change in Keyword Usage

Figure 4 compares the number of occurrences for the top 25 keywords in each year during the analyzed period. We can clearly observe the large difference in the number of occurrences between the most frequently used keywords and the remaining ones, and for the first part of the analyzed period, the fast growth in the number of occurrences.

In order to make it easier to spot the change in the number of occurrences (also for the less frequent keywords, Figure 5), the data for the same top 25 keywords are presented using relative values (100% denotes the maximum number of occurrences of a given keyword in any year during the analyzed period).

Based on usage trends visible in Figure 5, we can assign each of the top 25 keywords to one of the following groups:

• Consistent fast growth in popularity (“higher education”);
• Consistent growth in popularity, with minor fluctuations (“game-based learning”, “game”, “virtual reality”, “course”, “evaluation”, “training”, “active learning”, “educational technology”, “effects”);
• Passed peak of popularity (“learning”, “motivation”, “teaching”, “engagement”, “use”, “educational games”, “case study”, “MOOC”, “application”, “mobile learning”);
• Rebound of popularity (“e-learning”, “design”, “development”);
• High plateau of popularity (“serious games”, “augmented reality”).

Regarding group 1, and its only representative, it is startling to see how fast and consistent the interest in gamification in higher education was, growing throughout the 10 analyzed years. While this finding is consistent with prior research [31,38,41], none of these indicated the unique status of this research topic. It is interesting to think about how long this trend may continue, and whether the interest in gamification of higher education will form a plateau, staying at a high level, or will decrease more or less rapidly.

Regarding the keywords of group 2, despite there being some fluctuations in popularity (i.e., individual years with the number of publications over or below the trend line), they generally kept growing throughout the analyzed period. Especially worth noting is one member from this list: “evaluation”, indicating a growing interest in evaluating educational gamification applications, which is a good sign considering that this aspect of the educational gamification research has been somewhat neglected in the past, as indicated by the fact that only 20 out of 179 educational gamification papers analyzed in [37] included evaluation.

The popularity of the keywords belonging to group 3 seems to have passed its peak. It seems that the volume of writing on motivation and engagement in the educational-gamification context has reached some level of saturation. It is quite surprising to see a fall in the interest in the application of educational gamification to Massive Open Online Courses (“MOOC”) and to mobile learning. As for “learning” and “teaching”, this may indicate just a change in the selection of the used keywords from very general (as the two mentioned keywords are) to more specific, which is understandable as the field matures.

The group 4 keywords seem to have surpassed a decrease in interest and returned to the path of popularity growth. For “e-learning”, the explanation seems straightforward, as the rebound in its popularity can be clearly linked to the outburst of the COVID-19 pandemic and the emergency remote teaching it resulted in [69].

The popularity of keywords in group 5 reached their highest popularity level earlier (“serious games”) or later (“augmented reality”) and has stayed there since (with small fluctuations). The last member of this group could be seen as unexpected, as educational gamification applications using augmented reality, in spite of many of them being developed (see, e.g., [70]), have not been as widely used as non-educational augmented reality applications such as Pokémon GO (of course, we are aware of the educational use of Pokémon GO, as evidenced, e.g., in [71], but this is not a common practice either).

3.6. The Fresh and Abandoned Keywords

The list of all keywords has been screened to find the most popular keywords that have started to appear only in the last three years, as well as the most popular keywords that have no longer appeared in the last three years. The results of this screening are shown in

Table 2. The most popular fresh keywords.

Keyword	Uses in 2019	Uses in 2020	Uses in 2021	Uses in 2019–2021
COVID-19	0	18	34	52
immersive virtual reality	1	3	5	9
meta-analysis	1	5	3	9
board games	2	4	2	8
nutrition	3	1	3	7
online gamified learning	1	3	3	7
student behavior	4	0	3	7

and

Table 3. The most popular abandoned keywords.

Keyword	Uses in 2012–2018
persuasive technology	9
pervasive games	9
games and learning	8
social networking	8
virtual learning	7

, respectively (only keywords with a total of at least seven occurrences in the respective sub-period were included in order to keep the tables compact).

As could be expected, “COVID-19” made it to the top of the fresh keywords list. This is understandable, considering the huge disruption it caused to the whole educational domain [72].

The remaining fresh keywords attained much less popularity. The most unexpected is the presence of “meta-analysis” on this list, indicating that there were no meta-analyses on the topic of educational gamification published prior to 2019.

There are no highly popular keywords listed in Table 3, which means none of the wider sections of research on educational gamification ceased to exist. Quite unexpectedly, “persuasive technology” and “pervasive games” top this list, even though the general interest in these topics (i.e., outside of educational gamification) keeps increasing (see, e.g., [73]). Maybe even more puzzling, is the lack of continued interest in “social networking” among the educational gamification researchers, even though social network is often listed as one of primary gamification components (see, e.g., [74]). As for “games and learning”, both its component terms remain highly popular keywords, so, apparently, it is just that the educational gamification researchers simply no longer combine them into one keyword.

4. Conclusions and Implications for Future Work

Gamification in education is a quickly growing area of research, with hundreds of scientific papers devoted to it being published every year. Although this body of literature has been the subject of numerous secondary research studies [1,2,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45], so far, none of these focused on how the popularity of respective covered topics changed over time.

This paper contributes to addressing this gap via keyword analysis, helping to understand the evolution of the set of topics most often selected in the research on educational gamification.

The obtained results provide a handful of novel insights. Already, the identification of the most frequent keywords throughout the entire analyzed period revealed that two are in a non-obvious relationship to gamification (“game-based learning” and “serious games”, which are perceived by some authors as distinct from gamification [67]) and a lack of balance both between student–teacher perspectives (with much more attention given to the student perspective) and the different levels of education (with much more attention given to higher education than to the other levels of education). This brings implications to future work: firstly, unless a significant share of gamification research is to be ignored, a more comprehensive definition of gamification is needed, clearly placing, under its umbrella, the adoption of fully-fledged games for non-entertainment purposes (rather than only game elements). Secondly, more research attention is needed on those aspects of educational gamification that were insufficiently addressed by prior studies, i.e., the gamification of teaching (rather than learning) and the application of gamification at various education levels (rather than higher education only).

The analysis of keyword usage in subsequent years reveals that, despite the consistent growth in the volume of educational gamification research, the popularity of the most frequent keywords has evolved in differing ways. Only 10 of the top 25 most frequent keywords were characterized by steady growth in their yearly number of occurrences, whereas, for the same number of keywords, the number of occurrences decreased in the most recent years. The popularity level of three keywords has stabilized, and two keywords rebounded from a period of lower popularity. The analysis of these patterns also provides some implications for further research, such as the need for a meta-analysis of the models used in educational gamification or the need for the identification of the barriers hampering the spread of the numerous gamified augmented reality applications into educational practice.

We have also identified keywords that came into use only recently and keywords that are no longer used in spite of having some popularity in the past. As the former group indicates the emerging research fronts (the main of which regards COVID-19), the latter directs the educational gamification researchers’ attention toward topics that have lost it (in particular, “persuasive technology”, “pervasive games,” and “social networking”).