Gamification Techniques and Best Practices in Computerized Working Memory Training: A Systematic Literature Review

Abstract

Computerized Cognitive Training (CCT) programs have been widely used in the past decades, offering an alternative solution in enhancing cognitive functionalities, especially Working Memory (WM). CCT supports users to overcome the monotonous context of training by utilizing specific game elements (GE). Several previous studies focused on the efficacy of CCT, but only a few examined their potential in increasing motivation and adherence. This study aimed to (a) conduct a state-of-the-art systematic literature review to identify the most commonly used GEs in WM training and assessment; and to (b) investigate how they are utilized in regard to the audiences that are being applied. In total, seven online databases were searched using keywords related to WM and CCT, targeting studies from 2015 until early 2022. The systematic review identified 44 studies which were eligible for inclusion. The results report that the most widely used GEs are conflict (88.63%), feedback (84.9%), difficulty adjustment (73%), action points and levels (70.45%). On the other hand, GEs associated with competition and cooperation are not preferred except in very few cases. In conclusion, there is common ground in the use of GEs for WM training, but there is a need for further research to compare the GEs between them.

1. Introduction

Over the past three decades, there has been an increasing demand for discovering and fine-tuning alternative ways for cognitive enhancement and the support of brain functions in the context of cognitive training (CT), whose purpose is to maintain or improve someone’s cognitive abilities. These functions include notions such as learning and reasoning, attention and assessment, speech and language skills and more. Among these functions, memory is placed high on the hierarchy, as it is the faculty of the human brain where information is encoded, stored and retrieved when needed. Memory comprises a sensory processor, the short-term memory and the long-term memory [1]. Working Memory (WM), as part of the short-term memory, along with self-control and flexibility, are the core of executive functions of the brain [2,3]. Baddeley introduced a model of WM which contains three components: the central executive (attention), the phonological loop (for storing phonological information) and the visuo-spatial sketchpad (for storing visual and spatial information) [4]. In addition, it is the cognitive system which supports numerous cognitive abilities such as reasoning and problem solving [5]. WM is responsible for processing and encoding new incoming information, which leads to novel knowledge. It plays a critical role in performing simple tasks for daily living up to functioning at a higher level, which can be critical in work, business and academia.

WM performance can be measured by its capacity, which is related to the total items that one can retain in it. There are several aspects about the actual capacity, but the most prevalent is Miller’s, who supports that an individual can retain seven plus–minus two items each time [6], and another prevalent one is Cowan’s, who states that only four items can be held, considering excluding cognitive iteration or information storage, which are included in the long-term memory [7]. Regardless, capacity depends on the kind of items questioned, since it differs whether these items are numbers, letters or whatever other items possible. Moreover, the timespan of this capacity is limited as well, ranging from 10 to 15 s, but it can be increased only if the information processed is actively applied or if it is repetitive, which, in these cases, it becomes part of the long-term memory [8]. Given its limited capacity, and its importance in daily living activities and routines, as it affects learning, attention and problem-solving [9,10,11], it becomes imperative to keep WM in an optimal state.

Based on neuroplasticity (the ability of the brain to modify, change and adapt its structure and functions by forming new neural connections) the capacity of WM can be increased through targeted training CT. There were a number of studies that demonstrated improvements in WM through extensive training [5,12,13]. Increasing WM’s capacity by certain training has been accepted and utilized in past years, presenting encouraging and positive results in many different groups of people. WM training can utilize the same methods as in memory training, including basic training, strategic training and mnemonic strategies such as rehearsal and repetition of information [14]; chunking (organizing information into manageable bits or chunks) [15]; building mental representations for information encoding [16]. A simple strategy, according to Turley-Ames and Whitfield, is the repetition of information, which strengthens WM and is appropriate for people with low WM capacity, such as the elderly [17]. On the other hand, WM decongestion techniques are being used to free space for WM to function better. Therefore, not all improvements concern increasing capacity, but freeing space in WM can lead to similar results. Computer-based CT offers standardized tasks in a challenging way [18] that target certain cognitive functions. It is based on the assumption that cognitive abilities can potentially be improved by performing challenging tasks repetitively in a specific time frame [19]. The training activities typically include practice on tasks which are designed to enhance cognitive skills, such as attention, memory and processing speed [20].

The main purpose of cognitive training interventions, and particularly WM training programs, is to improve the subject’s WM capacity. However, traditional and conventional systems are characterized as tedious and repetitive, and that strongly affects users’ motivation to learn and adhere, which consequently reduces any potential for learning transfer [21,22,23,24]. Prins et al. showed that WM training can significantly improve motivation and training performance, if it is being decorated by GEs [25]. Either utilizing GEs in order to gamify a serious intervention program (gamification) [26] or using certain games with serious goals with entertainment coming as a secondary objective (serious games), the main outcome is to improve WM but with increased motivation and engagement at the same time. In reality, it is often hard to make a clear distinction between gamification and serious games [27]. In general, serious games are full-fledged games that have a typical game structure and, at the same time, add educational value beyond entertainment, and gamification utilizes game-like mechanics and embodies them in traditional methods and programs to increase engagement.

Over the past decades, several studies have demonstrated that an increase in WM capacity can be achieved with computer-based (computerized) training. It has been used in adults with the purpose of improving and enhancing cognitive skills, but it has also been applied to children (with typical or atypical development) [5,12,28,29,30,31,32]. The results so far are inconsistent and sometimes controversial, and the main debate is between near and far transfer effects. Most studies have reported that WM training leads to near-transfer improvements (related to the task that is being trained), such as verbal and visuo-spatial WM [5,12,32]. However, there are cases with little to no evidence regarding far-transfer improvements (effects in other cognitive functions) [33,34], and there are fewer cases reporting improvements in other cognitive functions. In particular, a transfer effect was found in attention inhibition, reasoning, reading and arithmetic [13,35,36,37]. In this direction, several commercial training programs have been developed, and their effectiveness has undergone testing, demonstrating positive effects (Brain Age [38], Cogmed [39], Lumosity [40]) or little to no effects [31,35,41].

Utilizing GEs, gamification or serious games allow computer-based interventions to be more entertaining and playful, be easy to comprehend and perform, and offer feedback and reinforcement, and as a result, they foster adherence and motivation, rendering this kind of intervention far better than the traditional, non-gamified programs [26,42,43,44].

The theory behind utilizing GEs in CCT is the Self-Determination Theory [45], which is the most prevalent in the field of gamified learning. SDT defines motivation as comprised by intrinsic and extrinsic motivation. The first one refers to the motivation that is developed when someone performs an activity for its inherent satisfaction and feels the competence during the activity. Extrinsic motivation refers to the satisfaction of performing an activity only for its outcome. Typically, a subject begins with intrinsic motivation, moving to extrinsic motivation until it ends up with no motivation at all for the activity. Intrinsic motivation has a crucial role in adherence and long-term participation in an activity, whereas extrinsic motivation is more suitable for short-term tasks [46]. Likewise, with interaction and collaboration with other peers, the processes of learning and problem solving can be constructive and facilitative, supported by theories such as the zone of proximal development [46] and scaffolding theory [47].

Defining GEs is difficult since there is no commonly accepted definition. Generally, such elements may include components such as points, badges and leaderboards, and mechanics such as competition, challenge and win state. It is worth mentioning that game components are often mistakenly overlapped by game mechanics. However, in reality, components should be treated as the subset of mechanics, since they are the basis that drive the mechanics. In order to overcome the obstacle of the absence of a well-defined context for GEs, by searching the literature for the most commonly accepted terms, and also based on the findings from [48,49,50,51,52,53,54,55], we decided to conclude GEs to the following nine: (a) Narrative/Storytelling; (b) Avatar; (c) Conflict (challenge); (d) Cooperation and Competition; (e) Difficulty adjustment; (f) Feedback; (g) Levels; (h) Progression; and (i) Action Points.

Gamification as a strategy has been developed over the last decade, and its effectiveness has been tested in numerous studies, as mentioned previously. The majority of studies have been heavily focused on the impact of gamification in cognitive performance, but fewer studies have actually tested the impact of certain GEs in a scientific framework. A possible explanation is that researchers have been developing computerized CT by simply converting traditional paper-and-pencil tasks to digital tasks by also adding arbitrary GEs, since there is a lack of scientific framework that dictates how to build electronic interventions. On the other hand, the introduction of gamification has been utilized as a tool to increase motivation and long-term engagement, and this aspect has been evaluated as well.

Several reviews and meta-analyses on gamified computerized CT have been conducted, providing useful and interesting results. For example, in the systematic review by Vermeir et al., which investigates the effect of gamification on process outcomes and on the training domain, they concluded that action points (rewards) and feedback are dominating the gamification landscape, but social features such as competition are underused [55]. Moreover, gamified tasks have been proven to be more motivating and demanding, but no effects on the training domain were found [55]. Similarly, in the study of Ferreira-Brito et al., which tried to identify what GEs are being applied for cognitive training, assessment or rehabilitation, they reported the scoring system and narrative context as the most used GEs and a strong association between usability and six out of the seven GEs that were analyzed. An interesting finding was that using GEs that act as extrinsic motivation promotors can potentially jeopardize patients’ long-term adherence to interventions, especially if associated with progressive difficulty [53]. In another meta-analysis, the effectiveness of computerized CT with game-like features in school-aged children with typical and atypical development was examined and showed that it can improve cognitive and behavioral performance in both populations, and it may help to make the training less burdensome for children, fostering motivation [56].

The work of Lumsden et al. is likely one of the most known works regarding GEs, in which they aimed to explore and evaluate how gamification has already been used for CT and evaluation purposes. The authors reported that certain elements such as action points (rewards) and feedback are suitable for people with Attention-Deficit Hyperactivity Disorder (ADHD), who are especially responsive to immediate feedback and to the clear definition of goals and objectives, but mixed effects of gamification on task performance were reported [49]. In another systematic review by Cao et al., regarding the investigation of training and transfer effects of computerized training on executive functions in children, results demonstrated a moderate effect size, and the transfer effect was more explicit in near-transfer conditions. Typically, developing children improved more during training, but the addition of GEs negatively affected the training and transfer effects [57].

On the other hand, there were meta-analyses which evaluated the impact of CT programs, such as the one by Bonnechere et al., in which they examined the use of commercial computerized cognitive games which targeted elderly people (>60 years old) without cognitive impairment. Statistically significant improvements were observed for processing speed, working memory, executive function and verbal memory, but not for attention or visuo-spatial abilities, concluding that these games are effective in improving cognitive function in such participants [58]. Finally, Lau et al. conducted a systematic review and meta-analysis that evaluated the effectiveness of serious games on symptoms of mental disorder, and despite the small number of the included studies, their findings suggest that serious games may have a positive effect in reducing disorder-related symptoms [59].

The previous reviews and meta-analyses provide valuable information about gamification, serious games and computerized CT, and they shed light on how GEs have been utilized. Although some of them do include a limited number of studies, or studies with no strict methodological frameworks, the results that have been exported seem to be consistent regarding the GEs being used. One big drawback, however, is that the outcomes of the studies were treated in total, despite the fact that there was discrimination between WM, executive functions, attention, etc., but, to our knowledge, there is no systematic review that targets WM specifically. In addition, this is important considering the part that WM plays in overall cognitive status. Furthermore, the included studies in some cases are more than 10 years old, rendering some systematic reviews and meta-analyses outdated, especially considering the increased usage of smartphones, tablets and similar portable devices, which can help computerized interventions to be more easily accessible and can allow interactions between people at any time and place [48]. Thus far, the existing studies on computerized CT have been implemented as pilot studies concerning different samples, and although there are some conclusions regarding which GEs are suitable for each category, the number of studies is limited and cannot lead to safe and solid results. So far, there is a lack of framework for the usage of GEs for treating different groups of people effectively.

The aim of the current study was to (a) conduct an updated systematic review of literature (following PRISMA guidelines [60]), which tries to identify which GEs are most frequently used in CCT targeting, mainly in WM performance; and to (b) attempt to categorize GEs based on the audience being applied (children, adolescents, adults, older adults) and provide any useful information for building computerized WM interventions to best serve their specific needs and limitations. In the next sections, we firstly present the methodology that we used for the identification of potential eligible articles that could be used in the systematic review under PRISMA guidelines, then we continued with the results of the search process and the presentation of the data being collected, followed by a discussion on these results and the extraction of any useful conclusions. Lastly, we finish with the research limitations that define this study.

2. Materials and Methods

This section describes the methodology used for the systematic literature review, which followed the PRISMA guidelines [60], and it is divided into (a) the eligibility criteria for the inclusion of articles; (b) the information sources that were used for searching available studies; (c) the search strategy along with the keywords that were used; (d) the process of the selection of studies; (e) the data extraction process; and finally, (f) the data items that were documented.

2.1. Eligibility Criteria

The inclusion criteria for the studies are the following:

• Computer-based gamified cognitive training tasks.
• Available empirical and original data related to gamification and/or GEs.
• Peer-reviewed articles available in English.
• WM performance measures as outcomes.
• Publication year between 2015 and early 2022 (January).

It was also decided to include the term “serious games”, as in many cases this term overlaps with the term “gamification”, and gamified tasks can be reported as serious games [27]. On the other hand, studies were excluded if they did not report any GEs in the training process, if they lacked any measure of WM performance and if they used commercial video games without serious purposes or simple representations of paper-and-pencil tasks.

2.2. Information Sources

A literature search was conducted in online scientific databases from December 2021 to February 2022. The databases that were included in this review were PubMed, Scopus, Web of Science, Institute of Electrical and Electronics Engineers (IEEE), Crossref and Google Scholar. In addition, reference lists from included studies and literature reviews were also manually searched for by spotting any other relevant works that could potentially be included. Since there were several literature reviews with similar objectives, they were used as reference points for building the current research but were mainly served as additional sources to identify more articles. Although they did provide relative information, nevertheless, all the included papers were studied again from the beginning.

2.3. Search Strategy

The search criteria included:

• Publication year from 2015 to January 2022.
• Empirical research studies, peer-reviewed articles (e.g., published papers, doctoral theses, study protocols, conference papers).
• Full text in English.
• Articles published in peer-reviewed journals and conferences.
• Computer-based interventions with WM performance measures as outcomes.
• Available information regarding the cognitive task being used.

Considering a PICO approach for the inclusion and exclusion criteria, we have the following:

• Population: Any participant (healthy or cognitively impaired) of any age (from children to older adults).
• Intervention: Studies using computerized WM training or assessment of WM with GEs.
• Comparison: Active or passive WM training.
• Outcomes: Outcomes focusing primarily on the performance of WM and secondarily on outcomes related to participants’ engagement.

Search terms were formed as a combination of cognitive training and gamification with every possible combined search phrase. Combinations included terms of the following: (a) cognitive training; (b) brain training; (c) cognitive rehabilitation; (d) serious game; (e) computerized/computer-based/electronic interventions; (f) game elements; and (g) gamification. We searched the titles, abstracts and keywords by combining computerized OR computer-based OR electronic AND cognitive training OR brain training OR cognitive rehabilitation OR memory training OR working memory training OR executive functions training AND serious games OR game elements OR gamification OR game mechanics OR game OR games OR video games. In addition, we also used terms with wildcards such as gamif*, cognit*, train* and comput*. By making use of the above keywords and their combinations, we hoped to minimize the risk of excluding any potential entries that could be under less common terms than the ones used. Titles, abstracts and keywords of database entries were searched using the search strategy.

2.4. Selection Process

As mentioned, since gamification and serious games tend to be treated as the same, the selection was careful and sensitive for articles that contained these terms, and the initial selection stage did not exclude any terms such as serious games, video games or computer games. For this stage, all records were included without further limitations. After documenting search results from all databases in a spreadsheet, any duplicates were removed first, and the remaining records were screened by both title and abstract according to the eligibility criteria. If it was unclear or not possible to determine the eligibility of a record from the title and abstract, the full-text search was followed. Full-text records were retrieved and evaluated against the inclusion criteria (AC). To check the reliability of the process, a second author (TS) assessed 80% of the selected full-text records, which resulted in no disagreement. Review authors were not blinded to the authorship, institution, journal or results.

2.5. Data Collection Process

After screening and finalizing the included studies, the data extraction process was followed. For this purpose, a spreadsheet was used as a standardized data extraction form. Data regarding research questions and other relative questions were extracted for each paper. Three main categories of data were identified: (a) the study’s main characteristics (e.g., title, author(s), publication year); (b) study design and participants (intervention strategy and characteristics, outcomes); and (c) GEs used. The response formats were mainly open-answer formats for data related to the studies’ information and closed-answer regarding the gamification data. When there was no available information, even after any further online search, the response was characterized as not available (N/A).

2.6. Data Items

For the documentation of the study process, the following data were included: (a) general study data, containing information such as the title, author name and publication year; (b) study characteristics, design and methods, including data such as the sample, sample size, sessions, follow-up, information about the gamified process being used, device used (computer/laptop, smartphone/tablet, VR equipment, console) and (c) data regarding the presence of any GEs, as these elements were defined previously (narrative/storytelling, avatar, conflict, cooperation/competition, difficulty adjustment, feedback, levels, progression and action points) [48,49,50,51,52,53,55]. Regarding the extraction of GEs, in order to have a better and detailed view of each training program that was used, additional online searches were conducted in order to locate supplementary material.

2.7. Assessment of the Risk of Bias of the Studies

In order to assess the risk of bias of each study, we used Version 2 of the Cochrane risk-of-bias tool for randomized trials [61], according to the description in the Cochrane Handbook for Systematic Reviews of Interventions. The tool is structured into five bias domains (bias arising from the randomization process, bias due to deviations from intended interventions, bias due to missing outcome data, bias in the measurement of the outcome and bias in the selection of the reported results). Judgments were made by 2 authors (AC and TT) independently, and a consensus was reached for existing variations. For each domain, the risk of bias was judged as either low risk, some concern or high risk.

3. Results

3.1. Study Selection

Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses, the flow diagram of the study selection and the selection process is depicted in Figure 1. Overall, 1847 papers matched the initial set of keywords used in the search process. After removing duplicates, 1017 papers were screened considering the titles and abstracts. In total, 895 papers were excluded on the basis of this analysis. From the 122 eligible papers for the full-text analysis phase, 44 were included in the current review.

Some frequent exclusion reasons were (a) paper not using any computerized tasks; (b) papers reporting the use of commercial video games without a serious purpose; (c) working memory was not reported as cognitive outcome; (d) literature reviews; and (e) the full-text not available in English.

3.2. Study Characteristics

A sample of the data collected for the 44 studies that were included in the systematic review are presented in

Table 1. Summary of studies included in the systematic review.

Author(s); Year	Sample; Size	Mean Age (years); %Female
Ackermann et al.; 2018 [62]	Adolescents; 60	13.8; 21.66
Areces et al.; 2018 [63]	Students with ADHD; 88	10.2; 25
Armando et al.; 2016 [64]	Schizophrenia patients; 8	38.6; 12.5
Ballesteros et al.; 2017 [65]	Healthy adults; 55	65.3; N/A *
Baniqued et al.; 2015 [66]	Healthy adults; 90	21; 57
Biel et al.; 2019 [67]	Healthy older adults; 83	63.93; 47
Bikic et al.; 2018 [68]	Adolescents with ADHD; 70	9.95; 15.71
Boendermaker et al.; 2018 [69]	Adolescents; 84	13.7; 60
Boletsis and McCallum; 2016 [70]	Healthy older adults; 5	67.6; N/A
Boot et al.; 2016 [71]	Older adults; 60	72.35; 57
Cujzek et al.; 2016 [72]	Older adults; 29	73.25; N/A
Dassen et al.; 2017 [73]	Overweight adults; 67	47.97; 75
De Giglio et al.; 2015 [74]	MS patients; 52	43.9; 74.3
De Vries et al.; 2015 [75]	Children with autism spectrum disorder; 90	10.56; 9
Dörrenbächer and Kray; 2018 [76]	ADHD children; 26	10.54; 30.76
Double and Birney; 2016 [77]	Older adults; 794	61.95; 77
Dovis et al.; 2015 [78]	Children with ADHD; 81	10.5; 20
Garolera et al.; 2015 [79]	Children with a clinical diagnosis of ADHD; 17	50–90 y.o; N/A
Goghari et al.; 2018 [80]	Healthy older adults; 97	70.5; 66.5
Gray et al.; 2019 [81]	Children; 28	11–23 y.o.; 42.85
Hessl et al.; 2019 [82]	Children and adolescents with fragile X syndrome; 100	15.28; 37
Janssen et al.; 2015 [83]	MS patients; 28	47.18; 75
Johann et al.; 2018 [84]	Children; 60	9.31; 38
Leung et al.; 2015 [85]	Older adults; 209	70; 78.4
Mohammed et al.; 2017 [86]	University students; 115	19.98; 58
Nagle et al.; 2015 [87]	Older adults; 14	82.7; 93
Nagle et al.; 2015 [88]	Healthy older adults; 51	69.9; 48
Ninaus et al.; 2015 [42]	University students; 30	23.8; 80
Olfers and Band; 2017 [89]	Healthy adults; 72	23; 56
Palumbo et al.; 2019 [90]	Schizophrenia patients; 8	36.37; 27.5
Peijnenborgh et al.; 2016 [91]	Normal development and ADHD children; 136	6.38; 40.1
Ramani et al.; 2016 [92]	Kindergarteners; 148	5.98; 52
Rosetti et al.; 2017 [93]	Children; 75	8.5; 49
Sanchez-Perez et al.; 2018 [94]	Children; 157	9.17; 45.78
Savulich et al.; 2017 [95]	Patients with amnestic MCI; 42	76.05; 40
Scase et al.; 2017 [96]	Older adults with MCI; 24	75.13; 92
Souders et al.; 2017 [97]	Older adults with MCI; 60	72.25; 57
Tacchino et al.; 2015 [98]	Cognitive-impaired patients with MS; 16	49.06; 81.25
Tacchino et al.; 2020 [99]	Cognitive-impaired patients with MS; 15	52.6; 66
Ten Brinke et al.; 2019 [100]	Older adults; 41	72.88; 73
Wan et al.; 2020 [101]	Healthy adults; 20	22.85; 30
Wuang et al.; 2018 [102]	Children with visual–perceptual dysfunction/delay; 60	7.51; 46
Zhang et al.; 2018 [103]	Primary school students and kindergarteners; 91	6.12; 50
Zhu et al.; 2018 [104]	Adults with methamphetamine use disorder; 40	34.2; 0

* N/A = not available information.

and

Table 2. Description of games and GEs of studies included in the systematic review.

Study	Game	Game Elements *
Ackermann et al. [62]	Cogmed	C, PR, LV, FB, DA
Areces et al. [63]	AULA Nesplora	N/ST, C, FB
Armando et al. [64]	3D VR Virtual Town	AV, CM/CP
Ballesteros et al. [65]	Lumosity	C, PR, LV, FB, DA
Baniqued et al. [66]	Mind Frontiers	N/ST, C, AP, PR, LV, FB, DA
Biel et al. [67]	Typical two-back working memory task	C, FB
Bikic et al. [68]	ACTIVATE	C, LV, FB, DA
Boendermaker et al. [69]	Gamified Working Memory Capacity Training	C, AP, PR, LV, FB, DA
Boletsis and McCallum [70]	CogARC	C, AP, LV, FB, CM/CP
Boot et al. [71]	Mind Frontiers	N/ST, C, AP, PR, LV, FB, DA
Cujzek et al. [72]	PC version of card game Belote	C, AP, PR, FB
Dassen et al. [73]	Gamified WM training	N/ST, C, AP, DA
De Giglio et al. [74]	Dr. Kawashima’s Brain Training	C, AP, PR, FB
De Vries et al. [75]	Braingame Brian	N/ST, AV, C, AP, PR, LV, FB, DA
Dörrenbächer and Kray [76]	Game-based CT	N/ST, C, AP, PR, LV, FB, DA, CM/CP
Double and Birney [77]	Active Memory	AP, PR, DA
Dovis et al. [78]	Braingame Brian	N/ST, AV, C, AP, PR, LV, FB
Garolera et al. [79]	ACTIVE-U	N/ST, AV, C, AP, PR, LV, FB, DA
Goghari et al. [80]	BrainGymmer	C, AP, PR, LV, FB, DA
Gray et al. [81]	BrainQuest	C, CM/CP, AP, PR, LV, FB, DA
Hessl et al. [82]	Cogmed	C, AP, PR, LV, FB, DA
Janssen et al. [83]	Space Fortress	AV, C, AP, PR, FB, DA
Johann et al. [84]	Game-based tasks	N/ST, C, AP, PR, LV, FB, DA
Leung et al. [85]	Brain Fitness Program	N/ST, C, PR, LV, FB, DA
Mohammed et al. [86]	Recall the Game	N/ST, C, AP, PR, LV, FB, DA
Nagle et al. [87]	The Serious Game	N/ST, FB, DA
Nagle et al. [88]	WM Training Game	N/ST, AP, DA
Ninaus et al. [42]	GAME	N/ST, C, AP, PR, LV, FB, DA
Olfers and Band [89]	Lumosity	N/ST, AV, C, AP, DA
Palumbo et al. [90]	Computerized Interactive Remediation of Cognition—Training for Schizophrenia (CIRCuiTS)	N/ST, C, PR, LV, FB, DA
Peijnenborgh et al. [91]	Timo’s Adventure	N/ST, AV, C, AP, PR, LV, FB
Ramani et al. [92]	WM training: “Recall Them All”	N/ST, C, AP, PR, LV, FB, DA
Rosetti et al. [93]	Towi video game	N/ST, AV, C, CM/CP, AP, FB
Sanchez-Perez et al. [94]	WM Training Game	N/ST, C, AP, PR, LV, FB, DA
Savulich et al. [95]	Game Show	C, AP, LV, DA
Scase et al. [96]	Find it, match it, solve it, complete it	ST, AP, PR
Souders et al. [97]	Mind Frontiers	N/ST, C, AP, PR, LV, FB, DA
Tacchino et al. [98]	Cognitive Training Kit (COGNI-TRAcK)	C, PR, LV, FB, DA
Tacchino et al. [99]	CMI-APP	C, PR, LV, FB, DA
Ten Brinke et al. [100]	Fit Brains	C, AP, PR, LV, FB
Wan et al. [101]	Simon game and Merry Snowballs game	C, AP, PR, LV, FB
Wuang et al. [102]	Game-Based Auxiliary Training System (GBATS)	C, AP, PR, LV, FB, DA
Zhang et al. [103]	WM Training Game	C, AP, PR, LV, FB, DA
Zhu et al. [104]	CCAT app	C, PR, LV, FB, DA

* N/ST = Narrative/Storytelling; AV = Avatar; C = Conflict; CM/CP = Cooperation and Competition; AP = Action Points; PR = Progression; LV = Levels; FB = Feedback; DA = Difficulty Adjustment.

(for more detail, see

Table A1. Summary of studies included in the systematic review in alphabetical order.

Title	Author	Year	Publication Type, Domain	Country	Study Design	Sample	N	Age	%Female	Site	Device	Intervention Duration	Follow-Up	Sessions (N)	Session Duration	Game
Cognitive Working Memory Training (CWMT) in adolescents suffering from Attention-Deficit/Hyperactivity Disorder (ADHD): A controlled trial taking into account concomitant medication effects.	Ackermann et al. [62]	2018	Journal, Psychiatry	Switzerland	Between-subjects (pre-post); 2 groups (control, cognitive working memory training)	Adolescents	60	13.8	21.66	Home	PC	25 days	2 months	25	Depending on the difficulty level	Cogmed
Analysis of cognitive and attentional profiles in children with and without ADHD using an innovative virtual reality tool	Areces et al. [63]	2018	Journal, Psychology	Spain	Between-subjects (pre-post); 2 groups (ADHD, control)	Students with ADHD	88	10.2	25	Laboratory	PC	Single session	None	1	20 min	AULA Nesplora
A Serious Game to Improve Cognitive Functions in Schizophrenia: A Pilot Study	Armado et al. [64]	2016	Journal, Psychiatry	France	Single group	Schizophrenia patients	8	38.6	12.5	Laboratory	PC	3 months	None	12	90 min	3D VR Virtual Town
Effects of Video Game Training on Measures of Selective Attention and Working Memory in Older Adults: Results from a Randomized Controlled Trial	Ballesteros et al. [65]	2017	Journal, Medicine	Spain	Between-subjects (pre-post) design; 2 groups (experimental, control)	Healthy adults	55	65.3	N/A	Laboratory	PC	10 to 12 weeks	None	16	40 to 50 min	Lumosity
Working Memory, Reasoning, and Task Switching Training: Transfer Effects, Limitations, and Great Expectations?	Baniqued et al. [66]	2015	Journal, Psychology	USA	Between-subjects (pre-post); 2 groups (Mind Frontiers, control)	Healthy adults	90	21	57	Laboratory	Portable handheld devices	4 to 5 weeks	None	20	72 min	Mind Frontiers
The gains of a 4-week cognitive training are not modulated by novelty	Biel et al. [67]	2019	Journal, Medicine	Germany	Between-subjects (pre-post); 3 groups (WMT, WMT with novel nature movies, control)	Healthy older adults	83	63.93	47	Home	Tablet	4 weeks	None	12	36 min	Typical two-back working memory task
Attention and executive functions computer training for attention-deficit/hyperactivity disorder (ADHD): results from a randomized, controlled trial	Bikic et al. [68]	2018	Journal, Psychology	Denmark	Between-subjects (pre-post); 2 groups (SBT, active control (Tetris))	Adolescents with ADHD	70	9.95	15.71	Home	PC	8 weeks	12 weeks	48	N/A	ACTIVATE
Training Working Memory in Adolescents Using Serious Game Elements: Pilot Randomized Controlled Trial	Boendermaker et al. [69]	2018	Journal, Computer Science	Netherlands	Between-subjects (pre-post); 3 groups (control, WMC, CWMC)	Adolescents	84	13.7	60	School	PC	2 weeks	None	10	30 min	Gamified Working Memory Capacity Training
Augmented Reality Cubes for Cognitive Gaming: Preliminary Usability and Game Experience Testing	Boletsis and McCallum [70]	2016	Journal, Computer Science	Norway	Single group	Healthy older adults	5	67.60	N/A	Laboratory	Tablet	45–55 min	None	1	45 to 55 min	CogARC
The Gamification of Cognitive Training: Older Adults’ Perceptions of and Attitudes Toward Digital Game-Based Interventions	Boot et al. [71]	2016	Journal, Computer Science	USA	Between-subjects (post); 2 groups (brain training games, control games)	Older adults	60	72.35	57	Home	Tablet	1 month	None	30	45 min	Mind Frontiers
Computerized tabletop games as a form of a video game training for old-old	Cujzek et al. [72]	2016	Journal, Psychology	Croatia	Between-subjects (pre-post); 2 groups (active, control)	Older adults	29	73.25	N/A	Home	PC	6 weeks	4 months	12	30 min	PC version of card game Belote
Gamified working memory training in overweight individuals reduces food intake but not body weight	Dassen et al. [73]	2017	Journal, Psychology	Netherlands	Between-subjects (pre-post); 2 groups (gamified WM training, control)	Overweight adults	67	47.97	75	Home	PC or tablet	33.57 days on average	1 and 6 months	20–25	38.44 min on average	Gamified WM training
A Low-Cost Cognitive Rehabilitation With a Commercial Video Game Improves Sustained Attention and Executive Functions in Multiple Sclerosis: A Pilot Study	De Giglo et al. [74]	2015	Journal, Medicine	Italy	Between-subjects (pre-post); 2 groups (experimental, waiting list)	MS patients	52	43.9	74.28	Home	Nintendo Switch	8 weeks	None	40	30 min	Dr. Kawashima’s Brain Training
Working memory and cognitive flexibility-training for children with an autism spectrum disorder:a randomized controlled trial	De Vries et al. [75]	2015	Journal, Psychology	Netherlands	Between-subjects (pre-post) design; 3 groups (adaptive WM training, adaptive cognitive flexibility training, non-adaptive control training)	Children with autism spectrum disorder	90	10.56	9	Home	PC	6 weeks	6 weeks	25	45 min	Braingame Brian
The Impact of Game-Based Task-Shifting Training on Motivation and Executive Control in Children with ADHD	Dörrenbächer and Kray [76]	2018	Journal, Psychology	Germany	Between-subjects (pre-post); 2 groups (LowMot, HiMot)	ADHD children	26	10.54	30.76	School	PC	2 to 3 weeks	None	18 to 21	30 to 45 min	Game-based CT
The effects of personality and metacognitive beliefs on cognitive training adherence and performance	Double and Birney [77]	2016	Journal, Psychology	Australia	Single group	Older adults	794	61.95	77	Online	PC or tablet	18 months	None	Participant’s choice	Participant’s choice	Active Memory
Improving Executive Functioning in Children with ADHD: Training Multiple Executive Functions within the Context of a Computer Game. A Randomized Double-Blind Placebo Controlled Trial	Dovis et al. [78]	2015	Journal, Medicine	Netherlands	Between-subjects (pre-post); 3 groups (full-active, partially active, full placebo)	Children with a clinical diagnosis of ADHD	81	10.50	20	Home	PC	5 weeks	3 months	25	35 to 50 min	Braingame Brian
ACTIVE-U: PLAYING TO STIMULATE YOUR BRAIN	Garolera et al. [79]	2015	Conference Paper, Computer Science	Spain	Between-subjects (post); 2 groups (Unlocked, Active-U)	Patients with MCI	17	50 to 90 y.o.	N/A	N/A	iPhone	N/A	N/A	3	N/A	Active-U
Self-Perceived Benefits of Cognitive Training in Healthy Older Adults	Goghari et al. [80]	2018	Journal, Psychology	Canada	Between-subjects (pre-post); 3 groups (WMT, Logic and planning, control)	Healthy older adults	97	70.5	66.5	Home	PC	8 weeks	N/A	40	30 min	BrainGymmer
BrainQuest: The use of motivational design theories to create a cognitive T training game supporting hot executive function	Gray et al. [81]	2019	Journal, Computer Science	UK	Single group	Children	28	11 to 12 y.o.	42.85	School	Smartphone	7 weeks	None	8	60 min	BrainQuest
Cognitive training for children and adolescents with fragile X syndrome: a randomized controlled trial of Cogmed	Hessl et al. [82]	2019	Journal, Psychology	USA	Between-subjects (pre-post); 2 groups (adaptive Cogmed, non-adaptive Cogmed)	Children and adolescents with fragile X syndrome	100	15.28	37	Home	PC	5 to 6 weeks	3 months	25	15 min	Cogmed
The effects of video-game training on broad cognitive transfer in multiple sclerosis: A pilot randomized controlled trial	Janssen et al. [83]	2015	Journal, Psychology	USA	Between-subjects (pre-post); 2 groups (tablet, control)	MS patients	28	47.18	75	Laboratory	PC	8 weeks	None	20	60 min	Space Fortress
Validation of new online game-based executive function tasks for children	Johann et al. [84]	2018	Journal, Psychology	Germany	Between-subjects (pre-post); 2 groups (game-based version, standard version)	Students	60	9.31	38.1	Laboratory (school)	PC	2 weeks	None	2	N/A	Game-based tasks
Neural Plastic Effects of Cognitive Training on Aging Brain	Leung et al. [85]	2015	Journal, Psychology	Hong Kong	Between-subjects (pre-post); 2 groups (control, CT)	Older adults	209	70	78.4	Laboratory	PC	13 weeks	None	39	15 min	Brain Fitness Program
The Benefits and Challenges of Implementing Motivational Features to Boost Cognitive Training Outcome	Mohammed et al. [86]	2017	Journal, Psychology	USA	Between-subjects (pre-post); 2 groups (Tapback, Recall)	University students	115	19.98	58	Laboratory	Tablet	4 weeks	None	20	20 min	Recall the Game
Increased enjoyment using a tablet-based serious game with regularly changing visual elements: A pilot study	Nagle et al. [87]	2015	Journal, Computer Science	Switzerland	Between-subjects (pre-post); 2 groups (DDA, DDA-visual)	Older adults of assisted living facilities	14	82.7	93	Assisted living facility	Tablet	1 week	None	3	24 min	The Serious Game
High User Control in Game Design Elements Increases Compliance and In-game Performance in a Memory Training Game	Nagle et al. [88]	2015	Journal, Psychology	Switzerland	Between-subjects (pre-post); 2 groups (AUTO, USER-CONTROL)	Healthy older adults	51	69.9	48	Home	Tablet	3 weeks	None	Participant’s choice	Participant’s choice	WM Training Game
Game elements improve performance in a working memory training task	Ninaus et al. [42]	2015	Journal, Computer Science	Austria	Between-subjects (post); 2 groups (NOGAME, GAME)	University students	30	23.8	80	Online	PC	25 min	None	1	25 min	GAME
Game-based training of flexibility and attention improves task-switch performance: near and far transfer of cognitive training in an EEG study	Olfers and Band [89]	2017	Journal, Psychology	Netherlands	Between-subjects (pre-post); 3 groups (flexibility, attention, control/active)	Healthy adults	72	23	56	Online	PC	4 weeks	None	20 (15 minimum)	45 min	Lumosity
The Efficacy, Feasibility And Acceptability Of A Remotely Accessible Use Of CIRCuiTS, A Computerized Cognitive Remediation Therapy Program For Schizophrenia: A Pilot Study	Palumbo et al. [90]	2019	Journal, Psychiatry	Italy	Single group	Schizophrenia patients	8	36.37	27.5	Home and Hospital	PC	3 months	None	40	60 min	Computerized Interactive Remediation of Cognition—Training for Schizophrenia (CIRCuiTS)
A Study on the Validity of a Computer-Based Game to Assess Cognitive Processes, Reward Mechanisms, and Time Perception in Children Aged 4–8 Years	Peijnenborgh et al. [91]	2016	Journal, Computer Science	Netherlands	Between-subjects (post); 2 groups (ND, ADHD)	Normal development and ADHD children	136	6.38	40.1	Laboratory (school)	PC	Single session	None	1	20 min	Timo’s Adventure
Racing dragons and remembering aliens: Benefits of playing number and working memory games on kindergartners’ numerical knowledge	Ramani et al. [92]	2019	Journal, Psychology	USA	Between-subjects (post); 3 groups (number-based game, WM game, control)	Kindergarteners	148	5.98	52	Laboratory (school)	Tablet	N/A	4 to 6 weeks	16	25 to 30 min	WM training: “Recall Them All”
A video game for the neuropsychological screening of children	Rosetti et al. [93]	2017	Journal, Computer Science	Mexico	Single group	Students	75	8.5	49	School	PC	Single session	None	1	20 to 40 min	Towi video game
Computer-Based Training in Math and Working Memory Improves Cognitive Skills and Academic Achievement in Primary School Children: Behavioral Results	Sanchez-Perez et al. [94]	2018	Journal, Psychology	Spain	Between-subjects (post); 2 groups (NOGAME, GAME)	Students	157	9.17	45.78	School	PC	13 weeks	1 week	26	30 min	WM Training Game
Cognitive Training Using a Novel Memory Game on an iPad in Patients with Amnestic Mild Cognitive Impairment (aMCI)	Savulich et al. [95]	2017	Journal, Psychiatry	UK	Between-subjects (pre-post); 2 groups (Game Show, clinic visits as usual)	Patients with amnestic MCI	42	76.05	40	NS	iPad	4 weeks	None	N/A	60 min	Game Show
Development of and Adherence to a Computer-Based Gamified Environment Designed to Promote Health and Wellbeing in Older People with Mild Cognitive Impairment	Scase et al. [96]	2017	Conference Paper, Computer Science	UK	Between-subjects (post); 2 groups (retirement village, living separately)	Older adults with MCI	24	75,13	92	Home	Tablet	47 days	None	2 to 59	29 min on average	Find it, match it, solve it, complete it
Evidence for Narrow Transfer after Short-Term Cognitive Training in Older Adults	Souders et al. [97]	2017	Journal, Psychology	USA	Between-subjects (pre-post); 2 groups (Mind Frontiers, active control)	Older adults	60	72.25	57	Home	Tablet	1 month	None	30	45 min	Mind Frontiers
A New App for At-Home Cognitive Training: Description and Pilot Testing on Patients with Multiple Sclerosis	Tacchino et al. [98]	2015	Journal, Computer Science	Italy	Single group	Cognitive-impaired patients with MS	16	49.06	81.25	Home	Tablet	8 weeks	None	40	30 min	Cognitive Training Kit (COGNI-TRAcK)
A New App for At-Home Cognitive Training: Description and Pilot Testing on Patients with Multiple Sclerosis	Tacchino et al. [99]	2020	Journal, Computer Science	Italy	Single group	Cognitive-impaired patients with MS	15	52.6	66	Laboratory	Tablet	8 weeks	None	20	45 to 60 min	CMI-APP
The Effects of Computerized Cognitive Training With and Without Physical Exercise on Cognitive Function in Older Adults: An 8-Week Randomized Controlled Trial	Ten Brinke et al. [100]	2019	Journal, Medicine	Canada	Between-subjects (pre-post); 3 groups (BAT, FBT, FBT + exercise)	Older adults	41	72.88	73	Home	Tablet	6 weeks	None	18	60 min	Fit Brains
Measuring the Impacts of Virtual Reality Games on Cognitive Ability Using EEG Signals and Game Performance Data	Wan et al. [101]	2020	Journal, Computer Science	China	Between-subjects (pre-post); 2 groups (3D, VR)	Healthy adults	20	22.85	30	Laboratory	PC, VR set	1 week	1 month	N/A	1.5 to 3 min	Simon game and Merry Snowballs game
Game-Based Auxiliary Training System for improving visual perceptual dysfunction in children with developmental disabilities: A proposed design and evaluation	Wuang et al. [102]	2018	Journal, Computer Science	Taiwan	Between-subject (pre-post); 2 groups (GBATS, Conventional Visual Perceptual Training Program)	Children with visual–perceptual dysfunction/delay	60	7.51	46	Laboratory	Tablet	8 weeks	None	16	30 min	Game-Based Auxiliary Training System (GBATS)
The malleability of executive function in early childhood: effects of schooling and targeted training	Zhang et al. [103]	2018	Journal, Psychology	China	Between-subjects (pre-post); 4 groups (SG, WMT, ICT and GC)	Primary school and kindergarteners	91	6.12	50	School	PC	4 weeks	3 months	20	15 min	WM Training Game
A Newly Designed Mobile-Based Computerized Cognitive Addiction Therapy App for the Improvement of Cognition Impairments and Risk Decision Making in Methamphetamine Use Disorder: Randomized Controlled Trial	Zhu et al. [104]	2018	Journal, Computer Science	China	Between-subjects (pre-post); 2 groups (CCAT, control)	Adults with methamphetamine use disorder	40	34.2	0	Laboratory	Tablet	4 weeks	None	20	60 min	CCAT app

and

Table A2. Game elements used in the studies of the systematic review.

Author	Narrative/Storytelling	Avatar	Conflict	Cooperation/Competition	Action Points	Progression	Levels	Feedback	Difficulty Adjustment
Ackermann et al. [62]	0	0	1	0	N/A	1	1	1	1
Areces et al. [63]	1	0	1	0	N/A	0	0	1	0
Armado et al. [64]	0	1	0	1	N/A	0	0	0	0
Ballesteros et al. [65]	0	0	1	0	N/A	1	1	1	1
Baniqued et al. [66]	1	0	1	0	1	1	1	1	1
Biel et al. [67]	0	0	1	0	0	0	0	1	0
Bikic et al. [68]	0	0	1	0	0	0	1	1	1
Boendermaker et al. [69]	0	0	1	0	1	1	1	1	1
Boletsis and McCallum [70]	0	0	1	0	1	0	1	1	0
Boot et al. [71]	1	0	1	0	1	1	1	1	1
Cujzek et al. [72]	0	0	1	0	1	1	0	1	0
Dassen et al. [73]	1	0	1	0	1	0	0	0	1
De Giglo et al. [74]	0	0	1	0	1	1	0	1	0
De Vries et al. [75]	1	1	1	0	1	1	1	1	1
Dörrenbächer and Kray [76]	1	N/A	1	1	1	1	1	1	1
Double and Birney [77]	0	0	0	0	1	1	0	0	1
Dovis et al. [78]	1	1	1	0	1	1	1	1	0
Garolera et al. [79]	1	1	1	0	1	1	1	1	1
Goghari et al. [80]	0	0	1	0	1	1	1	1	1
Gray et al. [81]	0	0	1	1	1	1	1	1	1
Hessl et al. [82]	0	0	1	0	1	1	1	1	1
Janssen et al. [83]	0	1	1	0	1	1	0	1	1
Johann et al. [84]	1	N/A	1	0	1	1	1	1	1
Leung et al. [85]	1	0	1	0	N/A	1	1	1	1
Mohammed et al. [86]	1	0	1	0	1	1	1	1	1
Nagle et al. [87]	1	0	0	0	0	0	0	1	1
Nagle et al. [88]	1	0	0	0	1	0	0	0	1
Ninaus et al. [42]	1	0	1	0	1	1	1	1	1
Olfers and Band [89]	1	1	1	0	1	0	0	0	1
Palumbo et al. [90]	1	N/A	1	0	N/A	1	1	1	1
Peijnenborgh et al. [91]	1	1	1	0	1	1	1	1	0
Ramani et al. [92]	1	0	1	0	1	1	1	1	1
Rosetti et al. [93]	1	1	1	0	1	0	0	1	0
Sanchez-Perez et al. [94]	1	0	1	0	1	1	1	1	1
Savulich et al. [95]	0	0	1	0	1	0	1	0	1
Scase et al. [96]	1	0	0	0	1	1	0	0	0
Souders et al. [97]	1	0	1	0	1	1	1	1	1
Tacchino et al. [98]	0	0	1	0	N/A	1	1	1	1
Tacchino et al. [99]	0	0	1	0	N/A	1	1	1	1
Ten Brinke et al. [100]	0	0	1	0	1	1	1	1	0
Wan et al. [101]	0	0	1	0	1	1	1	1	0
Wuang et al. [102]	0	0	1	0	1	1	1	1	1
Zhang et al. [103]	0	0	1	0	1	1	1	1	1
Zhu et al. [104]	0	0	1	0	0	1	1	1	1

in Appendix A). Most of them were published in 2015 and 2018 (N = 11), followed by 2017 in which 8 studies conducted, and then by 2016 and 2019, which had the same number of studies (N = 6), and only 2 were in 2020. Around 63% (N = 28) took place in Europe, following North and South America with 27.7% (N = 10). A total of 5 studies were conducted in Asia, and 1 was conducted in Australia. Finally, the majority of studies were published in journals (only 1 study was a conference paper), scoping mostly in psychology (45.5%; N = 20) and medical–psychiatric journals (around 20%; N = 9), and the remaining were related to computer science (N = 15, 35%). Most studies used a between-groups design with pre–post or only post-measures (N = 36), including two (N = 27), three (N = 8) and four (N = 1) groups. On the other side, 8 studies were single-group designs.

A total number of 3496 participants were included in the studies, with sample sizes ranging from 5 [70] to 794 [78] participants. A total of 7 studies included 5 to 19 participants, 13 studies included 20 to 59 participants, 17 cases contained 60–99 participants and only 7 studies had 100 and above participants. Regarding the age of participants, the mean ages included values from 6.12 [103] years old to 82.7 years old [87], although in [79], there was no specific mean value, as the only available information was that participants ranged from 50 to 90 years old. Overall, regarding mean age, the included studies can be characterized as balanced, since 36.36% (N = 16) had children and adolescent participants (<18 years old), 13 studies (29.54%) included adult participants (<60 years old) and 15 studies (34.1%) had participants over 60 years old. Moreover, most of the studies (N = 29, 66%) included females for over 50% of their samples (largest proportion was 81.25% in [98], and on the other hand, the smallest proportion was 9% [75]), and a single studied had only male participants [104]. It should be noted that there was no information for 4 studies [65,70,72,79]. A total of 19 studies included participants suffering from ADHD, Mild Cognitive Impairment (MCI), schizophrenia, visual perceptual dysfunction, multiple sclerosis, fragile X syndrome, autism and overweight (adults), while the rest of studies (25) included healthy children, adults or older people.

Regarding the outcomes that were reported, 19 studies presented data concerning participants’ motivation and enjoyment in addition to the training domain outcomes, which were included in the majority of studies (N = 38), and only 6 studies focused exclusively on the effects in motivation and engagement outcomes, excluding any training domain outcomes.

Data regarding the interventions’ duration showed that there were cases with a single session (N = 4), which were cases for the assessment and screening of participants [63,91,93] for 18 months [77]. In general, the majority of studies (N = 25) ranged from 4 to 8 weeks in intervention duration. Thus, the total number of distinct sessions varied from 1–18 (N = 17) and 20 to 40 (N = 22), 2 studies supported the participants’ free will to complete as many sessions as they liked [77,88] and there was missing information for 2 studies [95,101]. Session duration varied from 90 s [101] to 90 min [64]. Despite the fact that all studies presented post-measures and evaluation data, only 25% (N = 11) performed any follow-up evaluation, varying from 1 week [94] to 6 months [73].

Data about the site being used in the studies demonstrated 16 cases (36%) that took place at participants’ houses, 34% (N = 15) studies used laboratory conditions, 6 cases used schools, three were online, 1 study used a hybrid model with house and laboratory choices and another study used an assisted living facility. Regarding the devices being used, computers (PC/Laptop) were the major device, as reported in 52% of studies (N = 23), followed by tablet/smartphones devices (N = 17), and 2 studies used both computers and tablets [73,77]. Only one study used a portable console (Nintendo Switch) [74], and 2 studies used a Virtual Reality set [63,101].

Regarding the GEs being used in the 44 studies, a minimum of 2 elements per study were used [64,77] with a maximum of 8 elements [75,76]. The mean value of GEs used was 5.7 elements. The elements that were used the most were Conflict (88.63%; N = 39), Feedback (84%; N = 37), Progression (75%; N = 33), Difficulty Adjustment (73%; N = 32), Action Points (73%; N = 32) and Levels (70.45%; N = 31). Looking at GEs in more detail, and regarding the sample that was applied, starting with children and adolescents (N = 16; 50% were children with ADHD, autism spectrum disorder and visual perception delay), the most commonly used elements were Conflict and Feedback (N = 16), followed by Levels (N = 14), Action Points and Progression (N = 13 respectively) and Difficulty Adjustment (N = 12). On the other hand, Competition and Cooperation were underused (N = 3). Modality via PCs dominated (N = 13), followed by smartphones/tablets, and variations between study sites was balanced (Home 37.5%, Laboratory 31.25% and School 31.25%). Moving on to studies with adults (N = 13; 54% high-risk participants, suffering from multiple Sclerosis, schizophrenia, MCI and drug addiction), the most prevalent GEs were Conflict (N = 12), Difficulty Adjustment, Progression and Feedback (N = 10) and Action Points (N = 8), and the most underused were Competition and Cooperation (N = 1) and Avatar (N = 3). Again, computers and smartphones/tablets were the most commonly used devices (N = 5 respectively). However, there was one study that offered both devices (computers and tablets) [73], and a study used a portable console [74]. Laboratory interventions seemed to be the case here (N = 7), and houses were the next one (N = 5). There was a study with a hybrid model using a hospital and in-house sessions [90]. Finally, the GEs that were used the most in elderly studies (N = 15; 80% healthy, 20% patients with MCI) were Conflict, Feedback and Action Points (N = 11 respectively), Progression and Difficulty Adjustment (N = 10, respectively) and Narrative Context (N = 7), and Competition and Cooperation as well as Avatars were used less (N = 0, N = 1, N = 1). Interestingly, in the case of the elderly, tablets/smartphones dominated the devices (N = 10), with only 4 studies using computers (N = 4), and a single study offered both options [77]. Home interventions were the majority of studies (N = 9), and there were only 3 laboratory interventions. Moreover, there was a study that was conducted in an assisted living facility [87].

3.3. Risk of Bias

The risk of bias assessment for each included study is depicted in Figure 2, which indicates that, overall, the quality of the included studies was optimal. However, the most significant risk of bias lies in the randomization process and in missing outcome data. On the other hand, a lower risk of bias was observed in the deviations from intended interventions and in the measurement of the outcome.

4. Discussion

The aim of this study was to provide a state-of-the-art literature review in the gamification of cognitive training and especially in the training of Working Memory, in order to provide an updated overview of the existing research and evidence for the GEs being utilized in the training and assessment of Working Memory specifically. Following PRISMA guidelines [60], a systematic literature review was conducted, which identified 44 studies published from 2015 to 2020. The risk of bias assessment, which was conducted according to the Cochrane’s RoB tool, showed optimal quality of the included studies. The only domains that presented a higher risk of bias were the randomization process, for which some studies demonstrated poor or no randomization, and the missing of outcome data, which was caused by some dropouts during the intervention process. During the search process, we also identified several systematic reviews and meta-analyses, such as [49], which presented 33 studies published between 2007 and 2015. Another study identified 49 studies between 2008 and 2017 [55], Ref. [53] included 91 papers from 2006 to 2018 and Ref. [56] identified 24 studies between 2006 to 2018. This study identified 8 more studies after 2018, which were eligible to be included. For the inclusion of GEs, we searched the literature and previous studies, and we decided to include 9 GEs that were the most interesting to examine and that simultaneously had a greater impact on the training process, as far as we were concerned [48,49,50,51,52,53,55]. A major problem that we faced was the absence of any detailed information and descriptions about the gamified tasks/games that were used from the reported studies; thus, it was necessary to check any additional sources of information that were publicly available (web pages, videos, etc.), in order to extract the required information.

Regarding the use of GEs, there was a variety of the selected elements, which were scarce over the included studies. Overall, according to the results, Conflict was the most commonly used game element (N = 39), which, considering the demands of WM (which requires an amount of information to be held for a specific time frame), looks to be ideal and the most significant game element that can be incorporated to gamification tasks. Along with Feedback (N = 37), which may contain a scoring system, right or wrong answers or assessments of performed actions, they are the two most prevalent GEs that were used in the studies. This lies in contrast to the findings provided by [53], and Ref. [49] reported Action Points (as a scoring system, points, etc.) as the most frequently used game element. A possible yet reasonable explanation is that, for 8 studies, there was no information or evidence regarding the presence of GEs acting as Action Points; thus, it is very possible that the total number may be higher than the reported 31. As mentioned before, the second most commonly used game element was Feedback, which is a key element in behavior changes, and it acts as an indication of performance and can be delivered through visual or auditory stimuli. In addition, Progression and Difficulty Adjustment were also used in most of the cases (N = 33 and N = 32, respectively). Participants can highly appreciate the fact that they always have an overview of their progress, as the training tasks can be lengthy in time for completion. Difficulty Adjustment is critical in building personalized training tasks and is a key element for training success. Since participants can vary in measures of cognitive performance, the delivered training should reflect these measures. Extremely easy or difficult tasks may jeopardize adherence and engagement, either by causing boredom or disappointment in cases of uncompleted tasks. On the other side, the most underused tasks were associated with socialization context (Competition and Cooperation) and the use of avatars. Very limited social interaction elements were also reported in the reviews of [53,55], and it seems that this field has not received any more attention, which can be justified as a complex element that requires extensive implementation. Moreover, interacting with others and being exposed may cause negative effects for participants, such as anxiety and frustration, especially in high-risk cases. Finally, avatars were used less compared to the narrative context (N = 22), which was used with the purpose of providing a context to the trained activity and adding meaningful content for the user.

Regarding sample characteristics, studies varied in terms of sample size between 5 [70] and 794 [78]. Nearly one third of studies (29.5%) had 20 to 59 participants, and 38.6% included 60–99 participants. However, only 7 studies had 100 and above participants. It is a fact that gamification has a broad variety of target audiences, as it has been applied to children, adolescents, adults and the elderly. The systematic review revealed a balance between target groups, since 36.36% of the studies targeted children and adolescents, 29.54% included adult participants less than 60 years old and 34.1% had participants over 60 years old. Thus, there is a slight preference to children and older people in studies of gamification. Between samples, most of the studies targeted healthy (low-risk) participants, and 43% of studies targeted several forms of cognitive impairments, such as ADHD, autism spectrum disorder, multiple sclerosis, etc. It seems that gamified WM training was used not only as a therapeutic/rehabilitation tool, but it was also used to support and foster healthy people in order to cognitively function at higher rates or as a preventative tool for cognitive decline. However, gamification cannot be applied in the same way to every target group, since users can vary in training goals, motivation and cognitive level. It is necessary to provide personalized and adaptive training programs according to individual users. For example, the presented context may be suitable for kids, but it might look childish or unrealistic to adults and older people. Nevertheless, the systematic review spotted common patterns in gamification development for the three target groups, as it seems that Conflict, Difficulty Adjustment, Action Points, Feedback and Progression were the four common elements that were frequently used in all three categories.

Gamified WM training has been offered in a variety of devices according to the findings of this review; however, the most dominant device was a computer (PCs or laptops), as reported in 52% of the studies, with tablets/smartphones coming in second (38.6%). On the other hand, gaming consoles and virtual reality equipment were used less. Especially for studies including VR, they were primarily studies which gamification was utilized in evaluation and assessment of cognitive tasks. An interesting finding is associated with older people, as 10 studies used tablet devices, and only 4 cases made use of computers, verifying the fact that tablets and smartphones with touch functionality are more suitable for older adults, which differentiates from the findings in [53], which reported that none of the studies that used older adults as participants reported tablets as game platforms. Therefore, it looks as though progress has been made in this direction.

Intervention sites were split across participants’ homes and laboratories (36% and 34%, respectively), and there was a study that included both sites, having some sessions under laboratory conditions and some at home [90]. It seemed to be the case that gamified WM training can be equally feasible and effective either in laboratory or house conditions, providing the freedom of training everywhere and at any time and ensuring that transportation is not a limit that may hinder adherence. Finally, 63.5% of studies reported over 15 sessions, and some of them had single-session designs (11.5%). Regarding the number of sessions and overall intervention time, there is no clear indication of values that can be more effective in WM training. The majority of studies designed training to last 4 to 8 weeks, and the optimal approach might be somewhere in between. However, in the meta-analyses in [53], evidence reported no significant effect on the number of training sessions on effect sizes for motivation/engagement outcomes. More interestingly, regarding the follow-up assessment, which is important in order to measure how long the training effects last after the intervention, only 25% of the studies reported follow-ups, which reported measures from 1 to 6 months after the training. However, no indication of gamification effects in the follow-up evaluation could be extracted since there was no evidence, as researchers focused mostly on the evaluation of WM measures.

5. Limitations

This study was conducted under certain limitations, and its findings should be interpreted in a way that takes into account these limitations. First of all, there is the language limitation (included studies only in the English language), which might have led to possibly excluding studies that were written in a language other than English, and in that way, relevant published articles might have been unintentionally excluded from the current review.

Secondly, it is important to mention that the followed procedure and search strategy, although it used a substantial number of keywords and many of the most known electronic libraries, may have failed to identify any relevant studies that may have not met the search criteria. Moreover, relevant articles published in conferences may have been missed by accident, especially if, by the time of search procedure, they were not available online. However, the search strategy can be considered adequate, since it included seven different electronic libraries, and this should have minimized the danger of missing relevant studies. In addition, we used reference lists from the included studies and any other systematic literature reviews that were also found during the search process, strengthening the search results.

Third, there was missing information especially in the descriptions of the games (or GEs) which were used in the included studies, and any omitted details regarding the computer-based interventions overall, was another significant limitation. Although there was a supplementary search in every available source online in order to minimize the risk of missing data, sometimes it was impossible to define the included GEs with clarity. Thus, there were cases with subjective judgment in the data extraction if there was any strong supported evidence, and otherwise, the data were considered not available and were documented as such.

Fourth, the primary objective of this study was to identify the most common GEs that have been used in WM training or assessment, having a measured impact in WM enhancement and/or in the engagement and enjoyment of users. Most of the included studies presented WM outcomes exclusively, and less than half (N = 21) reported engagement and enjoyment outcomes in addition. Four contained only engagement metrics without WM outcomes, which are critical to the current research. Unfortunately, the primary focus for the majority of studies was to measure the effects of gamification in WM outcomes; thus, it is important that future studies focus more on measuring engagement, adherence and enjoyment outcomes.

Finally, the included studies did not attempt to examine the efficacy of individual GEs or to conduct any comparison among them, as they usually focused on gamification as a whole, combining more than one GE each time and making it difficult to distinguish any effects of individual elements as outcomes.

6. Conclusions

The purpose of this systematic review was to provide an update on gamified computer-based WM training, especially on the GEs that have been used in research studies, in order to examine which elements are the most preferable and which ones are widely used. The results show that little has changed in the most common GEs, as Conflict, Feedback, Progression and Difficulty Adjustment are the ones that dominated in the majority of the included studies. On the other hand, GEs that promote social interaction and cooperation or competition were significantly underused. An important issue that can be observed in the vast majority of studies is the lack of a theoretical framework or any theory that can justify the selection of the included GEs in each study. In almost every case, the GEs that were used were arbitrarily selected, or they were not based on a design or learning theory, which is something that has been observed in previous reviews. Future studies should base their design and justify their selection regarding GEs in order to document and clarify why and how the gamified tasks were developed in a certain way. Moreover, future research must focus on examining the impact of GEs and the effect that each one of them demonstrates by making comparisons between GEs themselves. This is of great importance, as it can answer the questions of choosing certain GEs among others and can potentially provide guidelines for future developers of cognitive training. In addition, for instance, it can lead to the strategic selection of GEs according to the health status of the user, age or any other demographic characteristic. Unfortunately, our research does not demonstrate any significant differences between the GEs that were used among the different audiences, as it seems that the selected GEs are common, regardless of the age or any health/pathological conditions of the participants. In the future, we plan to conduct a meta-analysis, which will be based on the current systematic review, as our main focus is to attempt to define a framework for building cognitive training interventions for WM, and the current research provides the potential to form and to answer useful research questions in this direction.