Meta-Analysis on the Effectiveness of Virtual Reality Cognitive Training (VRCT) and Computer-Based Cognitive Training (CBCT) for Individuals with Mild Cognitive Impairment (MCI)

Abstract

This meta-analysis aims to assess the effectiveness of virtual reality cognitive training (VRCT) and conventional computer-based cognitive training (CBCT) in five specific cognitive domains (i.e., global cognitive function (GCF), memory (Mem), executive function (EF), language (Lang) and visuospatial skills (VS)) of individuals with mild cognitive impairment. A total of 320 studies were yielded from five electronic databases. Eighteen randomized controlled trials met the PRISMA criteria, with 10 related to VRCT and 8 related to CBCT. A random-effect model was used in determining the main effect of cognitive training in five specific cognitive domains. VRCT provided the largest effect size on VS and Lang while the smallest on EF. CBCT provided the largest effect size on Mem and Lang while the smallest on EF. VRCT and CBCT generate an opposite effect on VS. VRCT outweighs CBCT in treatment effectiveness of GCF, EF, Lang and VS. More immersive and interactive experiences in VRCT may help individuals with MCI better engage in real-life experiences, which supports skill generalization and reduces external distractions. CBCT tends to improve Mem but no definite conclusions can be made. Further investigation with more stringent research design and specific protocol are required to reach consensus about the optimum intervention regime.

1. Introduction

According to the World Health Organization, approximately 50 million people are affected by dementia globally, and the number is projected to triple to 131 million in 2050 [1,2]. Mild Cognitive Impairment (MCI) is characterized by slight but noticeable deficits in attention, learning and memory, executive function, processing speed, and semantic language [3,4]. If cognitive intervention is absent, individuals with MCI may not maintain current cognitive abilities and spiral down into full-fledged demented conditions. Therefore, it is important to identify effective cognitive trainings [5].

Technological advancement and enhanced education level of aging population demand more effective and evidence-based cognitive trainings. Besides conventional computer-based cognitive training (CBCT), immersive virtual reality cognitive training (VRCT) has been gaining popularity over the last decade [6]. Previous studies have suggested that implementation of CBCT and VRCT among individuals with MCI may lower the risk for developing Alzheimer’s disease and related dementia [5,7]. VRCT and CBCT utilize distinct but similar approaches to provide immediate performance feedback, which is necessary for effective learning and rehabilitation [8].

VRCT differs from CBCT in terms of enhanced ecological validity, as high similarity between training environment and real world helps predict everyday functioning [8]. Moreover, VR is an advanced human–computer interaction and provides a realistic 3D environment to elicit individuals’ perceptions and reactions [7,9]. VRCT also offers a cost-effective, accessible, and flexible intervention for individuals suffering from disability, distance, and transportation problems [7,8].

While some reviews indicated positive effects of VRCT and CBCT on cognitive domains such as global cognitive function (GCF), memory (Mem), executive function (EF), language (Lang) and visuospatial skills (VS) [5,7,8,9,10], some had opposite conclusions and questioned their effectiveness on EF [11,12]. Moreover, there is limited information about effective intervention regime due to various training institutes, aims and objectives in different cognitive domains, of which their respective definitions are provided below.

GCF refers to mental processes in the acquisition of information and constitutes domains such as attention and decision making [13]; Mem involves the encoding, retaining and retrieval of information [14]. Deficits in Mem could lead to difficulties in learning and utilizing information in everyday tasks [14,15]; higher order EF (e.g., reasoning, problem solving) requires simultaneous use of multiple basic EFs (e.g., attention, inhibitory control), which are essential for goal-directed behaviors [16]. EF can be further categorized into five sub-domains, including attention, mental flexibility, inhibitory control, working Mem [17]; individuals with MCI are characterized by fluency and naming confrontation disorders as Lang is related to orthographic or phonological retrieval of semantic Mem and verbal production processes [18]; vs. refers to cognitive processes which identify, integrate, and analyze visual form, details, and spatial relations in multi-dimensions, and are essential for movement, depth perception, and spatial orientation [19].

The overall aim of this study is to assess the effectiveness on VRCT and CBCT in five cognitive domains, namely GCF, Mem, EF, vs. and Lang, for individuals with MCI. The specific objectives were to: (a) identify advantages and limitations of VRCT and CBCT; (b) observe and report the trend of cognitive trainings; and (c) suggest future research directions.

2. Materials and Methods

2.1. Search Strategy

A systematic search of relevant studies was conducted in April 2020 of published studies available on EMBASE, Virtual Health Library Search Portal databases (BVS), PsycInfo (Proquest), PubMed and MEDLINE (via EbscoHost). The search strategy combined keywords for three main concepts: virtual reality, cognitive training, and MCI.

The key search term (“virtual reality or VR”) was supplemented with (“technology-based training”) (“computer-based training”). The term (“cognitive training”) was supplemented with (“cognitive rehabilitation”) (“cognitive stimulation”) (“Memory training”). The population of interest was individuals with MCI aged 65 or above, and, to identify this group, multiple search terms were entered (“MCI” or “mild cognitive impairment”) (“questionable dementia”) (“mild Neurocognitive disorder” or “mild NCD”) (“memory impairment”) (Supplementary Materials S1).

2.2. Inclusion Criteria

Studies were selected from the initial search if they met the following criteria: (1) a VRCT or CBCT was evaluated; (2) individuals aged 65 or above having MCI or in a mixed data in which MCI individuals were available separately; and (3) study design was a randomized control trial (RCT); (4) published in a peer-reviewed English journal; (5) the result of at least one cognitive outcome measure is provided.

2.3. Exclusion Criteria

Studies were excluded if: (1) the effect of intervention on MCI individuals could not be extracted from effects among healthy or dementia individuals, (2) the publication was not in English, and (3) the study was published before 2009.

2.4. Appraisal of Study Quality

The review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (Supplementary Materials S2). The use of PRISMA checklists improved the reporting quality and provided substantial transparency in the article selection process. Physiotherapy Evidence Database (PEDro) scale was used to assess methodological quality within studies (Supplementary Materials S3). The final score was settled when all authors reached an agreement after repeated review and analysis.

2.5. Analytical Approach

Relevant data from methodology and results of the selected studies were extracted to calculate effect sizes, statistical power and clinical relevance. Important characteristics such as cohorts, intervention type, intervention regimes, and cognitive outcome measures were also analyzed. Based on the five cognitive domains, Cohen’s d was calculated by pooling standard deviations across pre-test and post-test measurements for intervention and control groups. Thus, Cohen’s d was obtained with 95% confidence intervals using the Practical Meta-Analysis Effect Size Calculator (retrieved from https://campbellcollaboration.org/research-resources/effect-size-calculator.html). Post-hoc power calculations were calculated with GPower Analysis Version 3.1 [20]. A random-effect model was used in determining the main effect of cognitive training in five specific cognitive domains. Forest plots of subgroup analysis were thus generated using Meta-Essentials Version 1.5 [21].

3. Results

3.1. Search Results

The initial search strategy yielded 320 studies (including duplicates) from five databases. By using PRISMA guidelines, eighteen studies met the criteria, with ten studies related to VRCT and eight studies related to CBCT. A summary of the decision pathway for the final inclusion of eligible studies into the review was presented in Supplementary Materials S4.

3.2. Data Extraction and Analysis

The eighteen studies were analyzed with cohort and individual characteristics (year of publication, country of origin, MCI sample size, MCI criteria, screening assessment, age and gender) (Supplementary Materials S5), intervention characteristics (technology-used: VRCT or CBCT, type of interaction technique, intervention regime) (Supplementary Materials S6), and effect sizes (Supplementary Materials S7a, S7b), respectively.

Major features of cohort and individual characteristics, compliance and attrition factors, and intervention outcomes are reported below.

3.3. Cohort Characteristics

Country of Origin

Studies were conducted in different countries: South Korea (n = 5), United States (n = 3), Italy (n = 2), Taiwan (n = 2), Australia (n = 1), France (n = 2), Greece (n = 1), Hong Kong (n = 1) and Norway (n = 1).

3.4. Individuals Characteristics

Age and Gender

The mean age of eighteen studies was 75.63 years old and age range was from 43 to 91 years old. Specifically, from the study done by Flak et al. (2019), age range was from 43 to 88 years old, with a mean age of 66 years old. Since our study interest is on individuals with MCI aged 65 or above, the study by Flak et al. (2019) was included in this review as the mean age was above the required age [22]. However, particular attention should be paid when analyzing this study.

The total number of the individuals with MCI included in this review is 733, including 300 males and 433 females. However, two studies reported inequality in gender ratios between treatment and control groups at baseline [23,24].

3.5. Compliance and Attrition Factors

The overall attrition rate of both VRCT and CBCT was satisfactory (all less than 20%), except the CBCT study done by Finn and McDonald (2012) recorded a 33% attrition rate. The attrition rate ranged from 0% [25] to 33% [15], though dropout rate was not specified in some studies [22,23,26,27,28,29,30]. The primary reason for dropouts was mainly independent of the intervention, such as unrelated medical or personal issues [15,31,32,33,34]. Other reasons include time commitment, negative training experiences, low motivation, technical problems and tenancy changeover [31,32,35].

3.6. Intervention Outcomes

This meta-analysis assesses the effectiveness of five specific cognitive domains (i.e., GCF, Mem, EF, Lang and VS) for individuals with MCI with application of VRCT and CBCT. These five cognitive domains, which are also areas that occupational therapists focus on, were highlighted as they are commonly targeted during cognitive training for functional enhancement. Cognitive measurements used and effect sizes of each cognitive domain were reported in this section.

3.7. Correlation between the Five Cognitive Domains

The five cognitive domains are inter-correlated as evidenced by different research. GCF is most commonly assessed to provide an overview of cognitive abilities, while the Mem component is also heavily focused on as it brings about prominent inconvenience in daily living [36]. Mem largely correlates with EF through improving attention and memorizing stimuli more effectively, as the generalization of training effect often occurs due to neuroplasticity [28,37,38]. Moreover, Lang correlates with Mem to a large extent that Mem retrieval is closely linked with word finding [39,40]. VS. is closely linked to EF as working Mem and resistance to interference are used in maintaining visuospatial information [41].

The effectiveness of CBCT and VRCT on these cognitive domains would be evaluated below.

3.8. Comparison between CBCT and VRCT

Cognitive Measurements Used

The eighteen studies reviewed adopted a wide variety of cognitive measurements in different cognitive domains. Cognitive measurements used in each of the cognitive domains are listed in

Table 1. Cognitive measurements used.

Cognitive Domains	Cognitive Measurements Used
	CBCT	VRCT
GCF	RBANS total score (5 RBANS index score), Dementia Rating Scale-2 (DRS-2), Mini-Mental State Examination (MMSE), Cambridge Neuropsychological Test Automated Battery (CANTAB).	Montreal Cognitive Assessment (MoCA), Mental Status in Neurology, Mini-Mental State Examination (MMSE), Korean Version of Mini Mental Status Examination (MMSE-K), Mini-Mental State Examination for Dementia Screening (MMSE-DS).
Mem	Immediate and delayed recall: Repeatable Battery for Assessment of Cognitive Status (RBANS), California Verbal Learning Test (CVLT-II), Word-List Mem Test (WLMT), Word List Recall Test (WLRT), Word List Recognition Test (WLRcT), Immediate word recall, Delayed word Mem test. Mem in everyday living: The Measurement of Everyday Cognition (ECOG), CANTAB- MFQ (Mem Functioning Questionnaire), Subjective Mem Complaints Questionnaire (SMCQ). Visual Mem: CANTAB- PAL (Paired Associates Learning), CANTAB- PRM (Pattern recognition Mem), Rey Complex Figure Test and Recognition Trial (RCFT), Wechsler Mem Scale WMS-III Faces I and II, visual recognition subtest from the Doors and People Mem battery, Delayed Matching-to-Sample Task 48 (DMS48). Verbal Mem: Logical Mem II Delayed recall, CVLT-II Long delay free recall, CVLT Total hits, Digit Span Test, 12-word-list recall from BEM-144 Mem battery, the 16-item free and cued reminding test (16-FR/CR test), sub-score recall of the MMSE.	Immediate and delayed recall: Fuld Object Mem Evaluation (FOME), the Word List Recall Test, the Word List Recognition Test. Mem in everyday living: Multifactorial Mem Questionnaire (MMQ). Visual Mem: Visual Span Test (VST). Verbal Mem: Chinese version of the Verbal Learning Test (CVVLT), The Rey Auditory Verbal Learning Test (RAVLT), Digit Span Test, Verbal Story Recall Test (VSR), Word List Learning Test.
EF	Attention: Repeatable Battery for Assessment of Neuropsychological Status (RBANS) attention subtest, Rapid visual information processing (RVP), Digit Span Forward (DSF), Digit Span Backward (DSB). Mental flexibility: Set shifting and flanker tasks from Executive Abilities: Measures and Instruments for Neurobehavioral Evaluation and Research (EXAMINER), Intra-/extra-dimensional set shifting (IED), Trail Making Test (TMT-A and TMT-B), Design fluency test. Inhibitory control: Delis–Kaplan EF System (D-KEFS) Working Mem: Digit Span Forward (DSF), Digit Span Backward (DSB) test, Spatial span test, Wechsler Mem Scale- Letter Number Sequencing, Dot counting and 1-back. Speed of processing: Trail Making Test (TMT-A and TMT-B).	Attention: Word Color Test (WCT), Digit Span Forward (DSF), Digit Span Backward (DSB) test. Mental flexibility: Trail making test (TMT-A, TMT-B), Dual Task Performance (DTP) test, Letter fluency test. Inhibitory control: Stroop test, Symbol Digit Modalities Test, The Stroop Color and Word Test (SCWT), Stroop interference test. Working Mem: Cognitive Estimation Test (CET), Digit Span Forward (DSF), Digit Span Backward (DSB) test. Speed of processing: Trail making test (TMT-A, TMT-B) test. Overall EF: Frontal Assessment Battery (FAB), The Executive Interview 25 (EXIT-25).
Lang	Verbal fluency: Controlled Oral Word Association Test (COWAT), Semantic Fluency (SF), phonemic and categorical task from Executive Abilities: Measures and Instruments for Neurobehavioral Evaluation and Research (EXAMINER), semantic fluency task from Repeatable Battery for the Assessment of Neuropsychological Status (RBANS). Naming: The Boston Naming Test (BNT), picture naming task from Repeatable Battery for the Assessment of Neuropsychological Status (RBANS).	Verbal fluency: Verbal Fluency Test (VF), Word Fluency Test (category and letter fluency), Category fluency tasks. Naming: 15-items short-form of the Boston Naming Test, the Korean version of Boston Naming Test.
VS	Repeatable Battery for Assessment of Cognitive Status (RBANS) visuospatial subtest, Clock Drawing Test, Rey–Osterrieth Complex Figure.	Rey-Osterrieth Complex Figure Test (ROCFT), Clock Drawing Test, the Attentional Matrices Test (AM), the Trail Making Test A (TMT-A), Constructional Praxis Test, Constructional Recall Test.

.

3.9. Intervention Results

For CBCT, a total of three studies assessed GCF, seven studies assessed Mem, five studies assessed EF, three studies assessed Lang and three studies assessed VS. For VRCT, a total of six studies assessed GCF, seven studies assessed Mem, nine studies assessed EF, three studies assessed Lang and and studies assessed VS.

The results of the five specific cognitive domains are shown in

Table 2. Results of the five specific cognitive domains.

Cognitive Domains	Citation	Sub-Domains	CBCT/VRCT	Cognitive Measurement Used	Statistically Significant Improvement (p < 0.05)	Effect Size	Remarks/Comments
GCF	Barnes et al., 2009	N.A.	CBCT	RBANS total score	Yes	d = 0.33
	Chandler et al., 2017	N.A.	CBCT	DRS-2	Yes	N.A.
	Han et al., 2017	N.A.	CBCT	MMSE	No	d = 0.19
	Liao et al., 2020	N.A.	VRCT	MoCA	Yes	d = 0.12
	Mrakic-Sposta et al., 2018	N.A.	VRCT	MMSE	Yes	d = 0.29
	Optale et al., 2010	N.A.	VRCT	MMSE	Yes	d = 1.40
		N.A.	VRCT	Mental Status of Neurology	Yes	d = 0.51
	Park et al., 2020	N.A.	VRCT	K-MMSE	No	d = −0.43	K-MMSE scores did not change significantly in either groups after 12 weeks
	Tarnanas et al., 2014	N.A.	VRCT	MMSE	No	d = 0.47
	Thapa et al., 2020	N.A.	VRCT	MMSE-DS	No	d = 0.21
Mem	Barnes et al., 2009	Immediate and delayed recall	CBCT	RBANS delayed Mem	Yes	d = 0.53
	Chandler et al., 2017	Mem Activities of Daily Living	CBCT	ECOG	No	d = 0.54
	Finn and McDonald, 2012	Overall Mem	CBCT	PAL errors	No	d = −1.15
	Flak et al., 2019	Visual Mem	CBCT	RCFT Delayed Recall	No	d = 0.29	The training group tended to perform better on both learning (encoding) and delayed recall (retrieval) on a word list task.
		Verbal Mem	CBCT	CVLT-II Long delay free recall	No	d = 0.20	The training group tended to perform better on both learning (encoding) and delayed recall (retrieval) on a word list task.
	Herrera et al., 2012	Immediate and delayed recall	CBCT	BEM-14412-word-list recall test	Yes	d = 1.45	The training group improved at both post-test and 6-month follow up, indicating training improved recall.
	Man et al., 2012	Immediate and delayed recall	VRCT	FOME score in total retrieval	Yes	d = 1.25	Improvement in immediate recall of episodic Mem.
		Mem in everyday living	VRCT	MMQ-strategy	Yes	d = 0.70	There was significant impact in subjective perception of Mem function and use of Mem strategies.
	Hwang and Lee, 2017	Visual Mem	VRCT	Visual span	Yes	d = 0.67
	Optale et al., 2010	Verbal Mem	VRCT	VSR	Yes	d = 1.12
	Tarnanas et al., 2014	Verbal Mem	VRCT	RAVLT	Yes	d = 0.133
	Liao et al., 2020	Verbal Mem Immediate and delayed recall	VRCT	CVVLT	Yes	Immediate recall: d = 0.21 Delayed recall: d = 0.19
	Mrakic-Sposta et al., 2018	Verbal Mem	VRCT	RAVLT_I	No	d = 0.61	The training group showed a greater improvement than control group in the Mem functions and the study was limited by relatively small number of individuals, i.e., 10 in total.
	Park et al., 2019	Immediate and delayed recall	VRCT	WLRT	No	d = 0.06
EF	Finn and McDonald, 2012	Visual sustained attention	CBCT	RVP A mean scores	Yes	d = 1.12	There is a combined effect of gains in the treatment group following training, and a decline in the waitlist group’s RVP A scores, leading to a significant effect on visual sustained attention.
	Lin et al., 2016	Processing speed	CBCT	CEN connectivity	Yes	d = 0.12
		Working Mem	CBCT	Dot counting and 1-back	Yes	d = 0.28
	Nousia et al. 2019	Mental flexibility	CBCT	TMT-A	Yes	d = 0.54	Even though EF scores were still low before and after the training, the training group improved significantly when compared to the control group.
	Barnes et al., 2009	Attention	CBCT	RBANS attention	No	d = 0.04	There was no consistent pattern for measures of attention or EF.
	Flak et al., 2019	Overall EF	CBCT	D-KEFS	No	d = 0.21	No significant differences in training effects on composite scores of EF.
	Hwang and Lee, 2017	Attention	VRCT	WCT	Yes	d = 0.05
	Liao et al., 2019	Inhibitory control	VRCT	SCWT	Yes	d = 0.01
		Mental flexibility	VRCT	TMT-A	Yes	d = 0.07
	Optale et al., 2010	Mental flexibility	VRCT	DTP	Yes	d = 1.08
	Tarnanas et al., 2014	Mental flexibility	VRCT	TMT-B	Yes	d = 2.11	Significantly improved at the 5-month follow-up.
	Thapa et al., 2020	Mental flexibility	VRCT	TMT-B	Yes	d = −0.48	Both groups showed improved EF but significant within group effects were only noted for the VR group.
	Liao et al., 2020	Overall EF	VRCT	EXIT-25	Yes	d = 0.65
	Mrakic-Sposta et al., 2018	Overall EF	VRCT	FAB	No	d = 0.42	Weak effect of treatment on EF was observed.
	Park et al., 2019	Mental flexibility	VRCT	TMT-B	No	d = 0.42
	Park et al., 2020	Inhibitory control	VRCT	Stroop test	No	d = −0.45
Lang	Nousia et al., 2019	Naming	CBCT	Boston Naming Test	Yes	d = 0.93
		Verbal fluency	CBCT	Semantic Fluency	Yes	d = 0.88
	Barnes et al., 2009	Verbal fluency	CBCT	Verbal fluency	No	d = −0.22	The effect size tended to favor control conditions, which included listening to books.
		Naming	CBCT	Boston Naming Test	No	d = −0.23	The effect size tended to favor control conditions, which included listening to books.
	Lin et al., 2016	Verbal fluency	CBCT	Verbal fluency	No	d = 0.45
	Mrakic-Sposta et al., 2018	Verbal fluency	VRCT	Verbal fluency	No	d = 0.23	Intervention group showed a greater improvement in Lang measures than the control group.
	Tarnanas et al., 2014	Verbal fluency	VRCT	Category fluency	No	d = 0.04
VS	Barnes et. al, 2009	N.A.	CBCT	RBANS visuospatial	Yes	d = −0.51
	Nousia et al., 2019	N.A.	CBCT	Clock drawing test	Yes	d = −1.58	CDT requires constructional praxis abilities. The ability may not be tested adequately for possible improvement in spatial orientation.
	Herrera et al., 2012	N.A.	CBCT	ROCFT	No	d = 0.06
	Tarnanas et al., 2014	N.A.	VRCT	ROCFT copy and immediate recall	Yes	d = 0.58	The immediate recall in the ROCFT was improved significantly at the 5-month follow-up.
	Park et al., 2019	N.A.	VRCT	Constructional Recall Test	Yes	d = 0.94
	Optale et al., 2010	N.A.	VRCT	Clock drawing test	No	d = 0.83
	Mrakic-Sposta et al., 2018	N.A.	VRCT	ROCFT	No	d = 0.19	Weak effect of the treatment on visuo-constructive abilities was observed.

.

3.10. Overall Analysis

Heterogeneity has been identified in studies, so a random-effect model was applied. Subgroup analysis for the five cognitive domains was performed per intervention types and cognitive outcomes. This meta-analysis determined the magnitude of effects using standardized mean difference. Relative effect sizes (Cohen’s d) were calculated by pooling standard deviations across pre-test and post-test for intervention and control groups. Thus, Cohen’s d was obtained with 95% confidence intervals with forest plots generated (Figure 1, Figure 2, Figure 3, Figure 4 and Figure 5).

3.11. Effect Sizes According to Different Cognitive Domains

For CBCT, the effect sizes of Mem (d = 0.39, CI = 0.05 to 0.73) and Lang (d = 0.39, CI = −0.24 to 1.02) are the largest while the effect size of EF (d = 0.08, CI = −0.14 to 0.30) is the smallest. This meta-analysis also found no significant effect on EF, which is in line with previous studies [11].

CBCT provides small-to-medium effect sizes on GCF, Mem, EF and Lang (GCF: d = 0.24, CI = 0.11 to 0.37; Mem: d = 0.39, CI = 0.05 to 0.73; EF: d = 0.08, CI = −0.14 to 0.30; Lang: d = 0.39, CI = −0.24 to 1.02), and a negative effect size on vs. (d = −0.70, CI = −1.64 to 0.24) that tended to favor the control group.

For VRCT, the effect sizes of Lang and vs. are the largest (Lang: d = 0.60, CI = 0.35 to 0.86; VS: d = 0.60, CI = 0.40 to 0.80). As is the case with CBCT, the effect size on EF is the smallest (d = 0.16, CI = −0.07 to 0.39). GCF, Mem and EF showed small-to-medium effect sizes, while Lang and vs. showed medium-to-large effect sizes. (GCF: d = 0.30, CI = −0.02 to 0.63; Mem: d = 0.35, CI = 0. 03 to 0.66; EF: d = 0.16, CI = −0.07 to 0.39; Lang: d = 0.60, CI = 0.35 to 0.86; VS: d = 0.60, CI = 0.40 to 0.80).

CBCT and VRCT on vs. generate an extreme and opposite effect (CBCT: d = −0.70, CI = −1.64 to 0.24; VRCT: d = 0.60, CI = 0.40 to 0.80). VRCT provides a medium-to-large effect size on vs., while CBCT provides an effect that favors the control group. However, the overall effect size of CBCT on the vs. calculated was incongruent with results reported in individual studies, which may be due to the heterogeneity in study populations and methodological differences between studies. Thus, careful interpretation of data is required.

Overall, compared with CBCT, VRCT generates a larger effect size in most cognitive domains. A more immersive environment in VRCT may indicate better impact in improving different cognitive domains, while CBCT tends to be more beneficial in improving Mem (CBCT: d = 0.39, CI = 0.05 to 0.73; VRCT: d = 0.35, CI = 0.03 to 0.66). No definite conclusions can be made from the small differences between Cohen’s d.

4. Discussions

The overall aim of this study was to assess the effectiveness of the five cognitive domains (i.e., GCF, Mem, EF, Lang and VS) for individuals with MCI with application of VRCT and CBCT. Factors affecting the effectiveness of VRCT and CBCT will be discussed. As usage of VR in clinical fields has been a new growth area, the trend of cognitive trainings will be reported. Both advantages and limitations of CBCT and VRCT will be discussed and future research directions that may influence the intervention effectiveness will be suggested.

4.1. Effectiveness of CBCT and VRCT

Both VRCT and CBCT showed promising results in improving different cognitive domains for individuals with MCI. According to the forest plots, VRCT appears to be more beneficial in improving the GCF, EF, Lang and VS, while CBCT tends to be more beneficial in improving Mem. However, it may not be possible to illustrate with certainty which cognitive domain benefited the most due to the high variability in intervention nature, different study aims and objectives, intervention regimes, and cognitive outcome measures used.

For Mem (Figure 2), CBCT has a slightly larger effect size than VRCT. The effect sizes ranged considerably due to the different sample sizes and cognitive measurements used, and it may not be possible to state with certainty which types of Mem benefited the most. While effect sizes show a trend that CBCT is slightly more effective, it should be noted that the improvement claimed by Barnes et al. (2009) is not statistically significant (p = 0.07). However, its effect sizes consistently favor the intervention group especially for those scored below mean at baseline [31]. Further studies comparing the effectiveness of VRCT and CBCT on Mem and a research design with closer scrutinization is advocated.

For EF (Figure 3), both CBCT and VRCT show the smallest effect. Due to the complexity and variety of EF, it takes considerable time to train [16,42]. The existing intervention regime may not be enough to improve EF. Optale et al. (2010) revealed that the score of DTP remained stable after initial VRCT but increased after the booster session. This suggests that EF training requires more time and learning consolidation is needed.

The small effect size on EF may be explained by its broad definitions and correlation with other cognitive domains. Firstly, the definition of EF includes subtypes such as mental flexibility, inhibition, speed of processing, attention, etc. [16]. Different studies may focus on solely one or more subtypes. For instance, Hwang and Lee (2017) focus on attention, while Park et al. (2020) focus on inhibitory control. Nevertheless, Mrakic-Sposta et al. (2018) and Liao et al. (2020) examined performances of overall EF. Variability of focus would produce an unclear effect. Secondly, EF is multidimensional, meaning that a robust and clear correlation between EF and Mem is suggested by authors [43]. Thus, the effect may not be obvious when evaluating EF alone.

For EF, VRCT produced a larger effect size than CBCT despite the small difference. This may be explained by the content and immersiveness of VRCT [29,32,33,34]. For example, Liao et al. (2020) and Mrakic-Sposta et al. (2018) adopted visually appealing scenes (e.g., park and garden), while Liao et al. (2019), Liao et al. (2020), Mrakic-Sposta et al. (2018) and Park et al. (2019) incorporated personalized scenes (e.g., home-setting, grocery shopping and taking public transport). Moreover, VR promotes cognition by enabling real-life experiences, new learning opportunities, and engagement [30,35]. VR helps to reenact past activities (e.g., visiting museum, going to firework party) in which individuals with MCI within aged care may not be able to join physically [30,35]. As enjoyment is critical for reaping cognitive benefits, the appealing elements in VRCT may foster participation and boost EF [42].

Some studies adopted a more immersive treatment, which supplemented VR learning with real-time visual and auditory feedback in triggering and improving EF [32,33,34]. Specifically, Mrakic-Sposta et al. (2018) revealed different visual and auditory supports (e.g., shopping list, guiding voice and hints) in the supermarket scenario were judged clear and useful by the individuals, which better improved their EF (e.g., planning and reasoning). Therefore, VRCT with an engaging virtual, dynamic and multisensory environment may be more effective in training EF in individuals with MCI, compared to CBCT.

For vs. (Figure 5), the results of CBCT and VRCT generate an extreme and opposite effect. VRCT provides a medium-to-large effect size while CBCT provides an effect that favors the control group. The difference may be explained by the effect of 3D stimuli and a more immersive environment in VRCT. Its immediate and internalized feedback improve learning efficiency, while higher similarity with the real world further engages individuals [8,29]. For instance, it resembles situations such as ‘riding a bike in the park’, ‘crossing roads, avoiding cars’, ‘grocery shopping in a supermarket’, ‘tasks in an in-home setting scenario’ and ‘visiting the virtual museum’, which further facilitates the transfer of training effect to real-life activities [28,29,35].

4.2. Trend of Rehabilitation

As technology advances, demand for more cost-effective and readily accessible training programs rises. Apart from CBCT, the use of VR in clinical fields has been a relatively new growth area [44]. VRCT addresses the previous concerns about CBCT by allowing more naturalistic and immersive experiences [9]. Simulating real-life situations, VRCT offers great potential in terms of generalization in functional areas, thus enhancing its transferability.

More than two-third (70%) of the studies in this systematic review on VRCT were published from 2017 to 2020, suggesting the growth in importance of VRCT. As this review has previously discussed, VRCT generates a greater effect size in most of the cognitive domains, which may indicate that VRCT is superior to CBCT in certain aspects of training. This may explain the recent shift in research interests from CBCT to VRCT in which an immersive and realistic training modality might prove to be a better alternative [9].

4.3. Advantages of CBCT

The training mode of CBCT is convenient and has lots of possibilities. CBCT enables tailor-made and standardized interventions, real-time monitoring of individual’s cognitive performance, adjustment of intervention levels, and reduced implementation costs, which makes CBCT simple and easy to operate [45,46]. The training mode is flexible to operate either in self-administered form or with therapists’ support. Meanwhile, it enables different training media, such as tablets or computers, providing high portability and flexibility for delivering various tasks.

4.4. Limitations of CBCT

CBCT shows limitations in higher levels of generalization. According to Han et al. (2017), the Ubiquitous Spaced Retrievalbased Memory Advancement and Rehabilitation Training (USMART) program could not improve individuals’ SMCQ and MMSE scores, indicating that efficacy cannot be transferred to general cognitive function, which is the highest level in the hierarchy of generalization. Achieving the lowest level of generalization indicates training efficacy can only be transferred to other non-trained tasks in the same cognitive domain [15,45]. In terms of set-up, regular updates of software packages and touchscreens are required depending on the system, which may increase the possible maintenance fee [47].

4.5. Advantages of VRCT

VRCT provides internalized real-time feedback that improves individual’s motivation, thus increasing adherence to cognitive trainings [27,32,33,34]. Improved motivation is essential in continuing cognitive trainings, which helps maintain and boost training effects in the long run. This concurs with previous studies that suggest that VR was effective in arousing interest and satisfaction in individuals with MCI and is less anxiety-provoking than paper-based tasks [8].

VRCT allows for adjustment in learning pace that reinforces learning efficiency [23,27,34]. Man et al. (2012) revealed that the VR group subjectively perceived better use of Mem strategies due to a self-pacing learning mode and non-threatening training medium [23]. Similarly, Hwang and Lee (2017) revealed that self-selection of training programs may further stimulate the inner drive. Self-paced learning is closely related to self-efficacy as individuals perceive a sense of autonomy in decision making, while a higher self-confidence may positively impact their performance [27,34]. Frequent verbal reinforcement also raises individuals’ satisfaction [23]. This implies that self-paced training with therapist reinforcement may prevent learning pressure and increase engagement, which in turn heightens the training effect and satisfaction.

4.6. Limitations of VRCT

There are possibilities of undesirable effects, such as anxiety in older adults when they first encounter VRCT, which largely depends on the design of the VR system. Tarnanas et al. (2014) suggested that older adults without prior experience with technologies may feel anxious at the beginning, implying that user-friendly technology design and clear instructions may engage individuals and maximize training effects. It is essential to provide stand-by technical support to reduce the likelihood of anxiety and confusion [7,35].

VRCT may bring about side effects though they are uncommon. Park et al. (2020) reported that few individuals experienced dizziness and fatigue after intervention. Previous studies also revealed safety risks for older adults with reduced vision or other sensory impairment during VRCT [7]. Thus, safety measures such as staff supervision, positioning and regular rest breaks need to be given during intervention [7,30].

Some VR programs may involve a 3D-graphic interface set-up and multiple rooms were needed for participants’ navigation to search certain targets [9]. Hence, multiple rooms may be needed for participants to navigate through to search for certain targets. VRCT may require a higher space demand and may not be suitable for crowded settings.

4.7. Comparison between CBCT and VRCT

Advantages of VRCT over CBCT

VRCT brings better training effects in several cognitive domains compared to CBCT due to distinct characteristics of VR. This study further explains how VRCT improved various cognitive domains through repeated presentation of real-world, dynamic, multisensory, and interactive environments.

In terms of skill generalization, VRCT may have an advantage over CBCT [7,9]. As discussed above, CBCT shows limitations in higher levels of generalization while real-life and tailored-made scenarios in VRCT facilitate the transfer of training effect on daily activities [33]. Simulation of real-world situations (e.g., home environment, supermarket, park, etc.) could familiarize individuals with the transfer of training settings [24,28,29]. Moreover, the immersive and interactive experience in VR reduces external distractions, which improves the training efficiency and further facilitates skill generalization.

VRCT provides multisensory stimulation. With the usage of multiple sensory modalities (e.g., supplemented with audio-visual stimuli), VR may benefit the encoding of episodic information, which influences the later retrieval of Mem as multisensory contexts provide an implicit identification to experienced events, thus facilitating subsequent recognition [23,28].

4.8. Common Limitations of CBCT and VRCT

The effectiveness of both CBCT and VRCT is largely affected by the intervention regime, sequencing of training, task difficulty, and types of combination.

For CBCT, a less intensive intervention regime may lead to a lack of generalization since it only stimulates temporary processes (i.e., neurotransmitters upregulation) instead of long term changes (i.e., neurogenesis and formulation of extensive neural networks) [15,31]. The authors also suggested the importance of training sequence in which non-Mem domains should be trained before proceeding to complex Mem training [15].

Similarly, the effectiveness of VRCT is largely determined by intervention regime and types of combination [30]. Park et al. (2020) stressed that insignificant improvement in cognitive domains may be due to a relatively short regime (i.e., 12 weeks). However, VRCT targeting both cognitive and physical training may be more effective [30,32]. This indicates that the benefits of VRCT might be enhanced with a longer training duration and combination with exercise training [30,32]. Further analysis will be discussed in Supplementary Materials S8.

4.9. Compliance and Attrition Factor

The overall attrition rate of the eighteen studies was satisfactory (all less than 20%), except for the study done by Finn and McDonald (2012), which recorded a 33% attrition rate. Regarding the high attrition rate, authors explained primary reasons for drop-out included unrelated medical or personal issues and individuals using their own computers and Internet access at home for CBCT, while training by other studies were conducted in hospital or community settings with the therapist’s support. This is in line with the findings by Lampit et al. (2014) stating that institutionalized CBCT is more effective than CBCT at home [11]. Thus, differences in training institute and availability of therapists’ support may correlate with a higher attrition rate. Further evaluation of the compliance and attrition factors is needed.

4.10. Limitations of the Study

Since current interventions are diverse and variable, further studies should compare the effectiveness of both VRCT and CBCT with less variability of study design to reach definite conclusions. Presently, there is no consensus on an optimal intervention regime for CBCT and VRCT. Further investigation with more stringent research design and specific protocol is required to develop and implement safe, effective cognitive training. Secondly, some studies did not produce a significant effect size due to small sample sizes and the inclusion of pilot tests. Careful elaboration in this review should be advocated and further studies should adopt RCTs with greater sample sizes.

5. Conclusions

This meta-analysis integrated studies about VRCT and CBCT for individuals with MCI. By effect size calculation and comparison, VRCT outweighs CBCT in the treatment effectiveness of GCF, EF, Lang and VS. This can be explained by the fact that more immersive and interactive experiences in VRCT help individuals with MCI to better engage in real-life experiences, which supports skill generalization and reduces external distractions. CBCT tends to be more beneficial in improving Mem but no definite conclusions can be made from small Cohen’s d differences. Although VRCT may be more effective for most cognitive domains, further investigation with more stringent research design and specific protocol is required to reach a consensus on the optimal intervention regime. However, it is certain that the trend for VRCT will grow continuously as it gains more momentum in the clinical field and continues in delivering promising results.