Validation of the Visual Cognitive Assessment Test (VCAT) for the Early Diagnosis of Cognitive Impairment in Multilingual Population in Malaysia

: As Malaysia undergoes a demographic transformation of population aging, the prevalence of dementia is expected to rise, posing a major public health threat issue. Early screening to detect cognitive impairment is important to implement appropriate clinical interventions. The Visual Cognitive Assessment Test (VCAT) is a language-neutral cognitive assessment screening tool suitable for multilingual populations. This study was aimed to validate the VCAT screening tool for the detection of cognitive impairment amongst the population of Malaysia. A total of 184 participants were recruited, comprising 79 cognitively healthy participants (CHP), 46 mild cognitive impairment (MCI) patients, and 59 mild dementia (Alzheimer’s disease and Vascular Dementia) patients from ﬁve hospitals between May 2018 and December 2019 to determine the usefulness of VCAT. Diagnostic performance was assessed using area under the curve (AUC), and receiver operating characteristic (ROC) analysies was performed to determine the recommended cutoff scores. ROC analyses for the VCAT was comparable with that of MoCA (Montreal Cognitive Assessment) in differentiating between CHP, MCI, and mild dementia (AD and VaD) participants. The ﬁndings of this study suggest the following optimal cutoff score for VCAT: Dementia 0–19, MCI 20–23, Normal 24–30. The mean ± SD time to complete the VCAT was 10.0 ± 2.75 min in the CHP group and 15.4 ± 4.52 min in the CI group. Results showed that 76.0% of subjects thought that the instructions in VCAT were similar or easier to understand compared with MoCA. This study showed that the VCAT is a valid and useful screening tool for patients with cognitive impairment in Malaysia and is feasible to be used in the clinical settings.


Introduction
Worldwide, over 46 million people are living with dementia and the number is estimated to increase to reach approximately 131.5 million people by 2050. Dementia has a huge economic impact, whereby the total estimated worldwide cost is USD $818 billion [1]. In Malaysia, the prevalence of dementia was 123,000 people in 2015. This number is projected to be double to about 261,000 people by 2030 and will continue to increase to 590,000 people by 2050 [2]. An urban community study done in Kuala Lumpur among the older Malay people revealed that the prevalence of dementia was 6% [3]. The disability weight of dementia was higher than many other health conditions, and its burdens on health services, community, and institutional care are substantial. Diagnosing dementia at an early stage is essential to establish effective medical interventions and social support networks [4]. Ultimately, the early diagnosis of cognitive impairment can reduce healthcare costs and dementia care burden for patients, caregivers, and the health system [2,[5][6][7][8].
The ethnic composition of Malaysians is composed of 69.3% Bumiputera (Malays and other indigenous group), 22.8% Chinese, 6.9% Indians, and 1.0% other ethnicities [9]. Apart from Bahasa Melayu being the national language, few other commonly conversed local languages include English, Mandarin, and Tamil. The diversity in local languages, dialects, cultures, and levels of illiteracy pose significant challenges for the use of appropriate neuropsychological testing instruments during clinical diagnosis [10].
Cognitive tests can provide clinicians with an objective measurement of cognitive function, offering a means to reliably identify the neuropsychological and brain changes reflecting the underlying disease process [11]. There are many cognitive screening tools such as the Mini Mental State Examination (MMSE) [12], the Brief Alzheimer Screen (BAS) test [13], the Mini-Cog test [14], and the MoCA test [15] with good diagnostic performance values to differentiate between demented subjects and normal individuals. Most existing cognitive screening tools originated from the West and were designed for native English speakers. Clinical raters are forced to translate the content of test items into different languages to cater to their non-English speaking clients. As such, this process of translation may alter the intended neuropsychological properties of the test items [16].
The Visual Cognitive Assessment Test (VCAT) is a brief and visual-based assessment tool that was developed by a group of experts in Singapore. It is designed to detect early stages of cognitive impairment by assessing five different cognitive domains -memory, visuospatial function, executive function, language, and attention. It may be more suitable for multicultural and multilingual societies around the world as it offers the advantage of circumventing any need to translate or adapt to other languages or dialects as long as the administrator and participant speak the same language. According to the study published by Kandiah and colleagues in 2015, the sensitivity and the specificity of the VCAT for the diagnosis of early cognitive impairment (MCI and mild AD) were 85.6% and 81.1%, which is comparable to those of the MoCA in the same cohort [17].
A study done by Lim and colleagues in 2018 demonstrated that VCAT was useful in detecting cognitively impaired patients across multinational, multiethnic and multilingual Southeast Asian countries, with MoCA scores but not VCAT performance differing by language of administration [18]. However, in that study, the participants from Malaysia were largely recruited from an urban population with higher education background; as such, the usefulness of VCAT in other states within Malaysia is still unknown. Validating the VCAT for the multilingual and multicultural population in Malaysia is important. If the VCAT is validated, it can be widely used in Malaysia to facilitate the diagnosis of early stages of dementia. Additionally, a standardized cognitive battery may facilitate information flow through the health system. Clinicians can use cognitive evaluation to chart progress as patients with dementia transit between the different healthcare settings for diagnosis, treatment, and management [19]. In this study, we aimed to evaluate the diagnostic performance of VCAT, its feasibility, and its reliability within multilingual rural and urban population settings in Malaysia.

Study Design, Setting and Duration
This prospective multicenter study was carried out across five hospitals in Northern Malaysia: Hospital Seberang Jaya and Hospital Pulau Pinang within the state of Penang, Hospital Raja Permaisuri Bainun and Hospital Taiping within the state of Perak, and Hospital Sultanah Bahiyah in the state of Kedah. The study was conducted between May 2018 and December 2019.

Sample Size and Participants Recruitment
The sample size was calculated in accordance to logical justifications as proposed by Lakens [20], and the calculation was calculated based on the probability proportionate to size sampling (PPS) technique. The sample size was computed based on the expected proportion of 25% of patients having memory or cognitively related conditions for the population within the Northern Region of Malaysia, with a desired precision estimate of 0.05 and a power of 90% [21], thus yielding a total sample size of 203. The average number of patients with memory or cognitively related conditions within the past six months was calculated from each hospital's outpatient psycho-geriatric or memory clinics. A 10% deflation rate on the average number of patients in each hospital was computed, assuming the non-follow-ups, drop-out or did not fulfill the study's inclusion or exclusion criteria. The proportion of patients to be recruited in each hospital was then calculated by the number of patients in each hospital divided by the total number of patients in all the hospitals, and subsequently each proportion was multiplied by the calculated sample size to determine the number of patients to be recruited for each hospital site based on pre-determined inclusion and exclusion criteria (Table 1). Although 203 patients were approached, the final participatory rate was 91.0%, with nineteen patients being excluded in the final analyses due to drop-out, failing to fulfill the study's inclusion or exclusion criteria or did not consent to participate in the study. All participants that fulfilled the inclusion and exclusion criteria and agreed to participate were recruited to this study. Of the 184 participants, 66 participants were recruited from the Neuro-Geriatric Clinic of Hospital Seberang Jaya, 45 participants were recruited from the Psycho-Geriatric Clinic of Hospital Pulau Pinang, 42 participants were recruited from the Memory Clinic of Hospital Raja Permaisuri Bainun, 17 participants were recruited from the Division of Geriatric Medicine, Hospital Taiping, and 14 participants were recruited from the Division of Geriatric Medicine, Hospital Sultanah Bahiyah.
Cognitively healthy participants (CHPs) were recruited from health screening programs organized by the respective psycho-geriatric or memory clinic team of the hospitals, while MCI, mild VaD and mild AD participants were recruited from follow-up patients within the specialist outpatient clinics. Of the 184 participants, 79 participants were CHPs, 46 participants were mild cognitive impairment (MCI), 38 participants were mild Alzheimer's disease (AD) and 21 participants were mild Vascular Dementia (VaD).

Study Inclusion, Exclusion and Diagnostic Criteria
Inclusion criteria were CHPs, MCI, VaD, and AD subjects. Classifications of MCI was based on the Petersen's criteria and the National Institute on Aging and Alzheimer's Association (NIA-AA) [22]; classification of mild VaD was based on National Institute of Neurological Disorders and Stroke and the Association Internationale pour la Recherché et l'Enseignement en Neurosciences (NINCDS-AIREN) criteria [23]; and classification for mild AD was based on NIA-AA criteria [24]. MCI participants were required to have a Clinical Dementia Rating (CDR) score of 0.5, whereas mild VaD and mild AD participants were required to have a CDR score of 1 [25]. CHPs were required to have no cognitive complaints, and a CDR score of 0. Only participants with at least six years of education and aged 50 years and older were recruited in this study. Participants with a Geriatric Depression Scale (GDS) score of six or more, suggestive of depression, were also excluded [26]. Participants with poor vision were also excluded. After that, participants were classified into two groups, either the CHP group or CI group, which consists of MCI, mild VaD, and mild AD participants.

Data Collection and Instruments Used
Basic demographic information was collected which include age, gender, ethnicity, years of education and employment status. During each interview, the first accessor administered the CDR, GDS, and MoCA assessments; the second accessor performed the VCAT assessment and blinded from the first accessor to prevent bias. All the assessments were conducted in the same setting. All participants were also given a semi-structured questionnaire to collect their feedback on VCAT.
The VCAT is a 30-point test whereby a higher score denotes higher cognitive functions. The VCAT is comprised of eleven items, which include three memory items (13 points), one attention item (3 points), three executive functioning items (6 points), two visuospatial items (3 points), and two language items (5 points). Instructions are communicated in a language that both the test administrator and the participant are comfortable in conversing in [17].

Statistical Analyses
Statistical analysis was performed using SPSS version 22. Descriptive analyses were presented for demographic and cognitive data. The association of categorical variables were analyzed using chi-square (X 2 ) or Fisher exact test. Independent t-test was used to perform between-group comparisons on normally distributed continuous data, while the Mann-Whitney test was used to analyze non-normally distributed continuous data. Further analyses were performed using a Generalized Linear Model (GLM) to adjust for confounding demographic variables. Receiver operating characteristic (ROC) analysis was used to determine the best VCAT cutoffs to differentiate: (1) CHPs from patients with cognitive impairment (MCI + mild AD + mild VaD); and (2) MCI from mild dementia (mild AD + mild VaD). Diagnostic performance was assessed using the area under the curve (AUC), sensitivity (Se), specificity (Sp), positive (PPV) and negative predictive values (NPV), and positive (PLR) and negative likelihood ratio (NLR). The ROC curves were compared according to Hanley & McNeil AUC comparison method [27]. Concurrent validity was evaluated using Pearson correlation coefficient (r) with the MoCA. The level of statistical significance tests was set at p < 0.05 with two-tailed distribution. The coefficient McDonald's Omega for the total VCAT score was yielded to interpret the reliability statistics.

Characteristics of Study Sample
The total sample was 184 participants which consisted of males (43.5%) and females (56.5%), with mean (SD) age of 68.8 (8.75) years old. The sample consisted of 71.7% Chinese, 14.7% Malays, 11.4% Indians and 2.2% of other ethnicities. The majority of the participants were retirees (60.9%) and the mean (SD) years of education was 12.0 (4.75) ( Table 2). There were 79 CHP and 105 CI participants. Between group comparisons, demographic variable age was found to be significantly different (p < 0.001), with the CI group being significantly older (71.5 ± 8.80) than the CHP group (65.3 ± 7.35) ( Table 1). Significant gender differences were identified with a higher proportion of females in the CHP group (65.8%), compared to the CI group (49.5%). Participants in the CI group tended to have fewer years of formal education (11.2 ± 3.07). In terms of employment, the CI group compared to the CHP group had more retirees (66.7% vs. 53.2%) and unemployed (21.0% vs. 13.9%).
On GLM analysis, after controlling for age, years of education, gender, and employment, the two groups were found to have significant differences in MoCA (25.9 ± 0.54 vs. 19.2 ± 0.47, p < 0.001), total VCAT score (24.6 ± 0.68 vs. 17.9 ± 0.58, p < 0.001), and all individual VCAT domain scores. In all of the cognitive tests, the CHP scored higher than the CI participants. GDS was not significantly different between two groups (1.1 ± 1.21 vs. 2.2 ± 1.43, p = 0.056) ( Table 3).

Test-Feasibility
There was no significant difference for completion time between VCAT vs. MoCA. A total of 180 participants provided feedback for evaluating test burden by using the semi-structured questionnaire. Compared with MoCA, 76.0% thought that the instructions in VCAT were similar or easier to understand and 76.6% thought VCAT test were similar or less tiring.

Discussion
This multicenter study was carried out across Northern Malaysia to investigate the VCAT performance in multi-ethnic, multicultural, and multilingual participants. The findings of this study demonstrated that the overall discriminative ability was comparable between the VCAT and MoCA in discriminating between CHPs, Cis, and mild dementia (AD + VaD) participants. ROC analyses revealed an optimal cutoff score, which was almost similar to the cutoff score by Low et al. (Dementia 0-19, MCI 20-24, Normal 25-30) [30].
This study showed that there was no statistical difference when comparing VCAT scores between Chinese, Malay, and Indians after correcting for age and education. Therefore, VCAT was well suited for the multiracial population in Malaysia. The majority of the multilingual patients in Malaysia found that the VCAT instructions were simple to follow and culturally appropriate.
The MoCA showed a slightly larger AUC than the VCAT in discriminating between CHP from those with CI. Comparison of AUCs using the method of Hanley and McNeil showed no significant difference between the MoCA and the VCAT [27]. In this study, the AUCs for the VCAT was slightly lower in discriminating between CHPs and CIs compared to the study done by Low et al. [30]. The cutoff score derived in this study was lower compared to the study done by Low et al. whose cut off score was ≤24. The sensitivity and specificity in this study was higher compare to the study done by Low et al, where the sensitivity was 75.4% and specificity was 71.1% [30]. Compared to a previous study done by Kandiah et al., for the cut of score of ≤22, the sensitivity was 85.6% and specificity was 81.1% [17].
The VCAT performed well compared to the MoCA in discriminating between MCI and mild dementia (mild AD + mild VaD). The VCAT showed a slightly larger AUC than the VCAT. The findings are consistent with previous study done by Low [17,30]. The cutoff score derived in this study was consistent with the study done by Low et al. where their cutoff score was ≤19 with sensitivity of 68.3% and specificity of 84.8% [30].
The mean time to complete VCAT and MoCA was similar in the CHP and CI groups. It was similar to the previous study done by Lim et al., whereby the mean time to complete the VCAT was 10.37 ± 3.70 min in the CHP group and 13.88 ± 6.18 min in the CI group [18].
Based on the VCAT qualitative assessment questionnaire surveying participants' opinions on the practicality of VCAT compared to MoCA, participants thought that the instructions in VCAT were similar or easier to understand and were similar or less tiring. Therefore, it was found to be feasible in the normal routine clinical settings.
There were several limitations in this study. First, the education level for our overall sample was relatively high, and therefore the results may not be generalized to individuals with lower levels of education. Secondly, the majority of our participants were Chinese, and the common language used was English, which did not represent the actual Malaysian populations. Thirdly, there is a lack of description on the precision of the test with respect to it's severity as it only compares CHPs versus MCI and MCI versus mild dementia (mild AD + mild VaD). Fourthly, the MoCA test was selected for comparison as it is widely used in clinical settings even with full knowledge of its limitations, especially the potential influence on language and culture on MoCA performance. Finally, the subjects were accessed once and were unable to be re-assessed for test-retest learning effect. Future work could compare visual test MemTrax with VCAT, and to assess the VCAT's ability to detect various forms of dementia, and to evaluate the influence of learning effects on the VCAT.

Conclusions
Overall, this multicenter study showed that the VCAT was comparable to the MoCA. This suggests that the VCAT was a valid and useful screening tool for patients with cognitive impairment in Malaysia. Further studies should include a larger sample size and longitudinal cohorts.