Validity of the Japanese Version of the Quick Mild Cognitive Impairment Screen

Early detection of dementia provides opportunities for interventions that could delay or prevent its progression. We developed the Japanese version of the Quick Mild Cognitive Impairment (Qmci-J) screen, which is a performance-based, easy-to-use, valid and reliable short cognitive screening instrument, and then we examined its validity. Community-dwelling adults aged 65–84 in Niigata prefecture, Japan, were concurrently administered the Qmci-J and the Japanese version of the standardized Mini-Mental State Examination (sMMSE-J). Mild cognitive impairment (MCI) and dementia were categorized using established and age-adjusted sMMSE-J cut-offs. The sample (n = 526) included 52 (9.9%) participants with suspected dementia, 123 (23.4%) with suspected MCI and 351 with likely normal cognition. The Qmci-J showed moderate positive correlation with the sMMSE-J (r = 0.49, p < 0.001) and moderate discrimination for predicting suspected cognitive impairment (MCI/dementia) based on sMMSE-J cut-offs, area under curve: 0.74, (95%CI: 0.70–0.79), improving to 0.76 (95%CI: 0.72 to 0.81) after adjusting for age. At a cut-off of 60/61/100, the Qmci-J had a 73% sensitivity, 68% specificity, 53% positive predictive value, and 83% negative predictive value for cognitive impairment. Normative data are presented, excluding those with any sMMSE-J < 27. Though further research is required, the Qmci-J screen may be a useful screening tool to identify older adults at risk of cognitive impairment.


Introduction
Worldwide, 46.8 million people are living with dementia, and this number is increasing [1]. Associated global medical and social care costs are estimated to be US $818 billion annually, and are projected to double by 2035 [1,2]. Epidemiological research suggests that potentially modifiable cardiovascular and lifestyle factors contribute to dementia risk [3], and multi-dimensional interventions are effective in delaying the progression of cognitive decline [4,5].

Translation of the Qmci Screen
The original English version of the Qmci screen was translated into Japanese by a bilingual health researcher, fluent in English, but a native Japanese speaker. The translation was reviewed by an expert panel including the original translator, Japanese health professionals specializing in older people, and researchers with extensive experience in cognitive test development and translation. The complete version was then back-translated into English by a non-expert panel member, a native English speaker proficient in Japanese. The final version of the test was then piloted with five older Japanese community-dwellers and reviewed and approved by the expert panel for conceptual, cultural and linguistic equivalence.
Like the original English version, the Qmci-J screen comprises of six subtests assessing six domains of cognition: (1) Orientation, (2) word registration (five word), (3) clock drawing, (4) delayed recall, (5) verbal fluency (category) and (6) logical memory (immediate verbal recall of a short story); the total score ranges between 0 (indicating marked impairment) and 100 (suggesting no impairment). Cut-off scores adjusted for age and education are published for those presenting with cognitive symptoms (23).
Some changes were required, for example in word registration and recall we replaced "love (a-i)" with "sense of security (an-sin)" to match the linguistic frequency of the other four words (i.e., dog, rain, butter, door) in a Japanese cultural context, considering the number of syllables and category of the original. We also replaced "fragrant (ka-o-ri-no-yo-i)" with "beautiful (ki-re-i-na)" in the logical memory test for the same reason.

Study Settings and Participants
This study was conducted as a part of the Neuron to Environmental Impact across Generation (NEIGE) study, a cohort study that aims to explore determinants of healthy ageing and longevity in rural areas of Japan. Between September and October 2017, 600 community-dwelling adults aged between 65 and 85 years old, randomly sampled in Tokamachi city (Niigata prefecture, Japan), were invited to participate in a face-to-face survey and health check-ups at community centers. Of these, 526 (87.7%) agreed to participate, and completed the Qmci-J screen and sMMSE-J as a part of the broader study. Both instruments were administered consecutively, but in random order.

Data Analysis
Descriptive statistics were used to summarize data. In the first analysis (scenario 1) cognition was classified using the established cut-offs for MCI and dementia on the sMMSE-J: According to scoring ranges provided in the sMMSE-J manual, participants were classified into likely normal cognition (sMMSE-J score: [27][28][29][30], suspected MCI (sMMSE-J score: [21][22][23][24][25][26] and suspected dementia (sMMSE-J score: ≤20). In the second analysis (scenario 2), cognition was classified according to age cut-offs for mean sMMSE-J scores, which were applied to the data. Education cut-offs were not used as the sMMSE-J is not overly impacted by this [22]. Taking 95% of the population's results as a normal reference range [23], we defined and removed values as outliers if they fell beyond the mean minus two standard deviations (SD) of sMMSE-J scores in those with normal cognition and MCI (the 2.5% of values at the bottom end of the normal range), and created new age-adjusted cut-offs for those aged 65-74 years (range for likely normal: 26.6-30, suspected MCI: 20.3-26.6, and suspected dementia: <20.3) and for subjects aged ≥75 (normal: 25.8-30, MCI: equal to or greater than 21.2, <25.8, dementia: less than 21.2). In the third analysis (scenario 3), normative data for the Qmci-J screen were examined for: those only scoring in the established normal range according to the sMMSE-J manual cut-off (i.e., ≥27) (scenario 3a), and those scoring ≥27 on the sMMSE-J, excluding those with subjective (self and informant reported) memory complaints, impairment in activities of daily living (ADL), history of diagnosed dementia, Parkinson's disease, stroke, those currently receiving treatment of depression or other psychiatric medication (scenario 3b). Participant characteristics were examined using the Chi square-test for categorical variables and the two-way analysis of variance (ANOVA) test for continuous variables. Spearman's correlation coefficients were calculated to examine correlations between the Qmci-J screen and sMMSE-J scores (both were positively skewed), and evaluate the concurrent validity of the Qmci-J. For an evaluation of diagnostic agreement with the sMMSE-J, we calculated the accuracy of the Qmci-J screen for predicting subjects with suspected cognitive impairment (suspected MCI and dementia) from the area under the curve (AUC) of receiver-operator curves (ROC) with 95% confidence intervals (CI). Finally, the maxim agreement with the MMSE-J classification was identified by computing the sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) at different Qmci-J cut-offs. The optimal cut-off was selected using Youden's Index (J = Sensitivity + Specificity − 1). All statistical analyses were performed using STATA 14.0 (StataCorp LLC, College Station, TX, USA).

Ethical Considerations
All subjects provided written, informed consent for inclusion before they participated in the study. The study was conducted in accordance with the Declaration of Helsinki , and the protocol was reviewed and approved by the Institutional Review Board of Niigata University (2666) and Tokyo Medical and Dental University (M2017-233).

Results
The present study included 526 participants with a mean age of 73.5 and SD of ±5.6 years. Most, 52.7%, were female. The largest proportion had at least completed secondary school education, 44.7%. Less than 5% had hearing or visual impairment. Only 3 participants (0.57%) had an established diagnosis of dementia. The median score for the Qmci-J screen was 62, with an interquartile range (IQR) of 55 to 68. Its median administration time was 6 min. The characteristics of all participants included, and according to their categorization in scenario 1, are presented in Table 1.

Scenario 1
Using established cut-offs for the sMMSE-J, 351 participants were classified with likely normal cognition (median sMMSE-J score of 28/30, IQR 27-29), 159 were suspected MCI cases (median 25, IQR 24-26), and 16 others were suspected dementia cases (median 19, IQR [17][18][19][20]. Participants who were older (p < 0.001), less educated (p < 0.001) and had impaired hearing (p < 0.001) were more likely to be classified as suspected cognitive impairment (MCI or dementia) but sex (p = 0.051) and visual impairment (p = 0.89) were not associated with the classification ( Table 1) The present study included 526 participants with a mean age of 73.5 and SD of ±5.6 years. Most, 52.7%, were female. The largest proportion had at least completed secondary school education, 44.7%. Less than 5% had hearing or visual impairment. Only 3 participants (0.57%) had an established diagnosis of dementia. The median score for the Qmci-J screen was 62, with an interquartile range (IQR) of 55 to 68. Its median administration time was 6 min. The characteristics of all participants included, and according to their categorization in scenario 1, are presented in Table 1.  [17][18][19][20]. Participants who were older (p < 0.001), less educated (p < 0.001) and had impaired hearing (p < 0.001) were more likely to be classified as suspected cognitive impairment (MCI or dementia) but sex (p = 0.051) and visual impairment (p = 0.89) were not associated with the classification ( Table 1). The Qmci-J screen scores were moderately and positively correlated with the sMMSE-J, where positive scores indicate higher cognitive function in both tests (Spearman's r = 0.49, p < 0.001) ( Figure 1).  The accuracy of the Qmci-J screen was moderate to good with an area under the curve (AUC) of 0.74 (95%CI: 0.70-0.79) for predicting suspected MCI and dementia, identified using established sMMSE-J cut-off scores, and was excellent for predicting suspected dementia, AUC of 0.90 (95%CI: 0.84-0.97) (Figure 2). Tables 2 and 3 show the sensitivity, specificity, PPV, NPV of the Qmci-J screen with 95% CIs for predicting suspected cognitive impairment (MCI and dementia) identified by the sMMSE-J at different Qmci-J screen cut-offs. Using Youden's Index, 60/61 was found to be the optimal the cut-off point for differentiating suspected cognitive impairment (MCI + dementia based on sMMSE-J scores), producing a sensitivity of 73%, specificity of 68%, PPV of 53% and NPV of 83% (Table 2). A cut-off point of 56/57 was best for discriminating between suspected dementia and non-dementia (normal and those with suspected MCI), with a sensitivity of 94.0%, specificity of 72.0%, PPV of 10.0% and NPV of 100.0% (Table 3). The accuracy of the Qmci-J screen was moderate to good with an area under the curve (AUC) of 0.74 (95%CI: 0.70-0.79) for predicting suspected MCI and dementia, identified using established sMMSE-J cut-off scores, and was excellent for predicting suspected dementia, AUC of 0.90 (95%CI: 0.84-0.97) (Figure 2). Table 2 and 3 show the sensitivity, specificity, PPV, NPV of the Qmci-J screen with 95% CIs for predicting suspected cognitive impairment (MCI and dementia) identified by the sMMSE-J at different Qmci-J screen cut-offs. Using Youden's Index, 60/61 was found to be the optimal the cut-off point for differentiating suspected cognitive impairment (MCI + dementia based on sMMSE-J scores), producing a sensitivity of 73%, specificity of 68%, PPV of 53% and NPV of 83% (Table 2). A cut-off point of 56/57 was best for discriminating between suspected dementia and nondementia (normal and those with suspected MCI), with a sensitivity of 94.0%, specificity of 72.0%, PPV of 10.0% and NPV of 100.0% (Table 3).

Scenario 2 (Using sMMSE-J Age-Adjusted Cut-Offs)
When sMMSE-J scores were adjusted for age, 260 participants were classified as having likely normal cognition, 222 with suspected MCI and 30 with suspected dementia. Using this classification, the accuracy of the Qmci-J screen for separating likely normal cognition from suspected cognitive impairment (MCI or dementia) improved to an AUC of 0.76, 95% CI 0.72 to 0.81. Similarly, its accuracy for differentiating suspected dementia from non-dementia improved to an AUC of 0.91, 95% CI 0.86 to 0.97.

Scenario 3a (Normative Data Based on sMMSE-J Scores ≥27 Only)
Normative values for the Qmci-J screen were developed based on scores grouped by age and education for participants with sMMSE-J scores ≥27 only (n = 352) included. These are presented in the Table 4 and show how Qmci-J screen scores vary by an individual's age and education, increasing in those with more education, and decreasing in those with greater age, except for those aged 80-84 years who had the lowest scores, but were not affected by their number of years in education. Finally, we present the normative values of the Qmci-J screen by age and education for participants with sMMSE-J scores ≥27, but without a history of dementia, subjective memory complaints, ADL impairment or other features or conditions associated with cognitive decline (n = 265). These were broadly similar to those developed with scores ≥27, and are presented in the Table 5. Table 5. Total Quick Mild Cognitive Impairment (Qmci-J) screen median scores (IQR) by age and education based on with sMMSE-J scores ≥27 for those without symptoms of cognitive impairment or suggestive medical conditions or medications (n = 265).

Discussion
The present study is the first to develop and evaluate a Japanese version of the Qmci-J screen to aid in the screening of community-dwelling older adults, and determining their likelihood of having cognitive impairment. It presents the concurrent validity of the Qmci-J screen against the established sMMSE-J using different modeling exercises (scenarios), and shows that the Qmci-J screen score was moderately, positively and significantly correlated with the sMMSE-J. A Qmci-J screen cut-off of 60/61 out of 100 points provided a 73% sensitivity and 68% specificity for differentiating older adults with suspected cognitive impairment (MCI and dementia) and likely normal cognition according to established cut-offs for these on the sMMSE-J.
Time constraints are a common and important barrier to using CSIs in busy primary care settings [21,24], though it is recognized that they must be interpreted with care, and are not a substitute for clinical assessment [24]. The median time to administer the Qmci-J screen was 6 min, longer than the previous reports of 3-5 min [22,25], but still acceptable. This additional time may in part be due to our study procedure, where we asked participants to name their highest educational institution to verify if it was an educational institution accredited by the Japan Ministry of Education, Culture, Sports, Science and Technology, instead of asking a total number of educational years. This questioning was included in the administration time. Also, our study participants were research participants, and they might have been more talkative than patients and their families attending a clinic for assessment.
The relatively low accuracy of the Qmci-J screen for cognitive impairment is in contrast with other studies showing good to excellent accuracy for the Qmci screen [15][16][17]26,27]. Yet despite the fact that the NEIGE cohort study does not collect information sufficient to diagnose patients with cognitive disorders according to standardized criteria (e.g., DSM-5) and the inherent issues of using the sMMSE-J to categorize impairments, the Qmci-J screen still showed moderate diagnostic agreement. The authors then adjusted the sMMSE scores for age in an attempt to improve the diagnostic classification, and found that the accuracy of the Qmci-J screen further improved.
In this study because the accuracy of the Qmci-J screen was dependent on the classification of cognitive impairment based on sMMSE-J scores, it was not possible to compare the diagnostic accuracy of the two instruments. This is another limitation of the available data and the subsequent analysis in the paper, and is important because the sMMSE-J was not designed to detect MCI, unlike the Qmci-J screen [27][28][29] In order to evaluate the clinical diagnostic accuracy of the Qmci-J screen in comparison with the MMSE, future studies should compare the Qmci-J and predictions by MMSE-J of subjects with cognitive impairment diagnosis based on clinical and neuropsychological examination.
The optimal Qmci-J screen cut-off for predicting those with suspected cognitive impairment identified by MMSE-J was 60/61 (<61/100). This is very similar to the proposed cut-off for the original English version with respect to predicting cognitive impairment (MCI and dementia) identified by virtue of clinical and neuropsychological examination [26,27]. A similar cut-off, 62 (<62/100) provided a sensitivity of 83% and specificity of 87% in an Irish sample [26], and the identical cut-off, 61 (<61/100) produced a sensitivity of 90% and specificity of 87% in a Canadian sample [27]]. The optimal Qmci-J cut-off for predicting suspected dementia identified by sMMSE-J was 56 (<56/100). This is higher than the proposed cut-off for the original English version, <54/100, though it is again similar [27]. It must be highlighted that our data was normative data, and only 3 subjects (0.57%) were currently receiving medical care for dementia. Higher cut-off points may be efficient enough to discriminate suspected cognitive impairment from normal cognition in community-dwelling older adults. Further study is now required to examine the diagnostic accuracy against a more clinically appropriate classification of cognitive impairment using internationally recognized criteria, and against MCI-specific instruments including the Japanese version of the Montreal Cognitive Assessment (MoCA) [28] in different healthcare settings.
This paper is also one of the first, after the recent publication of the Italian version of the Qmci screen [20], to provide normative data for the Qmci screen in likely normal subjects, and this study applied a similar approach to identify likely normal participants as the Italian paper [20]. Despite the challenges of defining normative data based on sMMSE scores, there was no marked difference in median Qmci-J screen scores, after maximizing the likelihood of selecting a truly cognitively normal sample by excluding those with symptoms (both subjective and informant reported), ADL impairment or conditions known to be associated with MCI and dementia, such as stroke and Parkinson's disease. This suggests that a high proportion of those presumed to have normal cognition based on the sMMSE-J scores likely did have normal cognition, increasing the chance that the results of ROC curve analysis are robust, though this is a limitation of the study. Another limitation is the relatively low PPV, reflecting the cut-off selected [29] and that the sample is community-based and largely asymptomatic with a low prevalence of cognitive impairment [30]. In this setting, the Qmci-J would be ideally used as a quick 'pre-screen' as part of a two-step screening and assessment approach to diagnosing cognitive impairment. The normative data shown here for an older Japanese population were remarkably consistent with the Italian sample [20]. For example in Italy, those aged 60-69 with 13+ years in education had a mean score of 70/100 on the Qmci-I screen [20], the same median score for Japanese participants aged 65-69. For Italians between 70-80 years with 13+ years education, 63.86 was the mean score [20] compared to a median score of 63.5 for those in Japan with the same time in education aged 75-79 years.

Conclusions
In conclusion, this study presents the concurrent validity of the Japanese version of the Qmci screen, showing that it correlates moderately, but has a shorter administration time than the most widely used CSI in Japan, the sMMSE-J. Although further research is required to demonstrate diagnostic accuracy, the Qmci-J screen is a short CSI that could be useful from a public health perspective to rapidly determine the likelihood of having cognitive impairment, and whether to proceed with further evaluation and with subsequent diagnostic tests or procedures in community-dwelling older adults at population level, either in acute hospital care or in busy primary care settings.