Next Article in Journal
SAFEvR MentalVeRse.app: Development of a Free Immersive Virtual Reality Exposure Therapy for Acrophobia and Claustrophobia
Previous Article in Journal
Selective Activation of the Spinal Cord with Epidural Electrical Stimulation
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Brain Sciences
Volume 14
Issue 7
10.3390/brainsci14070652
brainsci-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Machine Learning Classification of Patients with Amnestic Mild Cognitive Impairment and Non-Amnestic Mild Cognitive Impairment from Written Picture Description Tasks

by
Hana Kim
1,
Argye E. Hillis
2,3,4 and
Charalambos Themistocleous
5,*
1
Department of Communication Sciences and Disorders, University of South Florida, Tampa, FL 33620, USA
2
Department of Neurology, Johns Hopkins University School of Medicine, Baltimore, MD 21287, USA
3
Physical Medicine and Rehabilitation, Johns Hopkins University School of Medicine, Baltimore, MD 21287, USA
4
Department of Cognitive Science, Johns Hopkins University, Baltimore, MD 21287, USA
5
Department of Special Needs Education, University of Oslo, 0371 Oslo, Norway
*
Author to whom correspondence should be addressed.
Brain Sci. 2024, 14(7), 652; https://doi.org/10.3390/brainsci14070652
Submission received: 10 June 2024 / Revised: 24 June 2024 / Accepted: 25 June 2024 / Published: 27 June 2024
(This article belongs to the Section Computational Neuroscience and Neuroinformatics)
Browse Figures
Versions Notes

Abstract

Individuals with Mild Cognitive Impairment (MCI), a transitional stage between cognitively healthy aging and dementia, are characterized by subtle neurocognitive changes. Clinically, they can be grouped into two main variants, namely patients with amnestic MCI (aMCI) and non-amnestic MCI (naMCI). The distinction of the two variants is known to be clinically significant as they exhibit different progression rates to dementia. However, it has been particularly challenging to classify the two variants robustly. Recent research indicates that linguistic changes may manifest as one of the early indicators of pathology. Therefore, we focused on MCI’s discourse-level writing samples in this study. We hypothesized that a written picture description task can provide information that can be used as an ecological, cost-effective classification system between the two variants. We included one hundred sixty-nine individuals diagnosed with either aMCI or naMCI who received neurophysiological evaluations in addition to a short, written picture description task. Natural Language Processing (NLP) and a BERT pre-trained language model were utilized to analyze the writing samples. We showed that the written picture description task provided 90% overall classification accuracy for the best classification models, which performed better than cognitive measures. Written discourses analyzed by AI models can automatically assess individuals with aMCI and naMCI and facilitate diagnosis, prognosis, therapy planning, and evaluation.

1. Background

With the growth in the number of older adults, age-related neurodegenerative diseases such as Alzheimer’s disease (AD) have dramatically increased. These diseases cause a great deal of financial and emotional burden for patients, their caregivers, and society. The global cost of dementia care was estimated to exceed USD 500 billion in the United States [1]. It is expected to rise to USD 2 trillion by 2030 [2]. Research has suggested that the preclinical phase of dementia may start earlier than the diagnosis. Detecting the preclinical stage of dementia and providing an intervention will delay the onset of AD. Delaying the onset of AD will significantly minimize the socio-economic burden and is expected to reduce societal costs by 40% [3].
Mild cognitive impairment (MCI) is an intermediate stage between cognitively healthy aging and dementia [4]. It represents a critical preclinical stage of AD [5,6,7]. MCI includes four different clinical subtypes. Two main subtypes are amnestic MCI (aMCI) and non-amnestic MCI (naMCI); this subtyping is determined based on the impairment in memory. Individuals with aMCI are characterized by memory loss, while individuals with naMCI demonstrate impairment in domains such as executive functions, attention, and language [8,9]. Also, depending on the number of cognitive domains impaired, individuals can be categorized into single-domain and multi-domain MCI. Although a higher risk of developing dementia characterizes individuals with MCI, not all individuals with MCI will progress to dementia; some may remain stable, and others even regress to a condition of healthy aging [10,11,12]. Therefore, it is essential to discriminate against those who are more likely to progress to dementia for early intervention since most treatment strategies are more effective in the presymptomatic stage of dementia [13].
Depending on the two main subtypes of MCI, differences in the progression from MCI to dementia have been reported. In general, it has been suggested that aMCI represents the earliest symptomatic manifestation of AD pathophysiology, while naMCI is likely to progress to non-Alzheimer’s dementia [14,15,16]. A recent 20-year retrospective study supports this and adds more information with a large dataset (N = 1188). The authors demonstrated that aMCI represents a greater risk for progressing to dementia (not only for AD) compared to naMCI. The odds ratio of the progression to dementia between aMCI and naMCI was statistically different [17]. This highlights the clinical need for a robust, reliable system for classifying aMCI and naMCI [18].
There have been several approaches for MCI diagnosis. Behaviorally, a brief cognitive screening test can assist in identifying whether an individual has an apparent cognitive impairment [9]. Neuropsychological tests can be administered depending on the need for further assessments to determine the presence or degree of impairment in cognitive functions. The tests for MCI biomarkers require magnetic resonance imaging (MRI) or lumbar puncture for cerebrospinal fluid (CSF). An increased amyloid burden was found to be specific to aMCI, while naMCI does not exhibit a specific abnormality in neuroimaging (see review for Yeung et al. [19]). Blood biomarkers, which are considered a comparatively more straightforward means of testing, have also been investigated [20]. Unfortunately, such tests for MCI biomarkers are not routine care in clinical settings [21,22,23,24]. Moreover, the cost and availability of the testing technique (e.g., MRI) may limit its impact on individuals’ care [25].
Linguistic changes are considered to manifest as one of the earlier indicators of pathology in cognitive impairment. It has been reported that they emerge years before deficits in other cognitive systems become apparent [26]. In particular, writing is a cognitively and linguistically complicated activity. Writing consists of distinct phases: planning, generating, and revising [27]. Writers initially set a goal for organizing their knowledge and executing the plan in response to the topic of the writing activity. Then, writers revisit and revise their output. All phases should be well orchestrated to accomplish successful writing within cognitive systems such as executive functions, attention, and working memory.
The research directly focusing on writing abilities in patients with amnestic and non-amnestic MCI is very limited [28,29]. As mentioned above, the cognitive model of writing proposed cognitive functions involved in picture descriptions, but it has not specified what cognitive functions are required in each process of writing [27]. Considering the model, memory in which aMCI demonstrates impairment can be involved in the process of writing complete, cohesive, and coherent passages while memorizing previously written sentences. Contrarily, other cognitive systems like language, executive functions, and attention where patients with naMCI have difficulty can be associated with the overall process of writing. Due to the lack of research on writing abilities in MCI, research explicitly examining writing samples from individuals with aMCI and naMCI is needed to show any characteristic patterns of writing impairment associated with each type.
A recent review article highlighted the diagnostic value of writing tests, especially at the discourse level [29]. Discourse is any language beyond the sentence level [30,31]. Kim and colleagues [28] investigated the prognostic value of discourse-level writing tests. They conducted a chart review of individuals diagnosed with MCI and visited a neurology outpatient clinic more than once (N = 71). They classified the study participants into a stable MCI group and a converter group. The authors examined whether a written discourse task using the Cookie Theft picture [32] predicts clinical course in the MCI group. They found that the stable MCI group produced more core words than the converter group at their baseline assessment. This underscores the potential clinical utility of discourse-level writing tasks for early detection of those who are likely to progress to dementia from MCI.
However, the manual analysis of language production (both spoken and written modalities) is time-consuming and labor-intensive [33]. In recent years, computational methods such as Natural Language Processing (NLP) have been used to analyze discourse samples in individuals with neurodegenerative diseases [34,35,36,37,38,39] and improve screening methods [34,38]. Computational methods offer two advantages. First, they allow the elicitation and combination of measures from different linguistic domains. A decisive property of machine learning (ML) models is their ability to find patterns between features associated with a specific group of individuals, i.e., patients with aMCI and naMCI. The Cookie Theft picture description task is typically an oral task, used for employing connected speech, and was shown in several studies to distinguish between individuals with MCI and healthy controls (HCs) [34,35,37,38,39]. ML and NLP analysis can be employed to analyze and interpret the subtle linguistic patterns in language that might not be readily apparent to human observation. Using ML, earlier studies successfully distinguished healthy adults from individuals with MCI [35], MCI from dementia [40,41,42,43], and the subtypes of primary progressive aphasia [44,45]. These findings highlight the use of ML as an important tool that can contribute to existing approaches [38] and inform clinical assessment and therapy.
In this study, we leverage the written form of the Cookie Theft picture description task to explore its potential for classifying two subtypes of MCI: aMCI and naMCI. This study marks the first attempt to utilize discourse-level writing samples from patients with MCI subtyping. By focusing on the written form, we aim to develop a more accessible assessment tool for individuals with speech difficulties, potentially expanding the reach of early MCI detection. If writing samples differentiate individuals with two MCI subtypes, they could allow the assessment of individuals with speaking disorders unrelated to MCI. Also, since writing involves several cognitive functions (especially language, vision, and motor control), we hypothesized that a written picture description task could distinguish individuals with aMCI and naMCI. For example, Yan et al. [46] found that patients with AD and MCI “demonstrated slower, less smooth, less coordinated, and less consistent handwriting movements than their healthy counterparts”. This work could provide a quick and easy tool to facilitate the subtyping of patients with MCI and demonstrate the potential contribution of written language tasks in the automatic assessment of patients with cognitive impairments.

2. Methods

2.1. Participants

Our participants comprised 169 individuals diagnosed with either aMCI or naMCI (Table 1). All individuals were recruited through the Johns Hopkins Hospital and were diagnosed by an experienced neurologist (AEH). All of the cognitive and linguistic tests were part of routine care in the outpatient clinic. Participants were seated in a quiet room for testing with examiners. All procedures took approximately 45 to 60 min. The diagnosis was based on history, neuroimaging, neurological examination, and neuropsychological testing, and all individuals met the current criteria for MCI (Table 2). The exclusion criteria for the study included individuals who (1) were younger than 18 years old, (2) had a lack of English competence, (3) had significant psychiatric illness and alcohol and drug use, (4) had significant neurological problems affecting the brain (e.g., stroke, multiple sclerosis, and Parkinson’s disease), and (5) had uncorrected visual or hearing loss. All individuals with MCI fulfilled the recent criteria of the 2018 National Institute on Aging–Alzheimer’s Association (NIA-AA) research framework [47]. Demographic information for individuals with MCI can be found in Table 1.
Specifically, participants underwent a battery of standardized neuropsychological tests to assess their cognitive and linguistic abilities. These tests comprehensively evaluated various aspects of language and cognitive functioning, offering a detailed assessment of their cognitive strengths and weaknesses. The neurocognitive tests include the Mini-Mental State Examination (MMSE, Folstein et al. [48]), the Orientation and Information subset from the Wechsler Memory Scale-Third Edition (WMS-III; Wechsler [49]), the Digit span subtests of the WMS-III [49], the Rey Auditory Verbal Learning Test (RAVLT; Rey, 1941), Rey Complex Figure (RCF; Rey [50]), the Boston Naming Test [32], the Verbal Fluency Task (FAS), the Free narrative writing section from BDAE [32], the Trail Making Test (TMT; Reitan and Wolfson [51]), and the Stroop test [52]. The tests were carefully selected to provide a sensitive measure of abnormalities compared to individuals with normal cognitive functioning. Table 2 includes neurocognitive test results for all individuals with MCI. The study protocol underwent rigorous review and received approval from the Johns Hopkins Institutional Review Board.

2.2. Written Picture Description Task

Writing samples were collected using the Cookie Theft picture from the Boston Diagnostic Aphasia Examination-3 (BDAE-3; Goodglass et al. [32]). Participants were seated with the picture stimulus and a piece of paper. The clinicians used the prompt to encourage the participants to provide a written description: “Write as much as you can about what you see going on in this picture.” Once the participants completed the task, their writing samples were transcribed into a text document by experienced researchers.

2.3. Machine Learning Process

The analysis involved the preprocessing of the data (Figure 1), the extraction of significant features from the written picture description task, and the study of those measures.

2.4. Analysis of Narrative Speech

The texts were automatically processed and multiple measurements were exported to an Excel file (participant ID included) using Open Brain AI’s (http://openbrainai.com accessed on 5 September 2023) clinical computational toolkit [53]. We analyzed the written transcripts from the text documents using two NLP tools, including the tokenization of the text, the tagging of morphological categories, and the parsing of the syntactic constituents. Specifically, each word in the text was labeled using Open Brain AI’s Part of Speech (POS) tagger and syntactic dependency parser, which uses a variety of linguistic information to determine the dependency structure of a sentence [53]. Open Brain AI provided automatic measures that included counts and the ratio of each word/total count of words that appeared in the text for each participant.
Specifically, the automatically elicited morphosyntactic measures shown in Table 3 include POS categories (i.e., adjective, adposition, adverb, auxiliary verb, coordinating conjunction, determiner, interjection, noun, numeral, particle, pronoun, proper noun, subordinating conjunction, symbol, and verb), the number of words and characters and their character/word ratio, and syntactic dependency measures indicating the grammatical relationships between words in a sentence and their count to total word ratio [54].

2.5. Semantic Measures from BERT

To depict semantic relationships, we included word and sentence embeddings from BERT-large-uncased, a BERT (Bidirectional Encoder Representations from Transformers) pre-trained language model [55]. Word embeddings were integrated with linguistic measures to form the final database. This inclusion was motivated by the importance of semantic measures in differentiating between individuals with aMCI and naMCI. Specifically, the BERT-large-uncased is a deep neural network trained on a large dataset of text corpora and can be used for various NLP tasks, such as question answering, text summarization, and sentiment analysis. The BERT-large-uncased has been shown to achieve state-of-the-art performance on various NLP tasks. It consists of 12 encoder layers, each containing a self-attention mechanism and a feed-forward network. The self-attention mechanism allows the model to learn long-range dependencies between words in a sentence, while the feed-forward network adds non-linearity.

2.6. Addressing Imbalance and Cross-Validation

We employed Random Over-Sampling (ROS) to balance the class distribution and address the limitations of the relatively small dataset [56]. This technique alleviates the models’ tendency to favor the majority class, a common challenge in imbalanced datasets. Additionally, we implemented group 5-fold cross-validation. This approach minimized data leakage and provided a more reliable model performance evaluation. Furthermore, we standardized the non-BERT features to ensure uniformity in scale.

2.7. Model Evaluation and Selection

We selected ML models that do not require massive amounts of training data. To choose the best model for our data, we trained ML models that roughly belong to four main categories of models, namely ensemble learning models (Random Forest (RF)), Gradient Boosting (GB), XGBoost (XGB), and LightGBM (LGBM)). RF is an ML method combining several decision trees to enhance prediction accuracy. This approach can manage high-dimensional data and is resilient to overfitting. GB sequentially combines weak ML learners, each correcting the predecessor’s errors. GB is used in classification and regression tasks for large, complex datasets. XGB and LGBM implement gradient boosting with speed and accuracy. They are employed in scenarios requiring rapid processing of large datasets. Hist Gradient Boosting (HGB), a gradient boosting variant, uses histograms for feature representation, enhancing efficiency with large-scale, high-dimensional data structures. Each ML algorithm has unique strengths, making these models suitable for specific data types and prediction tasks. Only comparing and selecting ML models provides versatility, adaptability, and improved performance in the ML process, enabling the model to tackle the various underlying characteristics of the data. The selected ML models (RF, GB, XGB, LGBM, and HGB) were trained on the training data.

2.8. Hyperparameter Tuning and Model Comparison

A grid search with cross-validation was employed to evaluate and compare the performance of the different machine-learning models. The hyperparameter tuning involved finding the optimal hyperparameters for each model using grid search and calculating the evaluation metrics.
Grid search is a method for hyperparameter tuning that evaluates different combinations of predefined hyperparameter values to determine the combination that produces the best performance for a given model. In this case, a grid search was performed for each of the ML models included in the study.
We evaluated each model using five-fold cross-validation, which involves evaluating the performance of a model by splitting the data into multiple folds. Each fold is used as a validation set, while the remaining folds are used as the training set. The model is trained on the training set and evaluated on the validation set. This process is repeated for each fold, and the average performance across all folds is used as the final performance estimate.
Various evaluation metrics were used to assess the performance of the different machine learning models. These metrics included accuracy, F1 score, precision, recall, ROC AUC, and Cohen’s kappa score. Accuracy is the proportion of correct predictions. The F1 score measures a model’s ability to correctly classify positive and negative cases. Precision is the proportion of positive predictions that are positive. Recall is the proportion of positive cases correctly classified as positive. ROC AUC (Receiver Operating Characteristic Area Under the Curve) measures a model’s ability to distinguish between positive and negative cases.

3. Results

Written picture description tasks were processed using combined NLP analysis and BERT models to elicit measures representing the embeddings. We implemented two ML-supervised classification tasks.
A classification model was designed to distinguish individuals with aMCI and naMCI. The model included only information from the Cookie Theft picture description task. The model distinguished individuals with aMCI and naMCI. These results suggest that the written discourse from a picture description task provides sufficient information to identify the individuals with the two variants of MCI.
In the ML models, the ROC curves were nearly 98% for classifying individuals with aMCI and naMCI (Figure 2). This suggests that written discourse productions, as manifested in a picture description task, can distinguish the two groups of individuals from language measures. Regarding accuracy, the ensemble models with boosting had the best performance (Table 3). The consistency in the output of those models further demonstrates their effectiveness for real-world applications.
As indicated by the outcomes (Table 4), the utilization of machine learning models shows the potential of MLs in diagnosing and differentiating the two MCI subtypes. The reported standardized metrics—accuracy, F1 score, precision, recall, and ROC/AUC—indicate the effectiveness of these models, with one (1) being the best value.
  • Accuracy (0.90 for most models) reflects the ML model’s overall correctness in classifying the MCI type.
  • F1 score balances precision and recall, with values around 0.70–0.72, indicating a good balance between false positives and false negatives.
  • Precision (0.74–0.75) measures the proportion of correctly identified positive cases among all positive calls made by the model.
  • Recall (ranging from 0.66 to 0.70) indicates the model’s ability to identify all actual positive cases.
  • ROC/AUC (between 0.97 and 0.98) reflects the model’s ability to distinguish between the two classes across various thresholds, with values close to 1 indicating excellent performance.
We evaluated the feature importance and found that BERT features dominate the rankings of the 15 contributing factors for the RF classification. The following features contribute to RF classification, ordered from more important to less important: prepositional object, adposition, dependent, particle, auxiliary, root (verb), adjective, and subordinating conjunction.
These results suggest a reliable performance in distinguishing patients with naMCI vs. aMCI, highlighting the potential of advanced ML techniques in medical diagnostics, especially for complex conditions like MCI. The high performance of these models suggests that they could be valuable tools in clinical practice for early and accurate identification of MCI types, thereby enabling more tailored and effective treatment strategies.

4. Discussion

MCI is an early stage of cognitive decline due to pathology reasons [4]. Individuals with aMCI are characterized primarily by memory deficits, while individuals with naMCI are impaired in other cognitive functions, such as language, attention, and executive functions. Identifying the type of MCI is important for predicting the progression of the condition, as individuals with aMCI are more prone to progress into Alzheimer’s disease [57,58] or other types of dementia (Glynn et al., 2021). This study determined the potential diagnostic utility of computational methods in classifying two subtypes of MCI from writing. We found that a written picture description task can distinguish individuals with aMCI and naMCI at approximately 90% accuracy. This finding confirms that written discourse analysis, which is infrequently carried out in clinical settings, provides clinically essential information [28] and can be a powerful approach for better characterizing the subtypes of MCI.
Importantly, our study shows that a single behavioral task (i.e., a picture description task) can provide substantial information about domains that require multiple separate tasks. As mentioned earlier, either multiple pen-and-pencil tasks or neuroimaging techniques need to be conducted clinically to classify MCI. Previous studies using ML algorithms and neuroimaging data demonstrated an accurate classification of MCI subtypes [59,60]. However, data can be obtained only with advanced techniques. They are not often feasible for individual patients [61]. Behaviorally, multiple tasks that evaluate different cognitive components, such as memory and executive functions, need to be administered, which is considered a time-intensive process. From a clinical perspective, computational assessment of language with ML and NLP opens the door for exciting opportunities to expand the analysis to both longer and more complex test productions.
Besides being a cost-effective assessment, it is also significant to note that the current study used written discourse samples, which have received little attention in research [29] and are not often collected and evaluated in clinical settings [62]. He et al. [63] used a spoken discourse task to investigate the classification among healthy adults, subtypes of MCI, and dementia. In the study, the researchers used both linguistic and acoustic features, but the classification accuracy (aMCI vs. naMCI) was 88%. Our findings shed light on the clinical value of written discourse, as the linguistic features in writing lead to higher classification accuracy. This also indicates that linguistic features in writing can be potential markers of memory deficits and may provide enough information for the classification.
Written discourse offers a plethora of information about individuals’ linguistic functioning, including textual macrostructure and microstructure. However, it is not clear which components of written discourse in this population are more influenced by cognitive impairment in MCI. This is evidenced by 102 different measures used to quantify writing behaviors in research with little repetition of the same measure (Kim et al., 2024). In the current study, using written discourse samples, we calculated the POS of each word and syntactic relationships [64] that appear in the written picture description task [54]. Together, this can be an optimal approach for analyzing such language samples in that it adds to the efficiency of written picture description analysis. This also provides a comprehensive and detailed grammar analysis in a standardized and less subjective manner.
Moreover, we found that the BERT semantic features dominated the hierarchy of analytical constructs that we used. This finding is consistent with the consensus that impairments in semantic domains of language are a key manifestation of disease progress in neurodegenerative disorders [28,65,66]. These features can be seen in the literature to be associated with one or more elements of the writing skills of individuals with MCI, as they interface linguistic and semantic memory domains.
Specifically, context-sensitive embeddings from BERT [55] played a critical role in the high accuracy of the classification. These result from averaging the token-level embeddings from the last layer of a BERT model for each input text, which creates a single, comprehensive vector representation for the entire text, capturing its overall contextual meaning. Traditional word embedding techniques, such as Word2Vec [67] and GloVe [68], generate a single word embedding for each word in the vocabulary. The embeddings are decontextualized, which fails to capture the meanings of polysemous words. For instance, the word bank can mean a financial institution that accepts deposits and makes loans or the sloping edge of a river or other body of water. On the other hand, BERT uses a technique known as contextual embedding. This means that the representation of a word is based on sentence context. So, the word bank would have different representations in the sentences “I went to the bank to retrieve money” and “the little house next to the river bank”, which offers a better representation of ambiguous meanings, improving the accuracy of text classification. Again, the contextual embeddings utilized in this study demonstrate a better understanding of the syntactic and semantic relationships between words in a sentence. This is crucial for quantifying the overall thematic content of the written picture descriptions. Additionally, since individuals with amnestic and non-amnestic MCI differ in their semantic memory [69,70], the contextual sensitivity of BERT’s embeddings helps the model adapt to differences in vocabulary and jargon.
Although it is well known that picture description tasks are valuable for eliciting connected language samples in individuals with MCI [71], the Cookie Theft picture offers a less organic method of personal expression through writing. Such productions are constrained substantially in their context and in effectively identifying differences in pragmatic language usage and speech and voice parameters. Also, the task does not allow the assessment of non-epistemic domains, such as deontic modality expressions of wish and hope and non-present tense verb-tense semantics, as it does not provide opportunities to discuss past or future events. Additionally, picture description tasks do not offer opportunities for expressing emotional and other affective content, which might be necessary for assessing the interface of language, emotion, and pragmatics. An open-ended essay writing could have offered the potential to assess more stylistic, linguistic, and communicative speech characteristics. Nevertheless, written picture descriptions demonstrate the potential to detect speech and language characteristics in neurodegenerative diseases such as MCI and dementia, as suggested by a recent review [28]. Considering the brief time to elicit writing samples, NLP combined with discourse-level writing samples will enable more efficient methods for analyzing these linguistic and communicative features, further enhancing the diagnostic accuracy and the clinical utility of written discourse analysis.

5. Conclusions

The results of the current study suggest that written discourse samples can offer a quick and efficient means of gaining valuable insights into linguistic abilities while minimizing the burden placed on individuals with MCI. Future research is necessary to verify this finding with a balanced sample size between aMCI and naMCI. For a better diagnostic tool, future studies, including MCI-dementia conversion, are needed to test the predictive value of the automatic classification of MCI.

Author Contributions

Conceptualization, H.K. and C.T.; methodology, H.K. and C.T.; formal analysis, C.T.; investigation, H.K. and A.E.H.; resources, H.K., C.T. and A.E.H.; data curation, H.K. and C.T.; writing—original draft preparation, H.K. and C.T.; writing—review and editing, H.K., C.T. and A.E.H.; visualization, C.T.; supervision, A.E.H.; project administration, H.K. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

This study was conducted in accordance with the Declaration of Helsinki and approved by the Johns Hopkins University School of Medicine Institutional Review Board (IRB00266221, 11 October 2020).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Data are available upon request to the authors, subject to approval by the Johns Hopkins University School of Medicine Institutional Review Board.

Acknowledgments

We acknowledge the valuable contributions of our patients who consented to this study, without whom this work would not be possible.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Alzheimer’s Association. 2019 Alzheimer’s disease facts and figures. Alzheimer’s Dement. 2019, 15, 321–387. [Google Scholar] [CrossRef]
  2. Ong, S.C.; Tay, L.J.; Ng, T.; Parumasivam, T. Economic Burden of Alzheimer’s Disease: A Systematic Review. Value Health Reg. Issues 2024, 40, 1–12. [Google Scholar]
  3. Zissimopoulos, J.; Crimmins, E.; St Clair, P. The value of delaying Alzheimer’s disease onset. Forum Health Econ. Policy 2015, 18, 25–39. [Google Scholar]
  4. Petersen, R.C.; Smith, G.E.; Waring, S.C.; Ivnik, R.J.; Tangalos, E.G.; Kokmen, E. Mild cognitive impairment: Clinical characterization and outcome. Arch. Neurol. 1999, 56, 303–308. [Google Scholar] [CrossRef] [PubMed]
  5. López-Sanz, D.; Bruña, R.; Garcés, P.; Martín-Buro, M.C.; Walter, S.; Delgado, M.L.; Montenegro, M.; López Higes, R.; Marcos, A.; Maestú, F. Functional connectivity disruption in subjective cognitive decline and mild cognitive impairment: A common pattern of alterations. Front. Aging Neurosci. 2017, 9, 109. [Google Scholar] [CrossRef]
  6. Villemagne, V.L.; Chételat, G. Neuroimaging biomarkers in Alzheimer’s disease and other dementias. Ageing Res. Rev. 2016, 30, 4–16. [Google Scholar] [CrossRef] [PubMed]
  7. Tabatabaei-Jafari, H.; Shaw, M.E.; Cherbuin, N. Cerebral atrophy in mild cognitive impairment: A systematic review with meta-analysis. Alzheimer’s Dement. Diagn. Assess. Dis. Monit. 2015, 1, 487–504. [Google Scholar] [CrossRef]
  8. Gauthier, S.; Reisberg, B.; Zaudig, M.; Petersen, R.C.; Ritchie, K.; Broich, K.; Belleville, S.; Brodaty, H.; Bennett, D.; Chertkow, H.; et al. Mild cognitive impairment. Lancet 2006, 367, 1262–1270. [Google Scholar] [CrossRef] [PubMed]
  9. Winblad, B.; Palmer, K.; Kivipelto, M.; Jelic, V.; Fratiglioni, L.; Wahlund, L.O.; Nordberg, A.; Bäckman, L.; Albert, M.; Almkvist, O.; et al. Mild cognitive impairment–beyond controversies, towards a consensus: Report of the International Working Group on Mild Cognitive Impairment. J. Intern. Med. 2004, 256, 240–246. [Google Scholar] [CrossRef]
  10. Mitchell, A.J.; Shiri-Feshki, M. Rate of progression of mild cognitive impairment to dementia–meta-analysis of 41 robust inception cohort studies. Acta Psychiatr. Scand. 2009, 119, 252–265. [Google Scholar] [CrossRef]
  11. Ansart, M.; Epelbaum, S.; Bassignana, G.; Bône, A.; Bottani, S.; Cattai, T.; Couronné, R.; Faouzi, J.; Koval, I.; Louis, M. Predicting the progression of mild cognitive impairment using machine learning: A systematic, quantitative and critical review. Med. Image Anal. 2021, 67, 101848. [Google Scholar] [CrossRef]
  12. Iraniparast, M.; Shi, Y.; Wu, Y.; Zeng, L.; Maxwell, C.J.; Kryscio, R.J.; St John, P.D.; SantaCruz, K.S.; Tyas, S.L. Cognitive reserve and mild cognitive impairment: Predictors and rates of reversion to intact cognition vs. progression to dementia. Neurology 2022, 98, e1114–e1123. [Google Scholar] [CrossRef]
  13. Giau, V.V.; Bagyinszky, E.; An, S.S.A. Potential fluid biomarkers for the diagnosis of mild cognitive impairment. Int. J. Mol. Sci. 2019, 20, 4149. [Google Scholar] [CrossRef]
  14. Busse, A.; Angermeyer, M.C.; Riedel-Heller, S.G. Progression of mild cognitive impairment to dementia: A challenge to current thinking. Br. J. Psychiatry 2006, 189, 399–404. [Google Scholar] [CrossRef] [PubMed]
  15. Guo, T.; Korman, D.; Baker, S.L.; Landau, S.M.; Jagust, W.J. Longitudinal Cognitive and Biomarker Measurements Support a Unidirectional Pathway in Alzheimer’s Disease Pathophysiology. Biol. Psychiatry 2021, 89, 786–794. [Google Scholar] [CrossRef]
  16. Roberts, R.O.; Knopman, D.S.; Mielke, M.M.; Cha, R.H.; Pankratz, V.S.; Christianson, T.J.H.; Geda, Y.E.; Boeve, B.F.; Ivnik, R.J.; Tangalos, E.G.; et al. Higher risk of progression to dementia in mild cognitive impairment cases who revert to normal. Neurology 2014, 82, 317–325. [Google Scholar] [CrossRef]
  17. Glynn, K.; O’Callaghan, M.; Hannigan, O.; Bruce, I.; Gibb, M.; Coen, R.; Green, E.; ALawlor, B.; Robinson, D. Clinical utility of mild cognitive impairment subtypes and number of impaired cognitive domains at predicting progression to dementia: A 20-year retrospective study. Int. J. Geriatr. Psychiatry 2021, 36, 31–37. [Google Scholar] [CrossRef] [PubMed]
  18. Pelka, O.; Friedrich, C.M.; Nensa, F.; Mönninghoff, C.; Bloch, L.; Jöckel, K.-H.; Schramm, S.; Sanchez Hoffmann, S.; Winkler, A.; Weimar, C. Sociodemographic data and APOE-ε4 augmentation for MRI-based detection of amnestic mild cognitive impairment using deep learning systems. PLoS ONE 2020, 15, e0236868. [Google Scholar] [CrossRef]
  19. Yeung, M.K.; Chau, A.K.-Y.; Chiu, J.Y.-C.; Shek, J.T.-L.; Leung, J.P.-Y.; Wong, T.C.-H. Differential and subtype-specific neuroimaging abnormalities in amnestic and nonamnestic mild cognitive impairment: A systematic review and meta-analysis. Ageing Res. Rev. 2022, 80, 101675. [Google Scholar] [CrossRef] [PubMed]
  20. Qu, Y.; Ma, Y.-H.; Huang, Y.-Y.; Ou, Y.-N.; Shen, X.-N.; Chen, S.-D.; Dong, Q.; Tan, L.; Yu, J.-T. Blood biomarkers for the diagnosis of amnestic mild cognitive impairment and Alzheimer’s disease: A systematic review and meta-analysis. Neurosci. Biobehav. Rev. 2021, 128, 479–486. [Google Scholar] [CrossRef]
  21. van Maurik, I.S.; Zwan, M.D.; Tijms, B.M.; Bouwman, F.H.; Teunissen, C.E.; Scheltens, P.; Wattjes, M.P.; Barkhof, F.; Berkhof, J.; van der Flier, W.M.; et al. Interpreting biomarker results in individual patients with mild cognitive impairment in the Alzheimer’s biomarkers in daily practice (ABIDE) project. JAMA Neurol. 2017, 74, 1481–1491. [Google Scholar] [CrossRef]
  22. Handels, R.L.H.; Vos, S.J.B.; Kramberger, M.G.; Jelic, V.; Blennow, K.; van Buchem, M.; van der Flier, W.; Freund-Levi, Y.; Hampel, H.; Rikkert, M.O. Predicting progression to dementia in persons with mild cognitive impairment using cerebrospinal fluid markers. Alzheimer’s Dement. 2017, 13, 903–912. [Google Scholar] [CrossRef] [PubMed]
  23. van Harten, A.C.; Visser, P.J.; Pijnenburg, Y.A.L.; Teunissen, C.E.; Blankenstein, M.A.; Scheltens, P.; van der Flier, W.M. Cerebrospinal fluid Aβ42 is the best predictor of clinical progression in patients with subjective complaints. Alzheimer’s Dement. 2013, 9, 481–487. [Google Scholar] [CrossRef] [PubMed]
  24. Davatzikos, C.; Bhatt, P.; Shaw, L.M.; Batmanghelich, K.N.; Trojanowski, J.Q. Prediction of MCI to AD conversion, via MRI, CSF biomarkers, and pattern classification. Neurobiol. Aging 2011, 32, 2322-e19. [Google Scholar] [CrossRef] [PubMed]
  25. Snyder, P.J.; Lim, Y.Y.; Schindler, R.; Ott, B.R.; Salloway, S.; Daiello, L.; Getter, C.; Gordon, C.M.; Maruff, P. Microdosing of scopolamine as a “cognitive stress test”: Rationale and test of a very low dose in an at-risk cohort of older adults. Alzheimer’s Dement. 2014, 10, 262–267. [Google Scholar] [CrossRef] [PubMed]
  26. Fleming, V.B.; Harris, J.L. Complex discourse production in mild cognitive impairment: Detecting subtle changes. Aphasiology 2008, 22, 729–740. [Google Scholar] [CrossRef]
  27. Hayes, J.; Flower, L. Identifying the Organization of Writing Processes. In Cognitive Processes in Writing; Gregg, L.W., Steinberg, E.R., Eds.; Lawrence Erlbaum Associates: Hillsdale, NJ, USA, 1980; pp. 3–30. [Google Scholar]
  28. Kim, H.; Walker, A.; Shea, J.; Hillis, A.E. Written Discourse Task Helps to Identify Progression from Mild Cognitive Impairment to Dementia. Dement. Geriatr. Cogn. Disord. 2021, 50, 446–453. [Google Scholar] [CrossRef] [PubMed]
  29. Kim, H.; Obermeyer, J.; Wiley, R.W. Written discourse in diagnosis for acquired neurogenic communication disorders: Current evidence and future directions. Front. Hum. Neurosci. 2024, 17, 1264582. [Google Scholar] [CrossRef] [PubMed]
  30. Ulatowska, H.K.; Olness, G.S.; Williams, L.J. Coherence of narratives in aphasia. Brain Lang. 2004, 91, 42–43. [Google Scholar] [CrossRef]
  31. Ulatowska, H.K.; Streit Olness, G.; Samson, A.M.; Keebler, M.W.; Goins, K.E. On the nature of personal narratives of high quality. Adv. Speech Lang. Pathol. 2004, 6, 3–14. [Google Scholar] [CrossRef]
  32. Goodglass, H.; Kaplan, E.; Barresi, B. BDAE-3: Boston Diagnostic Aphasia Examination, 3rd ed.; Lippincott Williams & Wilkins: Philadelphia, PA, USA, 2001. [Google Scholar]
  33. Themistocleous, C.; Neophytou, K.; Rapp, B.; Tsapkini, K. A tool for automatic scoring of spelling performance. J. Speech Lang. Hear. Res. 2020, 63, 4179–4192. [Google Scholar] [CrossRef]
  34. Themistocleous, C.; Eckerström, M.; Kokkinakis, D. Voice quality and speech fluency distinguish individuals with Mild Cognitive Impairment from Healthy Controls. PLoS ONE 2020, 15, e0236009. [Google Scholar] [CrossRef] [PubMed]
  35. Themistocleous, C.; Eckerström, M.; Kokkinakis, D. Identification of Mild Cognitive Impairment From Speech in Swedish Using Deep Sequential Neural Networks. Front. Neurol. 2018, 9, 975. [Google Scholar] [CrossRef] [PubMed]
  36. Fraser, K.C.; Lundholm Fors, K.; Kokkinakis, D. Multilingual word embeddings for the assessment of narrative speech in mild cognitive impairment. Comput. Speech Lang. 2019, 53, 121–139. [Google Scholar] [CrossRef]
  37. Hernández-Domínguez, L.; Ratté, S.; Sierra-Martínez, G.; Roche-Bergua, A. Computer-based evaluation of Alzheimer’s disease and mild cognitive impairment patients during a picture description task. Alzheimer’s Dement. Diagn. Assess. Dis. Monit. 2018, 10, 260–268. [Google Scholar] [CrossRef] [PubMed]
  38. Fraser, K.C.; Lundholm Fors, K.; Eckerström, M.; Themistocleous, C.; Kokkinakis, D. Improving the Sensitivity and Specificity of MCI Screening with Linguistic Information. In Proceedings of the LREC 2018 Workshop “Resources and ProcessIng of Linguistic, Para-Linguistic and Extra-Linguistic Data from People with Various Forms of Cognitive/Psychiatric Impairments (RaPID-2)”, Miyazaki, Japan, 8 May 2018; pp. 19–26. [Google Scholar]
  39. König, A.; Satt, A.; Sorin, A.; Hoory, R.; Toledo-Ronen, O.; Derreumaux, A.; Manera, V.; Verhey, F.; Aalten, P.; Robert, P.H.; et al. Automatic speech analysis for the assessment of patients with predementia and Alzheimer’s disease. Alzheimer’s Dement. Diagn. Assess. Dis. Monit. 2015, 1, 112–124. [Google Scholar] [CrossRef]
  40. Calzà, L.; Gagliardi, G.; Rossini Favretti, R.; Tamburini, F. Linguistic features and automatic classifiers for identifying mild cognitive impairment and dementia. Comput. Speech Lang. 2021, 65, 101113. [Google Scholar] [CrossRef]
  41. López-de-Ipiña, K.; Solé-Casals, J.; Eguiraun, H.; Alonso, J.B.; Travieso, C.M.; Ezeiza, A.; Barroso, N.; Ecay-Torres, M.; Martinez-Lage, P.; Beitia, B. Feature selection for spontaneous speech analysis to aid in Alzheimer’s disease diagnosis: A fractal dimension approach. Comput. Speech Lang. 2015, 30, 43–60. [Google Scholar] [CrossRef]
  42. Toledo, C.M.; Aluísio, S.M.; Dos Santos, L.B.; Brucki, S.M.D.; Trés, E.S.; de Oliveira, M.O.; Mansur, L.L. Analysis of macrolinguistic aspects of narratives from individuals with Alzheimer’s disease, mild cognitive impairment, and no cognitive impairment. Alzheimer’s Dement. Diagn. Assess. Dis. Monit. 2018, 10, 31–40. [Google Scholar] [CrossRef] [PubMed]
  43. Clarke, N.; Barrick, T.R.; Garrard, P. A comparison of connected speech tasks for detecting early Alzheimer’s disease and mild cognitive impairment using natural language processing and machine learning. Front. Comput. Sci. 2021, 3, 634360. [Google Scholar] [CrossRef]
  44. Fraser, K.C.; Meltzer, J.A.; Graham, N.L.; Leonard, C.; Hirst, G.; Black, S.E.; Rochon, E. Automated classification of primary progressive aphasia subtypes from narrative speech transcripts. Cortex 2014, 55, 43–60. [Google Scholar] [CrossRef]
  45. Themistocleous, C.; Ficek, B.; Webster, K.; den Ouden, D.-B.; Hillis, A.E.; Tsapkini, K. Automatic subtyping of individuals with Primary Progressive Aphasia. bioRxiv 2020. bioRxiv:2020.04.04.025593. [Google Scholar] [CrossRef]
  46. Yan, J.H.; Rountree, S.; Massman, P.; Doody, R.S.; Li, H. Alzheimer’s disease and mild cognitive impairment deteriorate fine movement control. J. Psychiatr. Res. 2008, 42, 1203–1212. [Google Scholar] [CrossRef] [PubMed]
  47. Jack, C.R., Jr.; Bennett, D.A.; Blennow, K.; Carrillo, M.C.; Dunn, B.; Haeberlein, S.B.; Holtzman, D.M.; Jagust, W.; Jessen, F.; Karlawish, J. Research Framework: Toward a biological definition of Alzheimer’s disease. Alzheimers Dement. 2018, 14, 535–562. [Google Scholar] [CrossRef] [PubMed]
  48. Folstein, M.F.; Folstein, S.E.; McHugh, P.R. “Mini-mental state”: A practical method for grading the cognitive state of patients for the clinician. J. Psychiatr. Res. 1975, 12, 189–198. [Google Scholar] [CrossRef] [PubMed]
  49. Wechsler, D. WAIS-III: Wechsler Adult Intelligence Scale, 3rd ed.; The Psychological Corporation: Harcourt Brace & Company San Antonio: San Antonio, TX, USA, 1997. [Google Scholar]
  50. Rey, A. L’examen psychologique dans les cas d’encephalopathie traumatique. (Les problems.) The psychological examination in cases of traumatic encepholopathy. (Problem.). Arch. Psychol. 2014, 28, 215–285. [Google Scholar]
  51. Reitan, R.M.; Wolfson, D. A selective and critical review of neuropsychological deficits and the frontal lobes. Neuropsychol. Rev. 1994, 4, 161–198. [Google Scholar] [CrossRef] [PubMed]
  52. Trenerry, M.R.; Crosson, B.A.; DeBoe, J.; Leber, W.R. Stroop Neuropsychological Screening Test; Psychological Assessment Resources: Lutz, FL, USA, 1989. [Google Scholar]
  53. Themistocleous, C. Open Brain AI: An AI Research Platform. In Proceedings of the Huminfra Conference, Gothenburg, Sweden, 10–11 January 2024; Volodina, E., Bouma, G., Forsberg, M., Kokkinakis, D., Alfter, D., Fridlund, M., Eds.; Linköping Electronic Conference Proceedings: Gothenburg, Sweden, 2024; Volume 205, pp. 1–9. [Google Scholar]
  54. Themistocleous, C.; Webster, K.; Afthinos, A.; Tsapkini, K. Part of Speech Production in Patients With Primary Progressive Aphasia: An Analysis Based on Natural Language Processing. Am. J. Speech-Lang. Pathol. 2020, 30, 1–15. [Google Scholar]
  55. Devlin, J.; Chang, M.-W.; Lee, K.; Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv 2018, arXiv:181004805. [Google Scholar]
  56. Menardi, G.; Torelli, N. Training and assessing classification rules with imbalanced data. Data Min. Knowl. Discov. 2014, 28, 92–122. [Google Scholar] [CrossRef]
  57. Torres, V.L.; Rosselli, M.; Loewenstein, D.A.; Curiel, R.E.; Uribe, I.V.; Lang, M.; Arruda, F.; Penate, A.; Vaillancourt, D.E.; Greig, M.T.; et al. Types of Errors on a Semantic Interference Task in Mild Cognitive Impairment and Dementia. Neuropsychology 2019, 33, 670–684. [Google Scholar] [CrossRef] [PubMed]
  58. Buschke, H.; Mowrey, W.B.; Ramratan, W.S.; Zimmerman, M.E.; Loewenstein, D.A.; Katz, M.J.; Lipton, R.B. Memory Binding Test Distinguishes Amnestic Mild Cognitive Impairment and Dementia from Cognitively Normal Elderly. Arch. Clin. Neuropsychol. Off. J. Natl. Acad. Neuropsychol. 2017, 32, 29–39. [Google Scholar] [CrossRef] [PubMed]
  59. Jester, D.J.; Andel, R.; Cechov, K. Cognitive phenotypes of older adults with subjective cognitive decline and amnestic mild cognitive impairment: The Czech Brain Aging Study. J. Int. Neuropsychol. Soc. 2021, 27, 329–342. [Google Scholar] [CrossRef] [PubMed]
  60. Kwak, K.; Giovanello, K.S.; Bozoki, A.; Styner, M.; Dayan, E. Subtyping of mild cognitive impairment using a deep learning model based on brain atrophy patterns. Cell Rep. Med. 2021, 2, 100467. [Google Scholar] [CrossRef]
  61. McLane, H.C.; Berkowitz, A.L.; Patenaude, B.N.; McKenzie, E.D.; Wolper, E.; Wahlster, S.; Mateen, F.J. Availability, accessibility, and affordability of neurodiagnostic tests in 37 countries. Neurology 2015, 85, 1614–1622. [Google Scholar] [CrossRef]
  62. Beeson, P.; Rapcsak, S. Clinical Diagnosis and Treatment of Spelling Disorders. In The Handbook of Adult Language Disorders; Hillis, A., Ed.; Psychological Press: London, UK, 2015; pp. 145–170. [Google Scholar]
  63. He, R.; Chapin, K.; Al-Tamimi, J.; Bel, N.; Marqui, M. Automated classification of cognitive decline and probable Alzheimer’s dementia across multiple speech and language domains. Am. J. Speech-Lang. Pathol. 2023, 32, 2075–2086. [Google Scholar] [CrossRef] [PubMed]
  64. Love, T.; Oster, E. On the categorization of aphasic typologies: The SOAP (a test of syntactic complexity). J. Psycholinguist Res. 2002, 31, 503–529. [Google Scholar] [CrossRef] [PubMed]
  65. Pekkala, S.; Wiener, D.; Himali, J.; Beiser, A.; Obler, L.; Liu, Y. Lexical retrieval in discourse: An early indicator of Alzheimer’s dementia. Clin. Linguist. Phon. 2013, 27, 905–921. [Google Scholar] [CrossRef] [PubMed]
  66. Forbes-McKay, K.; Shanks, M.; Venneri, A. Charting the decline in spontaneous writing in Alzheimer’s disease: A longitudinal study. Acta Neuropsychiatr. 2014, 26, 246–252. [Google Scholar] [CrossRef] [PubMed]
  67. Mikolov, T.; Chen, K.; Corrado, G.; Dean, J. Efficient Estimation of Word Representations in Vector Space. arXiv 2013, arXiv:1301.3781. [Google Scholar]
  68. Pennington, J.; Socher, R.; Manning, C.D. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, 25–29 October 2014. [Google Scholar]
  69. Chang, H.T.; Chiu, M.J.; Chen, T.F.; Liu, M.Y.; Fan, W.C.; Cheng, T.W.; Hua, M.S. Deterioration and predictive values of semantic networks in mild cognitive impairment. J. Neurolinguistics 2022, 61, 101025. [Google Scholar] [CrossRef]
  70. Juncos-Rabadn, O.; Facal, D.; Lojo-Seoane, C.; Pereiro, A.X. Tip-of-the-tongue for proper names in non-amnestic mild cognitive impairment. J. Neurolinguistics 2013, 26, 409–420. [Google Scholar] [CrossRef]
  71. Mueller, K.D.; Hermann, B.; Mecollari, J.; Turkstra, L.S. Connected speech and language in mild cognitive impairment and Alzheimer’s disease: A review of picture description tasks. J. Clin. Exp. Neuropsychol. 2018, 40, 917–939. [Google Scholar] [CrossRef]
Brainsci 14 00652 g001
Figure 1. The machine learning (ML) classification process used to differentiate between amnestic MCI (aMCI) and non-amnestic MCI (naMCI). Linguistic measures extracted from written picture descriptions were combined with contextualized word embeddings generated by BERT. The features were used to train and evaluate five ML models: Random Forest (RF), Gradient Boosting (GB), Histogram-based Gradient Boosting (HGB), XGBoost (XGB), and LightGBM (LGBM). The optimal model was selected after training and through hyperparameter tuning and comparative performance analysis.
Figure 1. The machine learning (ML) classification process used to differentiate between amnestic MCI (aMCI) and non-amnestic MCI (naMCI). Linguistic measures extracted from written picture descriptions were combined with contextualized word embeddings generated by BERT. The features were used to train and evaluate five ML models: Random Forest (RF), Gradient Boosting (GB), Histogram-based Gradient Boosting (HGB), XGBoost (XGB), and LightGBM (LGBM). The optimal model was selected after training and through hyperparameter tuning and comparative performance analysis.
Brainsci 14 00652 g001
Brainsci 14 00652 g002
Figure 2. MLs performance on the classification task: individuals with aMCI vs. individuals with naMCI from language measures.
Figure 2. MLs performance on the classification task: individuals with aMCI vs. individuals with naMCI from language measures.
Brainsci 14 00652 g002
Table 1. Participants’ age and education across variants (amnestic and non-amnestic) and gender.
Table 1. Participants’ age and education across variants (amnestic and non-amnestic) and gender.
VariantGenderNMeanSDMedianMode
AgeAmnesticF7167.412.997053
M5369.715.287469
Non AmnesticF2154.213.485248
M2565.612.046665
EducationAmnesticF7016.13.191616
M5217.53.421816
Non AmnesticF2115.53.531616
M24163.0616.512
Table 2. Performance in neurocognitive tests in individuals with MCI.
Table 2. Performance in neurocognitive tests in individuals with MCI.
VariantMeanMedianModeSD
MMSEAmnestic27.5081 28281.746
Non-amnestic 28.047629291.821
WMSAmnestic13.2514140.942
Non-amnestic13.68041414 0.592
Digit forwardAmnestic6.7016771.169
Non-amnestic6.7391761.437
Digit backwardAmnestic4.2984441.044
Non-amnestic4.4565441.187
RAVLT (total)Amnestic 29.217729309.373
Non-amnestic37.8587373711.187
RAVLT (delayed)Amnestic3.5081332.95
Non-amnestic6.8333773.151
RCF (immediate)Amnestic7.8487705.934
Non-amnestic14.54351268.989
RCF (delayed)Amnestic6.2391505.25
Non-amnestic 13.173912.2508.568
BNTAmnestic49.20335256 10.265
Non-amnestic52.282654567.12
Verbal fluency (FAS)Amnestic35.5772353213.073
Non-amnestic34.326132.52312.994
BDAE writingAmnestic4.1441443.733
Non-amnestic3.7778440.56
TMT AAmnestic55.221848.53031.634
Non-amnestic45.599336.52524.149
TMT A errorAmnestic0.042000.302
Non-amnestic0.087000.354
TMT BAmnestic132.831911311099.71
Non-amnestic121.9254965775.288
TMT B error Amnestic 0.5439001.863
Non-amnestic 0.3696000.878
ColorAmnestic111.71681121122.647
Non-amnestic110.551121128.852
Color(Word)Amnestic67.26611229.593
Non-amnestic68.8158 64.5 11225.877
MMSE = Mini-Mental State Examination; WMS = Wechsler Memory Scale; RAVLT (total) = total score of the Rey Auditory Verbal Learning Test; RAVLT (delayed) = score for the delayed recall of the Rey Auditory Verbal Learning Test; (delayed recall); RCF (immediate) = score for the immediate recall of the Rey Complex Figure; RCF (delayed) = score for the delayed recall of the Rey Complex Figure; BNT = Boston Naming Test; BDAE writing = free narrative writing from the Boston Diagnostic Aphasia Examination; TMT A = Trail Making Test Part A; TMT A error = Errors made in the Trail Making Test Part A; TMT B = Trail Making Test Part B; TMT B error = Errors made in Trail Making Test Part B; Color = Color Stroop test; Color(Word) = Color Stroop and Word Tests; SD: Single-domain MCI.
Table 3. Means and standard deviations of features in individuals with non-amnestic and amnestic MCI.
Table 3. Means and standard deviations of features in individuals with non-amnestic and amnestic MCI.
Non-AmnesticAmnestic
MeanSDMeanSD
Adjectival clause0.0210.0540.0140.031
Adjectival complement0.0070.0160.0090.017
Adjective0.0220.0320.0280.033
Adposition0.0970.0540.1130.058
Adverb0.0130.0220.0150.025
Adverbial clause0.0220.0300.0210.031
Adverbial modifier0.0120.0210.0140.024
Agent0.0000.0040.0000.002
Adjectival modifier0.0210.0310.0170.033
Apposition0.0040.0160.0030.013
Attribute0.0030.0070.0020.007
Auxiliary0.0800.0600.0750.065
Auxiliary (passive)0.0010.0050.0020.007
Case marking0.0020.0080.0020.007
Coordinating conjunction0.0190.0260.0180.026
Clausal complement0.0200.0330.0210.035
Coordinating conjunction0.0190.0260.0180.026
Character–word ratio5.2440.4105.2560.517
Compound0.0320.0440.0340.057
Conjunction0.0200.0280.0200.028
Dative case0.0020.0080.0040.013
Dependent0.0460.0950.0320.056
Determiner0.1250.0860.1100.088
Direct object0.0860.0450.0840.067
Expletive0.0030.0070.0010.006
Interjection0.0010.0080.0010.006
Marker0.0180.0280.0070.016
Meta data0.0000.0040.0100.056
Negation modifier0.0050.0120.0040.012
Noun0.3620.1090.3760.119
Nominal subject0.1370.0550.1420.057
Nominal subject (passive)0.0010.0050.0020.007
Numeral0.0040.0120.0040.013
Numeric modifier0.0030.0110.0040.012
Object predicate000.0010.009
Parataxis000.0000.002
Particle0.0260.0260.0250.029
Prepositional complement0.0000.0020.0020.008
Prepositional object0.0780.0520.0960.052
Possessive modifier0.0110.0210.0110.021
Preposition0.0830.0570.0990.058
Pronoun0.0370.0410.0270.033
Proper noun0.0030.0140.0030.012
Particle0.0120.0170.0110.018
Punctuation0.1110.0730.1040.081
Relative clause0.0040.0100.0040.010
Root0.1050.0560.0990.061
Subordinating conjunction0.0180.0280.0080.017
Symbol0.0000.0030.0010.007
Verb0.1960.0710.1920.057
Other0.0020.0130.0100.051
Open clausal complement0.0150.0210.0180.024
Words [count] 131.94315.31829.65014.205
Characters [counts] 1170.92778.647158.82970.947
Note: All measures indicate the count/total word; features marked with the index (1) are counts.
Table 4. Model performance in the classification task: individuals with aMCI vs. individuals with naMCI from language measures.
Table 4. Model performance in the classification task: individuals with aMCI vs. individuals with naMCI from language measures.
RFGBHGBXGBLGBM
Accuracy0.900.900.890.890.89
F10.710.720.700.710.70
Precision0.740.750.750.750.75
Recall0.680.700.670.680.66
ROC/AUC0.980.970.980.970.98
Note: RF: Random Forest; GB: Gradient Boosting; HGB: Histogram Gradient Boosting; XGB: XGBoost; and LGBM: LightGBM.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Kim, H.; Hillis, A.E.; Themistocleous, C. Machine Learning Classification of Patients with Amnestic Mild Cognitive Impairment and Non-Amnestic Mild Cognitive Impairment from Written Picture Description Tasks. Brain Sci. 2024, 14, 652. https://doi.org/10.3390/brainsci14070652

AMA Style

Kim H, Hillis AE, Themistocleous C. Machine Learning Classification of Patients with Amnestic Mild Cognitive Impairment and Non-Amnestic Mild Cognitive Impairment from Written Picture Description Tasks. Brain Sciences. 2024; 14(7):652. https://doi.org/10.3390/brainsci14070652

Chicago/Turabian Style

Kim, Hana, Argye E. Hillis, and Charalambos Themistocleous. 2024. "Machine Learning Classification of Patients with Amnestic Mild Cognitive Impairment and Non-Amnestic Mild Cognitive Impairment from Written Picture Description Tasks" Brain Sciences 14, no. 7: 652. https://doi.org/10.3390/brainsci14070652

APA Style

Kim, H., Hillis, A. E., & Themistocleous, C. (2024). Machine Learning Classification of Patients with Amnestic Mild Cognitive Impairment and Non-Amnestic Mild Cognitive Impairment from Written Picture Description Tasks. Brain Sciences, 14(7), 652. https://doi.org/10.3390/brainsci14070652

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop