Predicting the Conversion from Mild Cognitive Impairment to Alzheimer’s Disease Using an Explainable AI Approach

Abstract

Mild Cognitive Impairment (MCI) is a cognitive state frequently observed in older adults, characterized by significant alterations in memory, thinking, and reasoning abilities that extend beyond typical cognitive decline. It is worth noting that around 10–15% of individuals with MCI are projected to develop Alzheimer’s disease, effectively positioning MCI as an early stage of Alzheimer’s. In this study, a novel approach is presented involving the utilization of eXtreme Gradient Boosting to predict the onset of Alzheimer’s disease during the MCI stage. The methodology entails utilizing data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI). Through the analysis of longitudinal data, spanning from the baseline visit to the 12-month follow-up, a predictive model was constructed. The proposed model calculates, over a 36-month period, the likelihood of progression from MCI to Alzheimer’s disease, achieving an accuracy rate of 85%. To further enhance the precision of the model, this study implements feature selection using the Recursive Feature Elimination technique. Additionally, the Shapley method is employed to provide insights into the model’s decision-making process, thereby augmenting the transparency and interpretability of the predictions.

1. Introduction

Mild Cognitive Impairment (MCI) is a clinical state marked by subtle yet quantifiable declines in an individual’s cognitive capabilities. These declines are more pronounced than the typical age-related changes but do not reach a severity level that substantially disrupts an individual’s daily life or independence. Such cognitive declines involve mild impairments in memory, cognitive skills, and cognitive functions [1]. Research has shown that MCI may serve as an indicator of Alzheimer’s disease (AD), with approximately 10–15% of individuals with MCI progressing to AD annually [2]. According to recent statistics, an estimated 6.7 million Americans aged 65 and older are currently living with Alzheimer’s dementia, with this number projected to reach 13.8 million by 2060 unless significant medical breakthroughs for prevention, slowing, or curing Alzheimer’s disease are achieved [3]. Despite years of clinical research, there is still no cure for AD. This makes knowledge of MCI presence a crucial factor for predicting an individual’s risk of developing AD and, in turn, enables the timely administration of effective treatment or intervention to slow down its progression [4]. To address this issue, we need to distinguish MCI patients who convert to AD (MCIc) from MCI patients who remain stable (MCInc) within a specific period. To accurately distinguish those two types of MCI (MCIc vs. MCInc), multiple studies have been conducted from multiple perspectives, such as genetics, medical imaging, and pathology [5]. At present, varying perspectives exist on the efficacy of biomarkers in faithfully depicting the progression of preclinical disease over time [6,7].

Traditionally, addressing such problems, such as distinguishing MCIc (MCI due to AD) from MCInc patients, has been performed manually by clinicians using specific guidelines. Recent advances in machine learning can help experts automate this process, resulting in lower error rates and speeding up the diagnostic process. In the field of AD research, the development of such algorithms tailored for automatic diagnosis and the prediction of an individual’s future clinical status based on biomarkers has become increasingly prevalent [8]. Studies have shown that the combination of machine learning techniques and the analysis of large amounts of heterogenous data collected with non-invasive techniques may be the future for precisive AD diagnosis. These datasets are sourced from a wide range of inputs, including medical imaging, biosensors for biofluid marker detection, and movement sensors, among others [9].

Furthermore, machine learning can assist in identifying cost-effective biomarkers for predicting the progression of Alzheimer’s disease in an efficient way. This is especially valuable given that current diagnostic methods depend on costly techniques [10]. Several novel biomarkers have shown promise in detecting Alzheimer’s disease (AD). These include neurofilament light (NFL) as a biomarker for neuronal injury, neurogranin, BACE1, synaptotagmin, SNAP-25, GAP-43, and synaptophysin as biomarkers related to synaptic dysfunction and/or loss, and sTREM2 and YKL-40 as biomarkers associated with neuroinflammation. Additionally, D-glutamate, a coagonist for NMDARs, has shown correlation with cognitive impairment and potential as a peripheral biomarker for detecting MCI and AD [10].

Through the analysis of such biomarker data, including Aβ-amyloid, neurofilament light, and BACE1, and the application of machine learning methodologies, we can create models that effectively learn patterns to distinguish between healthy individuals and those with Alzheimer’s disease [10]. When it comes to Aβ-amyloid positivity, machine learning can help predict Aβ-amyloid future presence in the non-demented population using data obtained with non-invasive techniques to improve screening processes in clinical trials [11]. Other machine learning approaches include the analysis of T1-weighted MRI scans to identify the presence of sensitive biomarkers to critical brain regions related to potential AD development to help distinguish MCIc from MCInc patients, resulting in an accuracy of 66% [12]. Radiomic feature extraction from medical images can also help predict AD and MCI in combination with random forest algorithms that can reach accuracies up to 73% [13]. Additionally, there have been approaches that classify dementia into three stages, including MCI-related conditions, which have shown promising prediction results [14].

However, hybrid approaches, which incorporate longitudinal and heterogeneous data along with supervised and unsupervised methods, have proven to be more effective under certain circumstances and generalize better, achieving results with up to 85% accuracy, particularly in distinguishing between MCI progressive from non-progressive cases [15]. Such approaches also extend to deep learning, a subset of machine learning, which combines convolutional neural networks and long short-term memory algorithms using multimodal imaging and cognitive tests to classify healthy individuals from those with early MCI (EMCI), with an accuracy of 98.5% [16].

The explainability of these models is crucial for understanding how predictions are made and, in turn, for pinpointing the key factors driving the progression of AD. This is particularly important when dealing with blackbox AI systems. Explainable model methods, such as Shapley Values, offer promising insights into individual model predictions, showcasing how specific features influence outcomes and revealing inter-feature relationships [17]. Particularly, XGBoost has demonstrated promising interpretability by utilizing Shapley Values to uncover specific biomarkers pivotal in the progression likelihood of Alzheimer’s disease [18].

In this light, we propose an explainable machine learning solution, which aims to predict cognitive decline in individuals, with a focus on distinguishing instances of MCI stable cases and MCI to AD cases within a 48-month timeframe from the initial examination with a specialized medical professional. Given that nearly all cases of AD originate from MCI, our goal is to discern which instances of MCI will progress to AD in the medium term (in less than 48 months) or will have a longer timeline before doing so (exceeding the 48-month timeframe). The importance of this prediction is high, as it can inform the healthcare professional of a potential worsening in the patient’s condition and can lead to earlier pharmacologic or non-pharmacologic intervention to delay the transition. Also, the explainability that is provided can help the treating physician identify the worsening cognition parameters and specifically target them. While diagnosing MCI can be challenging, its identification serves as an early indicator of potential progression to Alzheimer’s disease. Consequently, the development of a predictive model for MCI progression holds immense promise, offering significant advantages in both research and practical applications. Although there is significant literature in the application of AI techniques in the ADNI repository, there are only a few that focus on developing progression models, while most focus on diagnostic models. In addition, most of the previous research works utilize individual ADNI cohorts (ADNI1 or ADNI2 or ADNI3 or ADNI GO), depending on the features required for analysis. In our case, we used all four ADNI cohorts in an attempt to have a more complete cohort and increase the significance of the obtained results.

2. Materials and Methods

Data used in the preparation of this study were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). The ADNI was launched in 2003 as a public–private partnership, led by Principal Investigator Michael W. Weiner, MD. The primary goal of ADNI has been to test whether serial magnetic resonance imaging (MRI), positron emission tomography (PET), other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of Mild Cognitive Impairment (MCI) and early Alzheimer’s disease (AD).

2.1. Data and Preprocessing

We used the ADNIMERGE dataset to train and evaluate our proposed model. ADNIMERGE is an R package that combines trivial key predictors from all four phases of the ADNI dataset project coded for applying data analysis and machine learning techniques. The ADNI (Alzheimer’s Disease Neuroimaging Initiative) dataset is a comprehensive and longitudinally collected collection of clinical, imaging, genetic, and cognitive data from participants with Alzheimer’s disease, Mild Cognitive Impairment, and healthy controls. The principal goal of the Alzheimer’s Disease Neuroimaging Initiative (ADNI) is to explore and identify biomarkers that can be employed in clinical trials aimed at advancing the development of treatments for Alzheimer’s Disease [19,20].

The initial dataset comprises a cohort of 2378 patients, encompassing participants across all ADNI phases. The dataset incorporates a comprehensive set of 64 features, encompassing a diverse range of information from various domains. These features encompass both biological data, such as APOE4 status, and imaging data, including FDG-PET scans. Additionally, screening tests such as the Mini-Mental State Examination (MMSE) and Everyday Cognition Scale are incorporated into the dataset.

A brief description of subjects’ demographics used in this study is shown in Figure 1. This study predominantly involved male participants, averaging 73 years in age. Furthermore, most of the subjects identified as Caucasian and were married.

Given the longitudinal nature of the dataset, each patient is represented by multiple entries corresponding to distinct time points, aligning with the data collection protocol employed in the respective ADNI phase. Most diagnoses and visits are concentrated in the early months subsequent to the initial visit, with a discernible decline over time. One plausible interpretation of this pattern is that participants display a heightened level of compliance and engagement at the outset of the clinical trial, which diminishes as the study progresses. Notably, despite the diminishing frequency of visits over time, there is a consistent adherence to annual intervals, occurring in multiples of 12 months (e.g., 24 months, 36 months, etc.).

Since there is a high consistency for annual visits, we selected the baseline and follow-up visit records to predict the progression to AD from MCI within a 24- to 48-month timeframe. The refined subset then comprised 918 subjects, among whom 636 exhibited stability throughout the 48-month observation period, while 282 transitioned to Alzheimer’s disease (AD) within the predictive window. A comprehensive overview of disease progression within our curated dataset is depicted in the Sankey Diagram in Figure 2.

Missing values were also present in the dataset. Figure 3 illustrates an overview of missing values for baseline visits.

Considering the strict clinical guidelines defined by ADNI, we consider missing data entries as “Missing Not at Random” (MNAR). This designation signifies that data absence is not a random occurrence but rather stems from reasons related to the observed data, resulting in unanticipated lapses in data collection. Specifically, certain features were not obtained at baseline and follow-up timepoints, with the anticipation that these would be recorded at subsequent visits. One exception to this assumption is scales related to “Every Day Cognition Tests” and the “Montreal Cognitive Assessment (MOCA)”, which were not gathered during the ADNI1 phase and, consequently, were treated as missing during the training process [21]. For the remaining missing data, the k-nearest neighbors (KNN) imputation method with k = 5 was employed, considering the target class [22]. Notably, ADNI1 was imputed separately from the rest of the dataset to mitigate potential imputation bias, given the absence of data for the scales mentioned above during this phase. To enhance the representation of categorical variables, one-hot encoding was applied.

We modeled the progression from MCI to AD as a binary feature, assigning a value of “0” denoting the likelihood of the patient remaining stable in the MCI state over the subsequent 36 months and a value of “1” indicating the likelihood of transitioning to AD within the same timeframe. (See Figure 4 for a visual depication of the prediction window). This determination was made by analyzing the diagnosis records of each patient within the cohort utilized in this study to represent the longitudinal progression of the disease in one feature. It is worth noting that in some instances, patient visits were recorded under different ADNI protocols (e.g., some visits occurred during the transition from ADNI 1 to ADNI 2).

2.1.1. Model Selection

We employed a comprehensive approach to model selection by comparing the performance of various machine learning algorithms: XGBoost, CatBoost, LightGBM, Logistic Regression, Naïve Bayes, and Decision Tree. The goal was to identify the model that best captures the underlying patterns in the data and demonstrates robust generalization capabilities. To achieve this, we conducted a thorough evaluation using a 5-fold cross-validation strategy and evaluated each model’s performance.

2.1.2. XGBoost

XGBoost stands for eXtreme Gradient Boosting, and it is a popular machine learning algorithm that falls under the category of ensemble classifiers. The fundamental principle behind the algorithm involves the iterative incorporation of Decision Trees by learning the negative gradient of the loss function, computed with respect to the disparity between the predicted value from the preceding tree and the actual value. It systematically performs feature splitting to grow the ensemble tree. The algorithm leverages the second-order derivative of the loss function to learn the negative gradient. It also has regularization capabilities by using a penalty term to prevent overfitting and improve generalization. Additionally, the algorithm addresses class imbalance by computing the inverse ratio of positive to negative classes, employing it as a weighting operator for each class [23].

2.1.3. CatBoost

CatBoost, introduced by Prokhorenkova et al. in 2018 [24], is a gradient boosting algorithm known for its proficiency in handling categorical features, with minimal information loss. CatBoost introduces two innovative methods, contributing to its state-of-the-art performance. Firstly, it addresses the challenge of categorical features by employing a unique encoding approach, which involves dividing and substituting categories with several numeric features, thus minimizing information loss. This process involves random sorting of input samples and calculating average values for each category. Secondly, CatBoost introduces Ordered Boosting to overcome the gradient estimation bias in traditional Gradient Boosting Trees (GBDTs). This method utilizes sorting to generate a random arrangement order and number the dataset, enabling unbiased gradient estimation. Though it involves training a model for each sample, which increases space complexity, CatBoost optimizes Ordered Boosting and retains the foundational concepts of GBDT to improve model generalization and stability [25,26].

2.1.4. Light Gradient Boosting Machine

The Light Gradient Boosting Machine (LGBM) is another Gradient Boosting algorithm that uses leaf-wise techniques to grow trees vertically. To enhance the training process, LGBM uses the Gradient-based-One-Side-Sampling algorithm (GOSS) that is designed to emphasize the significance of data instances. GOSS focuses on data samples exhibiting larger gradients while intentionally disregarding samples with lower gradients, assuming that they have lower errors. To mitigate potential bias towards samples with larger gradients, GOSS performs random sampling on data instances with small gradients while retaining all samples with large ones. Additionally, GOSS introduces a mechanism for adjusting the weights of data samples with small gradients during the computation of information gain, aiming to rectify and account for the inherent bias in the dataset [27].

2.1.5. Standard Machine Learning Classifiers

We compared the Gradient Boosting algorithms against standard machine learning classifiers, such as Naive Bayes, Decision Trees, and Logistic Regression. By contrasting these advanced techniques with simpler, more traditional approaches, we aimed to provide a comprehensive understanding of the performance differences between them. Naive Bayes, recognized for its straightforward probabilistic approach, Decision Trees, prized for their intuitive structure and ease of interpretation, and Logistic Regression, esteemed for its linear model formulation, serve as benchmarks in this discourse. Comparing these standard classifiers against Gradient Boosting algorithms allows for a comprehensive evaluation of predictive performance, scalability, and computational efficiency.

2.1.6. Experiment Setup and Model Comparison

Instances of each model were trained on the dataset delineated in Section Shapley Additive Explanations using the default parameters. The implementation was conducted utilizing Python 3.10 and the scikit-learn library. Subsequently, 5-fold cross-validation was employed across all models, and the resultant average metrics encompassing accuracy, precision, and recall were scrutinized for comparative analysis. The selection of the most suitable model for our dataset was based on these metrics. Detailed findings of this comparative evaluation are presented in

Table 1. Machine learning algorithm comparison.

Algorithm	Class	Precision	Recall	Accurary
XGBoost	Stable	0.90	0.89	0.86
	Transition	0.79	0.77
CatBoost	Stable	0.90	0.88	0.85
	Transition	0.77	0.76
Light Gradient Boosting	Stable	0.89	0.90	0.86
	Transition	0.76	0.75
Decision Tree	Stable	0.85	0.86	0.79
	Transition	0.67	0.66
Logistic Regression	Stable	0.70	0.91	0.66
	Transition	0.32	0.11
Naive Bayes	Stable	0.87	0.62	0.66
	Transition	0.47	0.78

.

The comparative analysis of machine learning algorithms demonstrates that XGBoost is the most effective performer, exhibiting superior precision and recall scores of 0.79 and 0.77, respectively, alongside an accuracy of 0.86. CatBoost and Light Gradient Boosting closely follow with similar precision, recall, and accuracy metrics, achieving 0.77, 0.76, and 0.85, and 0.76, 0.75, and 0.86 respectively. Decision Tree exhibits moderate performance, with precision and recall rates of 0.67 and 0.66, resulting in an accuracy of 0.79. However, Logistic Regression shows notably lower precision and recall values at 0.32 and 0.11, respectively, yielding an accuracy of 0.66. Naive Bayes, while displaying satisfactory recall at 0.78, demonstrates lower precision at 0.47, leading to an accuracy of 0.66, showcasing its precision–recall trade-offs compared to other algorithms.

2.2. Training Pipeline

After selecting the best model, we performed the training process using the workflow presented in Figure 5. We first trained an XGBoost instance using the default parameters to perform feature selection using the Recursive Feature Elimination algorithm (RFE). This is a powerful wrapper feature selection algorithm first introduced by Guyon et al. 2002. RFE removes iteratively weak features whose removal has the lowest impact on training errors and keeps strong features to improve the model generalization. The weakness of each feature is determined by a criterion based on the model that the algorithm wraps. In most cases, the weakness criterion is the feature importance scores assigned by the model during training. During each iteration, RFE selectively removes a specified number (N) of features based on their perceived weakness, as indicated by the importance criterion. Following the removal of features, the algorithm retrains the model on the new set of features and assesses its performance using a predetermined evaluation metric. This iterative process is repeated until there is no further improvement in model performance, at which point the algorithm concludes and returns the set of selected features [28].

In our case, we used RFE combined with 5-fold cross-validation for evaluation purposes. We adopted a stepwise approach with the step size set to 1, indicating the removal of one feature at each iteration during the Recursive Feature Elimination with Cross-Validation (RFE-CV) process. Our evaluation criteria were based on the F1 score, as defined in Equation (1). The F1 score, encompassing both precision (Equation (2)) and recall (Equation (3)), is particularly instrumental in comprehensively assessing the performance of the model. To perform feature selection, we utilized the RFE-CV technique implemented in the scikit-learn Python package. The workflow and outcomes of our RFE-CV implementation are depicted in Figure 6, illustrating the systematic elimination of features and the corresponding impact on the model’s F1 score. This approach provides a robust means of feature selection, allowing us to identify the optimal subset of features that maximizes the model’s predictive performance.

$$F_{1}score=2\times \frac{precision\mathrm{*}recall}{precision+recall}$$

(1)

$$Precision=\frac{TruePositives}{TruePositives+FalsePositives}$$

(2)

$$Recall=\frac{TruePositives}{TruePositives+FalseNegatives}$$

(3)

RFE feature selection concluded with a subset of 75 features, chosen from an initial set of 105 features. The features selected after RFE can be found in Appendix A. It is important to note that the feature vector of 105 features included baseline observations that were engineered as a separate feature to maintain the baseline of each individual piece of knowledge across all samples.

Hyperparameter Tuning

After the feature selection process, we performed hyperparameter tuning to find the best set of hyperparameters to tune our XGBoost model and maximize the performance. Hyperparameters are a set of configuration parameters for a machine learning model that cannot be learned during the training process and require manual adjustment by the user. Selecting the optimal hyperparameters can prove challenging, often necessitating trial-and-error approaches when performed manually. To address this problem, various automatic hyperparameter search algorithms have been proposed throughout the years, treating the search problem as an optimization problem where the objective function is unknown or a blackbox function. In this paper, we chose the Bayesian Optimization (BO) algorithm to efficiently determine the optimal set of hyperparameters using a few samples of the dataset. Bayesian optimization proves to be a valuable approach in tackling functions where determining extrema is computationally expensive. Its applicability extends to functions lacking a closed-form expression and those that involve expensive calculations, challenging derivative evaluations, or exhibit non-convex characteristics. In summary of the process, BO models the relationship between a given set of parameter combinations and model’s performance by fitting a Gaussian Process (GP). Then, it optimizes the GP to find the next promising combination of parameters. Each combination is evaluated by training the model and evaluating it using a given evaluation metric. In this paper, the evaluation metric is the F1-score presented in Equation (1). The process is repeated until a specific criterion is met. In most cases, the ending criterion is the maximum number of iterations provided by the researcher [29,30].

After the application of Bayesian Optimization, the parameters presented in

Table 2. Parameters selected after Bayesian Optimization.

Parameter	Value
colsample bytree	0.6714223800630487
gamma	0.7244817045778367
Learning_rate	0.01
min_child_weight	10
n_estimators	1000
Scale_pos_weight	5.0
subsample	0.5676926067435525
Max_depth	7

were attained.

3. Results

We trained the final XGboost instance using the features selected from RFE and the hyperparameters chosen with Bayesian Optimization. To assess the robustness and generalization capability of the model, we employed stratified 5-fold cross-validation. This technique involves splitting the dataset into five subsets (folds), ensuring that each fold maintains the same class distribution as the original dataset. The model is trained on four folds and validated on the remaining one in each iteration. This process is repeated five times, and the performance metrics, including F1-score, precision, recall, accuracy, and ROC AUC score, are averaged over the folds to provide a comprehensive evaluation. ROC AUC score stands for Area Under the Receiver Operating Characteristic curve and shows how well a binary classification can distinguish the two classes. In medical diagnosis, such information is vital, as misclassifying patients can lead to greater disease severity. Summarized results from the table process are presented in

Table 3. Average results from 5-fold cross-validation.

Target Class	Precision	Recall	F1-Score	Accuracy	ROC AUC
Stable	0.94	0.84	0.90	0.85	0.86
Transition	0.71	0.88	0.79

.

As expected, the model demonstrates higher accuracy in predicting the stable class compared to the transition class, achieving precision and recall scores exceeding 0.90 in some folds. This can be ascribed to the absence of any rebalancing methodology applied during the dataset curation process, thereby facilitating a dataset composition that endeavors to mirror real-world conditions to the greatest extent possible. Also, the difference observed between recall and precision in the transition class is ascribed to the use of the scale_pos_weight parameter during the hyperparameter optimization phase. This parameter imparts a heightened sensitivity of the model to the transition class as a measure to mitigate class imbalance. This technique results in slightly higher recall scores accompanied by a marginal reduction in precision scores, which deviate slightly from anticipated levels. The rationale for accepting this trade-off lies in the heightened significance accorded to accurately classifying patients within the transition class (as indicated by the higher recall), even if it entails the inclusion of certain false-positive predictions in the resultant predictive outcomes. Additionally, it is imperative to recognize the inherent complexity of the Alzheimer’s disease (AD) diagnosis and prognosis, constituting an exceptionally challenging problem that necessitates a substantial volume of heterogeneous data for achieving accurate predictions. This effect can be visually observed in the summarized confusion matrix in Figure 7.

Shapley Additive Explanations

Interpreting the model’s predictions is crucial when it comes to medical diagnosis. To address this issue, there have been numerous sophisticated methodologies to interpret blackbox machine learning models. One of the most famous explainable AI methodologies is SHAP values, which find their origin in game theory. Fundamentally, SHAP transforms the feature space of XGBoost into a clinical variable space, wherein each transformed SHAP value corresponds to an original variable. This transformation facilitates a more clinically meaningful interpretation of the model’s output. Graphically, SHAP often visualizes XGBoost predictions to enhance interpretability. For instance, the SHAP summary plot succinctly illustrates the magnitudes and directions of predictions, wherein the size of a SHAP value signifies the contribution of a specific feature. Larger values indicate a more substantial contribution to prediction performance. Moreover, the SHAP dependency plot offers insights into the distribution of SHAP values across individuals for a given feature. As SHAP values vary among individuals, so do the predictions of the corresponding feature mappings for those individuals. This understanding provides a comprehensive view of the impact of individual features on model predictions, offering a valuable tool for researchers and practitioners in comprehending and validating the XGBoost model’s decision-making processes [18,31]. We applied SHAP Additive Explanations to interpret the model’s decision-making processes.

4. Discussion

In this study, an interpretable machine learning model was developed using eXtreme Gradient Boosting and the Shapley explanation framework. The model was trained on the ADNI dataset, including all four phases of ADNI, with the latest being ADNI-3 at the time of writing. The dual purpose of this study was to develop a machine learning model with the primary goal of accurately distinguishing between individuals with Mild Cognitive Impairment (MCI) who will experience stability in their cognitive function (MCIs) and those who will progress to Alzheimer’s disease soon (MCIc). Additionally, the secondary aim was to interpret the model’s decision process to help clinicians understand the reasons behind each prediction to make them trust the ML system. To achieve this, we chose to train the model across all ADNI phases in an effort to develop a framework that can be adopted in heterogeneous dataset types since the data collection protocol has changed gradually from ADNI1 to the ADNI3 phase. It is essential to acknowledge that we did not employ any direct balancing methodology in the development of our model. However, to mitigate potential issues arising from the unbalanced nature of our data, we utilized stratified k-fold cross-validation. This decision was deliberate, as it reflects a conscious choice to align our modeling approach with the prevalence of Mild Cognitive Impairment and Alzheimer’s disease patients in the general population. In real-life conditions, individuals diagnosed with MCI and AD represent a minority within the broader demographic landscape. This approach is particularly crucial in the development of medical models, where accurately mirroring real-world prevalence is essential for model reliability and relevance. In addressing the target class modeling problem, we adopted a unique approach. Our strategy involved grouping MCI patients, regardless of the onset time of AD up to their 48th visit since baseline, subsequently annotating the conversion using a binary target feature created in the dataset used. This approach was used to construct a model capable of providing predictions that reflect diverse real-life scenarios. Through exposing the model to a variety of samples showcasing conversion tendencies, our objective was to direct its learning process in discerning the factors indicative of a patient’s potential progression to AD. Our methodology surpassed similar techniques in performance, although it is worth acknowledging the current limitations of our comparative analysis. At the time of writing, most of the research available mainly focuses on developing diagnostic or potentially prognostic methodologies, rather than specifically tailored prognostic techniques. Additionally, while most prognostic methods predict within a fixed time window for conversion, our approach offers a more adaptable timeframe of 4 years (48 months) from the baseline visit. A method proposed by Fuliang et al. (2023) can partially be compared with our proposed solution since we both used similar approaches in terms of XGBoost and SHAP setup, but their approach is purely diagnostic. Our model showcased higher-sensitivity performance, which is crucial in medical models, despite their diagnostic orientation [18]. Other studies that use machine learning for the early diagnosis of AD analyzing MRIs and their variants also seem to slightly fall under in terms of performance when compared to our approach. Although they exhibit accuracy levels similar to ours, their sensitivity scores are slightly lower in the same time window [5,32,33].

In Figure 8, the SHAP summary plot illustrates the top-10 features generated by XGBoost. These features are arranged in descending order based on their SHAP values for all predictions. The SHAP values indicate the positive or negative associations of the respective features, and the absolute SHAP value for each feature is displayed on the left side. Every data point depicted in the plot represents an individual sample, with the horizontal axis indicating the SHAP value of a specific feature across subjects. This axis serves as a gradient, ranging from low (depicted in blue) to high (represented in red), reflecting the magnitude of the SHAP value. Apparently, cognitive test results have a moderate influence on model outcome. For example, high scores on the ADAS-Cog 13 scale indicate a possible cognitive impairment and, thus, a possible transition to AD in the near feature [34]. In contrast with ADAS-Cog-13, lower scores on the PACC scale (mPACCtrailsB and mPACCdigit) indicate potential cognitive decline [35]. As anticipated, additional biomarkers derived from imaging and clinical tests exert a considerable impact on the ultimate decision. Biomarkers such as ABETA and Entorhinal, indicative of the brain’s status, exhibit notably low values, contributing significantly to the overall outcome.

The features that impact the model based on their average SHAP value are presented in Figure 9. It is important to highlight the role of the “DX_bl” feature, representing the baseline diagnosis, in our prediction process. Our dataset includes Alzheimer’s disease (AD) patients who did not show cognitive decline at the start, aligning with our study’s focus. This feature mainly helps the model differentiate between AD and Mild Cognitive Impairment (MCI) patients, rather than distinguishing between progressing MCI (MCIc) and stable MCI (MCIs). As expected, biomarkers and assessment tests related to cognitive function have a moderate impact on the model’s outcome.

5. Conclusions

In real-world clinical studies, the early identification of Alzheimer’s disease is a complex and challenging task for clinicians involving various procedures, including cognitive, psychiatric, and biomarker tests. Not only are some of these assessments time-consuming for both clinicians and patients but they also involve invasive procedures, rendering them not only inefficient in terms of time but also cost-ineffective. Machine learning can improve the prognostic process for Alzheimer’s disease by identifying useful information within large amounts of clinical data and, thus, pave the way for more time-effective and non-invasive prognostic methods. At the same time, machine learning can help clinicians understand how the disease progresses in patients through biomarker analysis, enabling the timely administration of appropriate treatment. Our proposed framework can contribute to the above aims. Our results showed that the proposed XGBoost model can accurately predict the conversion from MCI to AD within 36 months from baseline with low error rates. Integrating the SHAP framework with the prediction model allows us to interpret the model’s decision process for each given sample, providing transparency to clinicians and enhancing the model’s reliability. Despite these promising results, further research is essential to develop models with minimal error rates and implement them in real-world conditions. To achieve this, it is imperative to train models on extensive datasets, representing real-life scenarios that may differ across various clinics and hospitals. Furthermore, the optimization of these models requires attention to factors influencing the decision-making process in different healthcare settings, always prioritizing the well-being and safety of patients.

6. Limitations

This study has potential limitations. The proposed model was specifically trained to predict the transition from Mild Cognitive Impairment (MCI) to Alzheimer’s disease (AD), while excluding the possibilities of transitions from Cognitively Normal (CN) to MCI and CN to AD. The proposed approach needs further clinical verification against an external dataset. The model needs two timepoints to make an accurate prediction, which translates to two physical visits.