Next Article in Journal
What Is the Trajectory of Recovery in the Early Postoperative Period after the Big 3 Shoulder Surgeries? Comparative Analysis Using 3 Previous Prospective Studies
Next Article in Special Issue
Evaluating Prediction Models with Hearing Handicap Inventory for the Elderly in Chronic Otitis Media Patients
Previous Article in Journal
A Comprehensive Multidisciplinary Approach to Diagnosing Chronic Inflammatory Bowel Diseases: Integration of Clinical, Endoscopic, and Imaging Modalities
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Diagnostics
Volume 14
Issue 14
10.3390/diagnostics14141531
diagnostics-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Predictors of In-Hospital Mortality after Thrombectomy in Anterior Circulation Large Vessel Occlusion: A Retrospective, Machine Learning Study

by
Ivan Petrović
1,*,
Serena Broggi
2,
Monika Killer-Oberpfalzer
3,4,
Johannes A. R. Pfaff
5,
Christoph J. Griessenauer
6,
Isidora Milosavljević
1,
Ana Balenović
1,
Johannes S. Mutzenbach
3 and
Slaven Pikija
3,*
1
Faculty of Medicine, University of Novi Sad, 21000 Novi Sad, Serbia
2
Neurology and Stroke Unit, ASST dei Sette Laghi, 21100 Varese, Italy
3
Department of Neurology, University Hospital Salzburg, Christian Doppler Klinik, Paracelsus Medical University Salzburg, 5020 Salzburg, Austria
4
Institute of Neurointervention, Christian Doppler Klinik, Paracelsus Medical University Salzburg, 5020 Salzburg, Austria
5
Department of Neuroradiology, University Hospital Salzburg, Christian Doppler Klinik, Paracelsus Medical University Salzburg, 5020 Salzburg, Austria
6
Department of Neurosurgery, University Hospital Salzburg, Christian Doppler Klinik, Paracelsus Medical University, 5020 Salzburg, Austria
*
Authors to whom correspondence should be addressed.
Diagnostics 2024, 14(14), 1531; https://doi.org/10.3390/diagnostics14141531
Submission received: 18 June 2024 / Revised: 8 July 2024 / Accepted: 14 July 2024 / Published: 16 July 2024
(This article belongs to the Special Issue Machine-Learning-Based Disease Diagnosis and Prediction)
Browse Figures
Review Reports Versions Notes

Abstract

Background: Despite the increased use of mechanical thrombectomy (MT) in recent years, there remains a lack of research on in-hospital mortality rates following the procedure, the primary factors influencing these rates, and the potential for predicting them. This study aimed to utilize interpretable machine learning (ML) to help clarify these uncertainties. Methods: This retrospective study involved patients with anterior circulation large vessel occlusion (LVO)-related ischemic stroke who underwent MT. The patient division was made into two groups: (I) the in-hospital death group, referred to as miserable outcome, and (II) the in-hospital survival group, or favorable outcome. Python 3.10.9 was utilized to develop the machine learning models, which consisted of two types based on input features: (I) the Pre-MT model, incorporating baseline features, and (II) the Post-MT model, which included both baseline and MT-related features. After a feature selection process, the models were trained, internally evaluated, and tested, after which interpretation frameworks were employed to clarify the decision-making processes. Results: This study included 602 patients with a median age of 76 years (interquartile range (IQR) 65–83), out of which 54% (n = 328) were female, and 22% (n = 133) had miserable outcomes. Selected baseline features were age, baseline National Institutes of Health Stroke Scale (NIHSS) value, neutrophil-to-lymphocyte ratio (NLR), international normalized ratio (INR), the type of the affected vessel (‘Vessel type’), peripheral arterial disease (PAD), baseline glycemia, and premorbid modified Rankin scale (pre-mRS). The highest odds ratio of 4.504 was observed with the presence of peripheral arterial disease (95% confidence interval (CI), 2.120–9.569). The Pre-MT model achieved an area under the curve (AUC) value of around 79% utilizing these features, and the interpretable framework discovered the baseline NIHSS value as the most influential factor. In the second data set, selected features were the same, excluding pre-mRS and including puncture-to-procedure-end time (PET) and onset-to-puncture time (OPT). The AUC value of the Post-MT model was around 84% with age being the highest-ranked feature. Conclusions: This study demonstrates the moderate to strong effectiveness of interpretable machine learning models in predicting in-hospital mortality following mechanical thrombectomy for ischemic stroke, with AUCs of 0.792 for the Pre-MT model and 0.837 for the Post-MT model. Key predictors included patient age, baseline NIHSS, NLR, INR, occluded vessel type, PAD, baseline glycemia, pre-mRS, PET, and OPT. These findings provide valuable insights into risk factors and could improve post-procedural patient management.

1. Introduction

Despite major advances in emergent treatment, stroke is still a devastating disease identified as one of the leading causes of death and disability on a global level [1,2,3,4,5,6,7,8]. Acute ischemic stroke (AIS), caused by the culprit cessation of brain circulation leading to brain infarction, accounts for more than two-thirds of strokes [8,9,10]. Current state-of-the-art therapy includes intravenous thrombolysis (IVT) with alteplase or tenecteplase and mechanical thrombectomy (MT), alone or in combination with so-called bridging IVT [10,11,12,13]. Following large clinical trials, MT, particularly when combined with IVT, has become the new standard of care in patients with large vessel occlusion (LVO)-related acute stroke, as it results in better functional outcomes and a higher degree of angiographic revascularization, without a significant difference in the incidence of symptomatic intracerebral hemorrhage (sICH) or death, compared to IVT alone [14,15,16,17,18,19]. Nevertheless, it is imperative to acknowledge that certain patients may exhibit unfavorable prognoses or succumb shortly after the intervention. Therefore, it is of the highest interest to investigate whether there are a priori features of patients that should be assessed, treated, or modified before the procedure, which could be relevant to later adverse outcomes.
In previously published work, the estimated rate of in-hospital mortality in stroke ranges from 11 to 15% [20]. The percentage of deaths escalate further if 90-day mortality is included, reaching as high as 27%, with 52% of all death cases occurring during the first week after the MT treatment [21]. About 2/3 of in-hospital deaths are attributed to non-modifiable, baseline factors such as age and baseline stroke severity, while post-stroke complications determine the remaining third [20,22]. In practice, a degree of independence prior to the stroke measured with the premorbid modified Rankin scale (pre-mRS) is an important determining factor in whether MT is going to be performed. It is a 7-point scale where 0 signifies no symptoms, 4 indicates mobility only with assistance from another person, and 6 denotes death [23]. Stroke patients with a pre-mRS score of 4 or 5 before the onset of symptoms are generally not considered for mechanical thrombectomy and are excluded from most trials on endovascular treatment (EVT) [24]. However, for stroke patients with a pre-mRS score of 0–3, mechanical thrombectomy should not be withheld based solely on other features. Nevertheless, in an ideal scenario, having a reliable predictive tool before intervention could enable stakeholders to anticipate potential outcomes better.
Machine learning (ML) has become apparent as a competent predictive tool in many medical fields since, through modeling, it can handle complex relationships in data, uncover subtle information, and use these insights to automatically generate and summarize new knowledge [25]. This cannot be said for classical statistical approaches, as the increase in data complexity may make classical statistical inference less tractable [26,27]. In the currently available literature, ML models have shown superiority when compared to the standard statistical and clinical prediction models [28], and since there is increasing complexity, numerosity, and multifactoriality of available data sets [29], interest in their usage is rapidly increasing. On the other hand, clinicians can still feel confused and uncertain about the use of these models in everyday practice. The ML’s “black box” represents a major barrier to this progress since an understanding of a model’s decision-making process is crucial for its implementation in the clinical environment [30]. This concern could play an important role in daily interactions with patients, because, without an explanation of how a diagnosis, prognosis, or treatment plan was made, a lack of trust can be expected [31]. Fortunately, significant progress has been accomplished within the last years by developing interpretation methods, such as the Sharpley Additive Explanation (SHAP) and Local Interpretable Model-Agnostic Explanations (LIMEs), which show the potential in helping with the interpretational riddle on both feature and individual levels. To the best of our knowledge, interpretable machine learning (IML) models and their role in the prediction of in-hospital death after anterior circulation MT in AIS patients are still an unexplored research area.
In this research, we aimed to develop and internally validate IML models that would predict the probability of in-hospital death after anterior circulation MT in LVO-AIS patients. We hypothesized that ML would generate an easily understandable predictive model, highlighting the key features contributing to the unfavorable outcome and offering a better understanding of this complexity. The built model could be used to identify AIS patients less likely to benefit from recanalization and patients with a high risk of dying after the procedure.

2. Materials and Methods

2.1. Analyzed Group and Data Processing

This retrospective study analyzed the clinical data of consecutive patients with anterior circulation LVO-related ischemic stroke, treated with mechanical thrombectomy (MT) at the tertiary university middle-volume endovascular treatment center, Christian-Doppler-Klinik (Salzburg, Austria). The local ethics committee approved the retrospective collection of data. The data were collected from the clinical information system during the analyzed period of 9 years (2012–2020), and the inclusion criteria were that (I) the patient was older than 18 years; (II) MT was used as a therapeutic approach; (III) occluded vessel(s) were in the anterior circulation of the brain; and additionally to be included into the in-hospital death (IHD) group, (IV) if a patient died, it occurred due to the nature of stroke or stroke-related complications [22,32] during the time of hospitalization. On the contrary, patients with occlusion of the posterior circulation were excluded from the study, and patients who died after the period of in-hospital stay were not included in the IHD group.
The gathered data were divided into several subcategories, based on their similarities (Table 1), and were analyzed on three levels: (I) the whole group; (II) the in-hospital death group (miserable outcome); and (III) the in-hospital survival group (favorable outcome). Python 3.10.9, provided by Anaconda, Inc. (Austin, TX, USA), was the programming language used in this research. In the first steps of preprocessing, variables from the data were screened for missing values, and if this number went beyond 20%, they were excluded from any further analysis. A categorization of the variables followed, in which the division was made into categorical and continuous groups. Missing values were imputed using the most common value method for categorical variables and the median value for continuous ones.
Following a prior procedure, the data were divided into the outcome (target variable) and the input features. Outcome was represented by the parameter ‘in-hospital death’, which included two options: (I) miserable outcome, if a patient died during an in-hospital stay, and (II) favorable outcome if a patient survived. In this research, the terms in-hospital death and miserable outcome were alternately used to describe the same result, as we wanted to differentiate it from the unfavorable, or poor outcome, which could refer to different degrees of disability and therefore diminish the significance of the issue. The input features consisted of 53 variables, which were statistically compared between the two possible outcomes (Supplementary Table S1). Categorical features were expressed as numbers (percentages), and the chi-squared test was used to determine differences between the two outcome groups. Continuous features were presented as medians (interquartile range) and analyzed by the Student t-test or Mann–Whitney U test, based on sample normality. Two-tailed tests were used, and statistical significance was observed at level p < 0.05 for every variable. To address the collinearity between features, a correlation analysis was performed. We used the chi-squared test of independence for categorical features, presenting results as p values, with a statistically significant correlation being p < 0.05. The Pearson and Spearman Correlation coefficients were calculated for continuous features, based on their distribution, representing results as coefficients ranging from −1 to +1, presenting the strength and direction of the linear relationship between two continuous variables.
Data normalization followed, which is one of the first preprocessing tasks to be performed during analysis [33], and it implies that the data are transformed or scaled so that an equal contribution of each feature is achieved. This was carried out by Z-score normalization, in which the values of a feature are normalized based on the mean (average) value and standard deviation, as this process reduces numerical instabilities between the analyzed features [34,35].

2.2. Two-Step Feature Selection Process

After the previous steps, variables were divided into two categories: (I) Baseline features (n = 42) that were gathered or present during the patient’s admission, and (II) Intervention-related features (n = 11) collected or calculated after the endovascular treatment. As the input features included 53 variables, the feature selection technique was applied as it enables the selection of those features that contribute the most to the prediction variable or the outcome in machine learning (ML) algorithms. It is primarily carried out to remove unimportant features, improve analysis efficacy, and adapt the data set to best suit ML classifiers [36]. First, irrelevant features were removed after the univariate analysis, and then, redundant and useless features were excluded by a wrapper approach [37]. Univariate analysis was based on the features’ type. The Shapiro–Wilk normality test was conducted before the continuous variables analysis [38], and as asymmetric distribution was observed, the Mann–Whitney-Wilcoxon test was applied [39], while the chi-squared test was used in the categorical variables analysis [40]. After the removal of the irrelevant features, Recursive Feature Elimination (RFE) was used, as a method of wrapper models [41]. This algorithm eliminates one backward feature during one iteration and prevents information loss while showing high classification performances when compared to many other feature reduction methods [42]. This algorithm was used with four different classifiers—logistic regression (LG), random forest (RF), gradient boosting (GB), and extreme gradient boosting (XGB)—and the number of possible features to be included in the classification was not limited, and it solely depended on the model. The RFE process was performed for every classifier separately, and two features’ sets were gained: (I) Pre-MT set, including only baseline features, and (II) Post-MT set, which included both baseline and intervention-related variables.
To statistically address the influence of chosen variables on the miserable outcome, binomial regression analysis was performed, calculating the odds ratios and plotting the estimated marginal means plots. The most optimal cut-off values to predict the miserable outcomes were determined based on sensitivity and specificity.

2.3. Data Sampling and Construction of Two Predictive Models

As the outcome group was unbalanced (miserable outcome, n = 133 vs. favorable outcome, n = 469), we performed a data sampling technique to reduce the disparity. There are two basic sampling techniques, including random oversampling (ROS) and random undersampling (RUS). Despite the lack of conclusive evidence concerning the superiority of any approach, it has been reported that oversampling may lead to overfitting of the model [43,44,45,46], which is why we chose the RUS as more appropriate. In this approach, the majority classes are randomly eliminated, so that equal distribution can be achieved [47].
By random splitting, data were divided into three groups: (I) a Training set that included 70% of the input data, (II) an Evaluation set consisting of 15%, and (III) a Test set comprising 15% of the original data. Using Python’s Scikit-learn library, four classifiers were trained, internally evaluated, and tested: logistic regression (LR), random forest (RF), gradient boosting (GB), and extreme gradient boosting (XGB). As previously mentioned, two predictive models were created for every classifier based on the input features’ sets: (I) Pre-MT model, and (II) Post-MT model. During the training, each classifier used RFE-preselected feature sets, and the training included 10-fold cross-validation, during which the training set was divided into 1 validating and 9 training subsets (Figure 1). After the internal evaluation and testing, results were expressed through evaluation metrics such as accuracy, precision, recall, F1 score, and the area under the receiver operating characteristic (AUC-ROC) curve. Since the AUC has been proven to be a better measure than accuracy in the learning algorithm evaluation [48], it was used as a measure of discrimination [49], while the model calibration was expressed through calibration curves and Brier scores, where lower scores reflected better model calibration [50].

2.4. Interpretable Framework-Sharpley Additive Explanation (SHAP) and Local Interpretable Model-Agnostic Explanations (LIME)

Best-performing classifiers, based on the AUC value, were more thoroughly analyzed through Sharpley Additive Explanation (SHAP) values. This locally interpretable explanatory method can rationalize machine learning (ML) algorithm predictions, regardless of their complexity, as it identifies and maps features that reduce or increase the probability of the predicted outcome [51]. Additionally, SHAP can help with the understanding of features’ importance and, therefore, enhance their clinical interpretability [25]. Features were presented based on their order of importance (Absolute Mean SHAP Plot) in the predictive model.
To better understand the classification-making process on an individual level, Local Interpretable Model-Agnostic Explanations (LIMEs) were applied, as they can extract the contribution of key features during it [52]. This analysis was performed for both models, randomly choosing a patient from both groups. This figure (Figure 5)/Figure 5. shows the overall predicted probability of in-hospital death and favorable outcome on the left, classification details in the middle, and feature values and categories on the right. Continuous features were presented with their true values, while the categorical features were labeled based on the included category.

3. Results

3.1. Analyzed Group

A total of 602 patients fulfilled the inclusion criteria. In our cohort, around 54% (n = 328) of all patients were women, 133 (22%) had miserable outcomes, and the median age was 76 years (interquartile range (IQR) 65–83 years). Statistical feature comparison is presented in Table 1, and the correlation matrices are stored in Supplementary Table S2 for categorical features and Supplementary Table S3 for continuous ones. Out of all baseline variables, age (81 (IQR 74–87) vs. 74 (IQR 62–82), p < 0.001), baseline NIHSS value (20 (IQR 16–24) vs. 17 (IQR 11–21), p < 0.001), and peripheral artery disease (19/133 (14%) vs. 14/469 (3%), p < 0.001) showed the most statistically significant disparity between the two analyzed groups, while the most significant intervention-related feature was the ‘Number of steps’ (3 (IQR 2–4) vs. 2 (IQR 1–4), p = 0.011). Correlation analysis showed high collinearity between the previous stroke, the presence of peripheral arterial disease, and higher pre-mRS values.

3.2. The Results of a Two-Step Feature Selection Process

During a two-step process, variables for the predictive models were chosen as a product of univariate feature selection (UFS) and Recursive Feature Elimination (RFE). In the baseline set, those 8 features were age, baseline NIHSS value, neutrophil-to-lymphocyte ratio (NLR), international normalized ratio (INR), the type of the affected vessel (‘Vessel type’), peripheral arterial disease (PAD), baseline glycemia, and premorbid modified Rankin scale (pre-mRS). In the intervention-related set, the RFE method did not include pre-mRS, but it added two features: puncture-to-the-procedure end time (PET), and onset-to-puncture time (OPT). The RFE-selected sets were used in further analysis. The whole process is summarized in Figure 1.
Binomial logistic regression results are presented in Table 2, showing the influence of chosen features on in-hospital death (IHD) occurrence. Out of continuous features, INR had the highest impact with the most optimal cut-off value of 1.29, and an increase of 0.2 led to 37.2% higher chances of IHD. One unit increase in baseline NIHSS increased the risk of IHD by approximately 9%, with the optimal cut-off value of 20, one unit of NLR (the optimal cut-off = 3.8) as well as 10 mg/dL of blood glucose (the optimal cut-off = 120 mg/dL) by 7%, and one year of age (the optimal cut-off value = 78 years) by 6%. Selected time frames showed a lower influence on IHD, where every 10 min of OPT (the optimal cut-off value = 285 min) increased risk by 1% and 10 min of PET (the optimal cut-off value = 54 min) by 2%. The cut-off values tables are stored in Supplementary Table S4. Categorical variables showed high influence, as the presence of peripheral arterial disease (PAD) increased the chances of developing IHD 3.5 times and a premorbid mRS value of 2 or higher by 80%. The results are plotted in Figure 2.

3.3. Predictive Models Evaluation

After the training, internal evaluation, and testing, the performances of the classifiers were expressed through the previously mentioned parameters (Table 3). Their ROC curves and the calibration plot with reliability curves are visualized in Figure 3. For the Pre-MT model, gradient boosting (GB) showed itself as the best-performing classifier, during all phases, with an AUC of almost 80% (0.7903) and the lowest Brier score of 0.2009. The post-MT classifier evaluation showed that the extreme gradient boosting (XGB) had the highest performance with an AUC score of 84% (0.8372) and a Brier score of 0.1194.

3.3.1. SHAP Analysis–Feature Level Interpretation

For the best-performing classifier, in both models, SHAP values were calculated, and the order of importance is presented in Figure 4. Based on the SHAP analysis of the Pre-MT model (Figure 4a), the most influential features that increase the risk of developing a miserable outcome were baseline NIHSS, age, NLR, glycemia, and INR. In addition, pre-mRS of two or higher, internal carotid artery occlusions (type L and T ICA terminus occlusions), and present peripheral arterial disease were also important contributing factors. The SHAP analysis of the XGB classifier (Figure 4b) shows that the same factors as previously mentioned influence the outcome prediction, with the addition of longer ‘Puncture-to-end time’ (PET) and ‘Onset-to-the-puncture time’ (OPT).

3.3.2. LIME Analysis–Individual Level Interpretation

The Pre-MT model’s LIME analysis is visualized in Figure 5a, simplifying the GB classifier’s decision-making algorithm, and Figure 5b visualizes the post-MT model, based on the prediction of XGB. Continuous features are presented with their true values, while the categorical features are labeled as follows. pre-mRS category 0 corresponds to a score of 2 or higher, peripheral arterial disease (PAD) category 1 represents the presence of the disease, while the vessel type is associated with the division in Table 1, within the 5 categories (1, M1 segment of the middle cerebral artery; 2, M2 segment of the middle cerebral artery; 3, internal carotid artery occlusion–type I; 4, internal carotid artery occlusion–type L; 5, internal carotid artery occlusion–type T). In chosen patients, probability scores, based on eight features in the Pre-MT model, which is nine in the Post-MT one, predict both favorable and miserable outcomes.

4. Discussion

To the best of our knowledge, this is the first study that utilizes interpretable machine learning models for the in-hospital mortality analysis of patients who underwent thrombectomy for anterior circulation occlusive stroke. Four classifiers were trained, internally evaluated, and tested for both models. The two best-performing classifiers were the gradient boosting (GB) for the Pre-MT model and the extreme gradient boosting (XGB) for the Post-MT predictive model, which included both baseline and intervention-related parameters. Based on eight variables, GB achieved an AUC score of almost 80%, with a Brier score of 0.2, while the XGB accomplished better performances, an AUC score of 84%, and a Brier score of 0.12, using nine features. In addition to seven identical features that were incorporated into both models (age, baseline NIHSS value, neutrophil-to-lymphocyte ratio (NLR), international normalized ratio (INR), the type of the affected vessel, baseline glycemia, and the presence of peripheral arterial disease), the baseline model also included premorbid modified Rankin scale (pre-mRS), while the second model involved puncture-to-the-procedure-end time (PET) and onset-to-puncture time (OPT). We aimed to more thoroughly explain the “black box” of ML models through Sharpley Additive Explanation (SHAP), which interpreted the model at the feature level, and Local Interpretable Model-Agnostic Explanations (LIMEs), used for the interpretation at the individual level.

4.1. Baseline Features Prior to Thrombectomy

In our baseline, Pre-MT predictive model, baseline NIHSS value and age stood out as the most significant features, which is similar to the previous research [20,22]. Out of all analyzed laboratory findings, the neutrophil-to-lymphocyte ratio followed by baseline glycemia and INR value have proven themselves as the most appropriate for mortality prediction.
Hyperglycemia was linked to unfavorable clinical outcomes and mortality, in patients with large vessel occlusion (LVO) that were treated with the MT [53], which probably occurs as a consequence of metabolic alterations, such as intracellular acidosis, decreased mitochondrial function, and increased levels of reactive oxygen species (ROS), leading to neuronal damage [54]. In a previously published study, in which interpretable machine learning was used to predict the development of 24 h early neurological deterioration (END) in MT-treated ischemic stroke patients, hyperglycemia was the most important feature that contributed to the unfavorable outcome [25]. Even though hypoglycemia was proven to predict higher 90-day mortality in MT-treated patients [54], it did not demonstrate itself as significant in this analysis. A second laboratory feature, the neutrophil-to-lymphocyte ratio (NLR), was also linked with the stroke outcome. Higher NLR was associated with a more common death, unfavorable, or poor 90-day outcome in ischemic stroke patients who underwent reperfusion therapy [55,56]. Accompanied by hyperglycemia, which is an important factor for the development of thromboinflammation [57], elevated NLR represents a response to the inflammatory nature of stroke, which is a major contributor to its pathobiology and outcome [58]. Although MT can be safely and effectively performed in emergent large vessel occlusion (ELVO) patients with high INR [59], these patients are more prone to worse clinical outcomes and higher mortality risk [60], which was also found in our study.
Taking into consideration occluded vessels, our analysis has shown that the T-type internal carotid occlusion was mostly associated with in-hospital death, while the isolated middle cerebral artery (MCA) occlusion, specifically in the M1 segment, was more abundant in the favorable outcome group. In the available literature, no research has explicitly analyzed the connection between the occlusion type and in-hospital mortality. However, it was found that the internal carotid artery (ICA) occlusion is a significant predictor of 90-day mortality [61] and that the L- and T-types of terminal carotid occlusion were more frequent in the 90-day mortality group [62]. Outcomes of the thrombectomy of the MCA M1 and M2 segments showed that the 90-day functional outcomes did not differ between segments [63]. Although peripheral artery disease showed no association with the 90-day functional or safety outcomes in a large cohort study [64], it certainly is a strong independent predictor for stroke [65], and in our analysis, it was an important contributing factor to the miserable outcome.
It is estimated that only around 20% of the pre-stroke-disabled patients (mRS > 2) return to their premorbid values after the MT, and, additionally, a good outcome is less frequent, while the mortality and risk of symptomatic intracranial hemorrhage (sICH) are higher [66,67,68,69]. Based on the previous information, higher premorbid mRS could be classified as an important unfavorable factor, which was also chosen by our feature selection process.

4.2. Features Pertinent to Mechanical Thrombectomy

Similar to the previous model, in previously published works, most of the intervention-related features have shown significance in the 90-day functional outcome and survival prediction, but their role in the context of in-hospital mortality has not been analyzed. It is shown that shorter onset-to-puncture (OPT) is associated with a lower degree of 3-month disability and higher independence at discharge [70,71,72,73], and besides long-term outcomes, in this research, it was shown that the longer OPT is an important predictor of in-hospital death. Other than this, the feature selection model also included PET, which showed even higher predictive capabilities, compared to the OPT. In the available literature, this parameter is associated with 3-month mortality, especially if it exceeded 60 min, and post-interventional complications if this period went beyond 35 min. Possible explanations for this occurrence might be that the longer procedure duration usually happens when there is an occlusive material resistant to extraction. Besides this, an extended intervention timeframe can burden compensatory collaterals and lead to irreversible injuries [72,73,74]. Even though these factors individually demonstrated low predictive power, they collectively contributed to the model’s overall predictions. This highlights the importance of analyzing multiple features simultaneously rather than isolated, a task that is effectively performed by machine learning models.
There are several study limitations. First, all included patients died due to stroke, or its complications, but the precise cause of death was not more thoroughly differentiated in the research. As some of the most common conditions that contribute to in-hospital mortality are respiratory infections and brain edema [20,22], future studies may focus on these as an outcome and not the overall in-hospital mortality of ischemic stroke. Second, in our study, we used a hybrid feature selection approach, which is just one of many techniques that can be used during this preprocessing phase. It means that a variety of different methods could potentially be applied to this phase in the forthcoming studies, so that their influence on the results may become a subject of the analysis. Third, regardless of the high predictive power, for the evaluation and implementation of tested models, a higher number of included patients is necessary. Finally, this study only included patients with anterior circulation ischemic stroke, so it does not apply to the posterior vasculature.

5. Conclusions

In this study, two supervised ML models were constructed to predict in-hospital death after the anterior circulation MT, in large-vessel-occlusion-related acute ischemic stroke patients. The constructed models achieved good predictive performances, and the prediction-making process was more thoroughly understood with the usage of SHAP and LIME. The baseline parameters showed the strongest impact on the miserable outcome prediction, but the highest predictive power was obtained when time frames, OPT and PET, were also included in the model. We found that baseline predictors NIHSS and age, neutrophil-to-lymphocyte ratio, followed by baseline glycemia and INR value prove to be the most appropriate for mortality prediction. In addition, a longer time elapsed from arterial puncture to the end of the intervention was also associated with higher mortality. The majority of features related to endovascular treatment did not show an essential role in the miserable outcome prediction, since factors like the success of recanalization (TICI) and intervention-related complications demonstrated low or almost no impact on in-hospital death prediction.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/diagnostics14141531/s1, Table S1: Breakdown of features used in the research; Table S2: Categorical features correlation; Table S3: Continuous features correlation; Table S4: Cut-off values for continuous features.

Author Contributions

Conceptualization, I.P., S.B. and S.P.; methodology, I.P.; software, I.P.; formal analysis, I.P., S.B., I.M. and A.B.; resources, M.K.-O., J.A.R.P. and C.J.G.; data curation, M.K.-O., J.A.R.P. and C.J.G.; writing—original draft preparation, I.P., S.B., I.M. and A.B.; writing—review and editing, S.P.; visualization, I.P.; supervision, J.A.R.P., J.S.M. and S.P. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study protocol was per the guidance of the Christian Doppler Klinik (Salzburg, Austria) committee for the protection of human subjects (protocol UN 2553). The local ethics committee obtained written approval for the retrospective study of patients with acute ischemic stroke (415-EP/73/750-2017).

Informed Consent Statement

According to Austrian regulations, informed consent is not required for routinely collected clinical and radiological data as used in this study.

Data Availability Statement

All data generated or analyzed during this study are included in this published article.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Yang, C.; Zhu, C.; Sui, Y.; Guo, Y.; Wu, N.; Dong, Q.; Xu, R.; Qian, J.; Li, J. Synergetic impact of lipoprotein(a) and fibrinogen on stroke in coronary artery disease patients. Eur. J. Clin. Investig. 2024, 54, e14179. [Google Scholar] [CrossRef] [PubMed]
  2. Kamel, H.; Healey, J.S. Cardioembolic Stroke. Circ. Res. 2017, 120, 514–526. [Google Scholar] [CrossRef]
  3. Feske, S.K. Ischemic Stroke. Am. J. Med. 2021, 134, 1457–1464. [Google Scholar] [CrossRef]
  4. Krishna, J.T.V.; Kumar, P. Risk Factor Assessment, Etiology, Clinico-Radiological Profile and Prognosis in Cva; NaN, 2023; pp. 37–39. Available online: https://www.citedrive.com/en/discovery/risk-factor-assessment-etiologyclinico-radiological-profile-and-prognosis-in-cva/ (accessed on 17 June 2024).
  5. Deng, G.; Chu, Y.-H.; Xiao, J.; Shang, K.; Zhou, L.-Q.; Qin, C.; Tian, D.-S. Risk Factors, Pathophysiologic Mechanisms, and Potential Treatment Strategies of Futile Recanalization after Endovascular Therapy in Acute Ischemic Stroke. Aging Dis. 2023, 14, 2096–2112. [Google Scholar] [CrossRef] [PubMed]
  6. Tsalta-Mladenov, M.E.; Andonova, S.P. Quality of life after ischaemic stroke-accent on patients with thrombolytic therapy. Egypt. J. Neurol. Psychiatry Neurosurg. 2021, 57, 163. [Google Scholar] [CrossRef]
  7. Matsumoto, K.; Nohara, Y.; Soejima, H.; Yonehara, T.; Nakashima, N.; Kamouchi, M. Stroke Prognostic Scores and Data-Driven Prediction of Clinical Outcomes After Acute Ischemic Stroke. Stroke 2020, 51, 1477–1483. [Google Scholar] [CrossRef]
  8. Phipps, M.S.; Cronin, C.A. Management of acute ischemic stroke. BMJ 2020, 368, l6983. [Google Scholar] [CrossRef] [PubMed]
  9. Abdu, H.; Tadese, F.; Seyoum, G. Comparison of Ischemic and Hemorrhagic Stroke in the Medical Ward of Dessie Referral Hospital, Northeast Ethiopia: A Retrospective Study. Neurol. Res. Int. 2021, 2021, 9996958. [Google Scholar] [CrossRef] [PubMed]
  10. Berge, E.; Whiteley, W.; Audebert, H.; De Marchis, G.M.; Fonseca, A.C.; Padiglioni, C.; de la Ossa, N.P.; Strbian, D.; Tsivgoulis, G.; Turc, G. European Stroke Organisation (ESO) guidelines on intravenous thrombolysis for acute ischaemic stroke. Eur. Stroke J. 2021, 6, I–LXII. [Google Scholar] [CrossRef]
  11. Turc, G.; Turc, G.; Bhogal, P.; Fischer, U.; Khatri, P.; Lobotesis, K.; Mazighi, M.; Schellinger, P.D.; Toni, D.; De Vries, J.; et al. European Stroke Organisation (ESO)-European Society for Minimally Invasive Neurological Therapy (ESMINT) Guidelines on Mechanical Thrombectomy in Acute Ischemic Stroke. J. Neurointerv. Surg. 2023, 15, e8. [Google Scholar] [CrossRef]
  12. Powers William, J. Acute Ischemic Stroke. N. Engl. J. Med. 2020, 383, 252–260. [Google Scholar] [CrossRef]
  13. Wang, R.P.C.; Li, S.; Hao, L.; Wang, Z.B.; Ge, Z.B.; Yang, S. A meta-analysis of intravenous thrombolysis versus bridging therapy for ischemic stroke. Medicine 2022, 101, e30879. [Google Scholar] [CrossRef]
  14. Berkhemer, O.A.; Fransen, P.S.S.; Beumer, D.; Berg, L.A.V.D.; Lingsma, H.F.; Yoo, A.J.; Schonewille, W.J.; Vos, J.A.; Nederkoorn, P.J.; Wermer, M.J.H.; et al. A randomized trial of intraarterial treatment for acute ischemic stroke. N. Engl. J. Med. 2015, 372, 11–20. [Google Scholar] [CrossRef] [PubMed]
  15. Campbell, B.C.; Mitchell, P.J.; Kleinig, T.J.; Dewey, H.M.; Churilov, L.; Yassi, N.; Yan, B.; Dowling, R.J.; Parsons, M.W.; Oxley, T.J.; et al. Endovascular therapy for ischemic stroke with perfusion-imaging selection. N. Engl. J. Med. 2015, 372, 1009–1018. [Google Scholar] [CrossRef] [PubMed]
  16. Goyal, M.; Demchuk, A.M.; Menon, B.K.; Eesa, M.; Rempel, J.L.; Thornton, J.; Roy, D.; Jovin, T.G.; Willinsky, R.A.; Sapkota, B.L.; et al. Randomized assessment of rapid endovascular treatment of ischemic stroke. N. Engl. J. Med. 2015, 372, 1019–1030. [Google Scholar] [CrossRef]
  17. Jovin, T.G.; Chamorro, A.; Cobo, E.; De Miquel, M.A.; Molina, C.A.; Rovira, A.; Román, L.S.; Serena, J.; Abilleira, S.; Ribo, M.; et al. Thrombectomy within 8 hours after symptom onset in ischemic stroke. N. Engl. J. Med. 2015, 372, 2296–2306. [Google Scholar] [CrossRef]
  18. Stent-Retriever Thrombectomy after Intravenous t-PA vs. t-PA Alone in Stroke|NEJM [Internet]. Available online: https://www.nejm.org/doi/full/10.1056/nejmoa1415061 (accessed on 14 February 2024).
  19. Goyal, M.; Menon, B.K.; Van Zwam, W.H.; Dippel, D.W.J.; Mitchell, P.J.; Demchuk, A.M.; Dávalos, A.; Majoie, C.B.L.M.; Van Der Lugt, A.; De Miquel, M.A.; et al. Endovascular thrombectomy after large-vessel ischaemic stroke: A meta-analysis of individual patient data from five randomised trials. Lancet 2016, 387, 1723–1731. [Google Scholar] [CrossRef] [PubMed]
  20. Johnston, S.C.; Mendis, S.; Mathers, C.D. Global variation in stroke burden and mortality: Estimates from monitoring, surveillance, and modelling. Lancet Neurol. 2009, 8, 345–354. [Google Scholar] [CrossRef]
  21. Sluis, W.M.; Hinsenveld, W.H.; Goldhoorn, R.-J.B.; Potters, L.H.; Bruggeman, A.A.; van der Hoorn, A.; Bot, J.C.; van Oostenbrugge, R.J.; Lingsma, H.F.; Hofmeijer, J.; et al. Timing and causes of death after endovascular thrombectomy in patients with acute ischemic stroke. Eur. Stroke J. 2023, 8, 215–223. [Google Scholar] [CrossRef]
  22. Bustamante, A.; García-Berrocoso, T.; Rodriguez, N.; Llombart, V.; Ribó, M.; Molina, C.; Montaner, J. Ischemic stroke outcome: A review of the influence of post-stroke complications within the different scenarios of stroke care. Eur. J. Intern. Med. 2016, 29, 9–21. [Google Scholar] [CrossRef]
  23. Broderick, J.P.; Adeoye, O.; Elm, J. Evolution of the Modified Rankin Scale and Its Use in Future Stroke Trials. Stroke 2017, 48, 2007–2012. [Google Scholar] [CrossRef]
  24. Benali, F.; Kappelhof, M.; Ospel, J.; Ganesh, A.; McDonough, R.V.; Postma, A.A.; Goldhoorn, R.-J.B.; Majoie, C.B.L.M.; Wijngaard, I.v.D.; Lingsma, H.F.; et al. Benefit of successful reperfusion achieved by endovascular thrombectomy for patients with ischemic stroke and moderate pre-stroke disability (mRS 3): Results from the MR CLEAN Registry. J. Neurointerv. Surg. 2023, 15, 433–438. [Google Scholar] [CrossRef]
  25. Yang, T.; Hu, Y.; Pan, X.; Lou, S.; Zou, J.; Deng, Q.; Zhang, Q.; Zhou, J.; Zhu, J. Interpretable Machine Learning Model Predicting Early Neurological Deterioration in Ischemic Stroke Patients Treated with Mechanical Thrombectomy: A Retrospective Study. Brain Sci. 2023, 13, 557. [Google Scholar] [CrossRef]
  26. De Bin, R.; Benner, A.; Ambrogi, F.; Lusa, L.; Boulesteix, A.-L.; Migliavacca, E.; Binder, H.; Michiels, S.; Sauerbrei, W.; McShane, L.; et al. Statistical analysis of high-dimensional biomedical data: A gentle introduction to analytical goals, common approaches and challenges. BMC Med. 2023, 21, 182. [Google Scholar] [CrossRef]
  27. Statistics for Machine Learning|Packt [Internet]. Available online: https://www.packtpub.com/product/statistics-for-machine-learning/9781788295758 (accessed on 21 May 2024).
  28. Hoffman, H.; Wood, J.; Cote, J.R.; Jalal, M.S.; Otite, F.O.; Masoud, H.E.; Gould, G.C. Development and Internal Validation of Machine Learning Models to Predict Mortality and Disability After Mechanical Thrombectomy for Acute Anterior Circulation Large Vessel Occlusion. World Neurosurg. 2023, 182, E137–E154. [Google Scholar] [CrossRef] [PubMed]
  29. Campagnini, S.; Arienti, C.; Patrini, M.; Liuzzi, P.; Mannini, A.; Carrozza, M.C. Machine learning methods for functional recovery prediction and prognosis in post-stroke rehabilitation: A systematic review. J. Neuroeng. Rehabil. 2022, 19, 54. [Google Scholar] [CrossRef]
  30. Hu, C.; Li, L.; Huang, W.; Wu, T.; Xu, Q.; Liu, J.; Hu, B. Interpretable Machine Learning for Early Prediction of Prognosis in Sepsis: A Discovery and Validation Study. Infect. Dis. Ther. 2022, 11, 1117–1132. [Google Scholar] [CrossRef] [PubMed]
  31. Ting Sim, J.Z.; Fong, Q.W.; Huang, W.; Tan, C.H. Machine learning in medicine: What clinicians should know. Singap. Med. J. 2023, 64, 91–97. [Google Scholar]
  32. Bustamante, A.; Giralt, D.; García-Berrocoso, T.; Rubiera, M.; Álvarez-Sabín, J.; Molina, C.; Serena, J.; Montaner, J. The impact of post-stroke complications on in-hospital mortality depends on stroke severity. Eur. Stroke J. 2016, 2, 54–63. [Google Scholar] [CrossRef]
  33. Izonin, I.; Tkachenko, R.; Shakhovska, N.; Ilchyshyn, B.; Singh, K.K. A Two-Step Data Normalization Approach for Improving Classification Accuracy in the Medical Diagnosis Domain. Mathematics 2022, 10, 1942. [Google Scholar] [CrossRef]
  34. Mukhyber, S.J.; DhahirAbdulhadeAbdulah, A.D.M. Effect Z-score Normalization on Accuracy of classification of liver disease. Turk. J. Comput. Math. Educ. (TURCOMAT) 2021, 12, 658–662. [Google Scholar]
  35. Mihaela, G.; Ruxandra, P.S. The importance of normalization methods for mining medical data. Int. J. Comput. Technol. 2015, 14, 6014–6020. [Google Scholar] [CrossRef][Green Version]
  36. Jain, S.; Saha, A. Rank-based univariate feature selection methods on machine learning classifiers for code smell detection. Evol. Intell. 2021, 15, 609–638. [Google Scholar] [CrossRef]
  37. [PDF] A Two-Phase Feature Selection Method Using Both Filter and Wrapper|Semantic Scholar [Internet]. Available online: https://www.semanticscholar.org/paper/A-two-phase-feature-selection-method-using-both-and-Yuan-Tseng/1d46dc187f99524367948ae7b3683f27f333c36b (accessed on 21 May 2024).
  38. Yap, B.W.; Sim, C.H. Comparisons of various types of normality tests. J. Stat. Comput. Simul. 2011, 81, 2141–2155. [Google Scholar] [CrossRef]
  39. Martínez-Murcia, F.J.; Górriz, J.M.; Ramírez, J.; Puntonet, C.G.; Salas-González, D. Computer Aided Diagnosis tool for Alzheimer’s Disease based on Mann–Whitney–Wilcoxon U-Test. Expert Syst. Appl. 2012, 39, 9676–9685. [Google Scholar] [CrossRef]
  40. Sarkar, S.D.; Goswami, S.; Agarwal, A.; Aktar, J. A Novel Feature Selection Technique for Text Classification Using Naïve Bayes. Int. Sch. Res. Not. 2014, 2014, e717092. [Google Scholar] [CrossRef] [PubMed]
  41. (PDF) Improving the Classification Accuracy Using Recursive Feature Elimination with Cross-Validation [Internet]. Available online: https://www.researchgate.net/publication/344181117_Improving_the_Classification_Accuracy_using_Recursive_Feature_Elimination_with_Cross-Validation (accessed on 21 May 2024).
  42. Lu, X.; Yang, Y.B.; Wu, F.; Gao, M.B.; Xu, Y.; Zhang, Y.B.; Yao, Y.B.; Du, X.B.; Li, C.B.; Wu, L.B.; et al. Discriminative analysis of schizophrenia using support vector machine and recursive feature elimination on structural MRI images. Medicine 2016, 95, e3973. [Google Scholar] [CrossRef] [PubMed]
  43. Liu, A.Y.C. The Effect of Oversampling and Undersampling on Classifying Imbalanced Text Datasets. Available online: https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=cade435c88610820f073a0fb61b73dff8f006760 (accessed on 17 June 2024).
  44. Mohammed, R.; Rawashdeh, J.; Abdullah, M. Machine Learning with Oversampling and Undersampling Techniques: Overview Study and Experimental Results. In Proceedings of the 2020 11th International Conference on Information and Communication Systems (ICICS), Irbid, Jordan, 7–9 April 2020; pp. 243–248. Available online: https://ieeexplore.ieee.org/abstract/document/9078901 (accessed on 22 May 2024).
  45. Chawla, N.V.; Japkowicz, N.; Kotcz, A. Editorial: Special issue on learning from imbalanced data sets. SIGKDD Explor. Newsl. 2004, 6, 1–6. [Google Scholar] [CrossRef]
  46. Batista, G.E.A.P.A.; Prati, R.C.; Monard, M.C. A study of the behavior of several methods for balancing machine learning training data. SIGKDD Explor. Newsl. 2004, 6, 20–29. [Google Scholar] [CrossRef]
  47. Yap, B.W.; Rani, K.A.; Rahman, H.A.A.; Fong, S.; Khairudin, Z.; Abdullah, N.N. An Application of Oversampling, Undersampling, Bagging and Boosting in Handling Imbalanced Datasets. In Proceedings of the First International Conference on Advanced Data and Information Engineering (DaEng-2013), Kuala Lumpur, Malaysia, 16–18 December 2013; Herawan, T., Deris, M.M., Abawajy, J., Eds.; Springer: Singapore, 2014; pp. 13–22. [Google Scholar]
  48. Ling, C.X.; Huang, J.; Zhang, H. AUC: A Better Measure than Accuracy in Comparing Learning Algorithms. In Advances in Artificial Intelligence; Xiang, Y., Chaib-Draa, B., Eds.; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2003; pp. 329–341. [Google Scholar]
  49. Discrimination and Calibration of Clinical Prediction Models: Users’ Guides to the Medical Literature|Users’ Guide to the Medical Literature|JAMA|JAMA Network [Internet]. Available online: https://jamanetwork.com/journals/jama/fullarticle/2656816 (accessed on 22 May 2024).
  50. Jelovsek, J.E.; Hill, A.J.; Chagin, K.M.; Kattan, M.W.; Barber, M.D. Predicting Risk of Urinary Incontinence and Adverse Events after Midurethral Sling Surgery in Women. Obstet. Gynecol. 2016, 127, 330–340. [Google Scholar] [CrossRef]
  51. Rodríguez-Pérez, R.; Bajorath, J. Interpretation of Compound Activity Predictions from Complex Machine Learning Models Using Local Approximations and Shapley Values. J. Med. Chem. 2019, 63, 8761–8777. [Google Scholar] [CrossRef] [PubMed]
  52. Katuwal, G.J.; Chen, R. Machine Learning Model Interpretability for Precision Medicine [Internet]. arXiv 2016. Available online: http://arxiv.org/abs/1610.09045 (accessed on 22 May 2024).
  53. Perez-Vega, C.; Domingo, R.A.; Tripathi, S.; Ramos-Fresnedo, A.; Kashyap, S.; Quinones-Hinojosa, A.; Lin, M.P.; Fox, W.C.; Tawk, R.G. Influence of glucose levels on clinical outcome after mechanical thrombectomy for large-vessel occlusion: A systematic review and meta-analysis. J. Neurointerv. Surg. 2021, 14, 17–21. [Google Scholar] [CrossRef]
  54. Wang, L.; Zhou, Z.; Tian, X.; Wang, H.; Yang, D.; Hao, Y.; Shi, Z.; Lin, M.; Wang, Z.; Zheng, D.; et al. Impact of Relative Blood Glucose Changes on Mortality Risk of Patient with Acute Ischemic Stroke and Treated with Mechanical Thrombectomy. J. Stroke Cerebrovasc. Dis. 2019, 28, 213–219. [Google Scholar] [CrossRef] [PubMed]
  55. Pikija, S.; Sztriha, L.K.; Killer-Oberpfalzer, M.; Weymayr, F.; Hecker, C.; Ramesmayer, C.; Hauer, L.; Sellner, J. Neutrophil to lymphocyte ratio predicts intracranial hemorrhage after endovascular thrombectomy in acute ischemic stroke. J. Neuroinflamm. 2018, 15, 319. [Google Scholar] [CrossRef]
  56. Sharma, D.; Spring, K.J.; Bhaskar, S.M.M. Role of Neutrophil-Lymphocyte Ratio in the Prognosis of Acute Ischaemic Stroke After Reperfusion Therapy: A Systematic Review and Meta-analysis. J. Central Nerv. Syst. Dis. 2022, 14, 11795735221092518. [Google Scholar] [CrossRef]
  57. Desilles, J.-P.; Syvannarath, V.; Ollivier, V.; Journé, C.; Delbosc, S.; Ducroux, C.; Boisseau, W.; Louedec, L.; Di Meglio, L.; Loyau, S.; et al. Exacerbation of Thromboinflammation by Hyperglycemia Precipitates Cerebral Infarct Growth and Hemorrhagic Transformation. Stroke 2017, 48, 1932–1940. [Google Scholar] [CrossRef] [PubMed]
  58. Inflammation and Stroke: An Overview—PubMed [Internet]. Available online: https://pubmed.ncbi.nlm.nih.gov/27730544/ (accessed on 22 May 2024).
  59. Pandhi, A.; Tsivgoulis, G.; Ishfaq, M.F.; Katsanos, A.; Magoufis, G.; Malhotra, K.; Krishnan, R.; Arthur, A.; Hoit, D.; Elijovich, L.; et al. Mechanical thrombectomy outcomes in large vessel stroke with high international normalized ratio. J. Neurol. Sci. 2019, 396, 193–198. [Google Scholar] [CrossRef]
  60. Chen, H.; Ahmad, G.; Colasurdo, M.; Yarbrough, K.; Schrier, C.; Phipps, M.S.; Cronin, C.A.; Mehndiratta, P.; Cole, J.W.; Wozniak, M.; et al. Mildly elevated INR is associated with worse outcomes following mechanical thrombectomy for acute ischemic stroke. J. Neurointerv. Surg. 2022, 15, e117–e122. [Google Scholar] [CrossRef]
  61. Nogueira, R.G.; Liebeskind, D.S.; Sung, G.; Duckwiler, G.; Smith, W.S. Predictors of good clinical outcomes, mortality, and successful revascularization in patients with acute ischemic stroke undergoing thrombectomy: Pooled analysis of the Mechanical Embolus Removal in Cerebral Ischemia (MERCI) and Multi MERCI Trials. Stroke 2009, 40, 3777–3783. [Google Scholar] [CrossRef]
  62. Liebeskind, D.S.; Flint, A.C.; Budzik, R.F.; Xiang, B.; Smith, W.S.; Duckwiler, G.R.; Nogueira, R.G. Carotid I’s, L’s and T’s: Collaterals shape the outcome of intracranial carotid occlusion in acute ischemic stroke. J. Neurointerv. Surg. 2015, 7, 402–407. [Google Scholar] [CrossRef] [PubMed]
  63. Mönch, S.; Boeckh-Behrens, T.; Maegerlein, C.; Berndt, M.; Wunderlich, S.; Zimmer, C.; Friedrich, B. Mechanical Thrombectomy of the Middle Cerebral Artery—Neither Segment nor Diameter Matter. J. Stroke Cerebrovasc. Dis. 2019, 29, 104542. [Google Scholar] [CrossRef] [PubMed]
  64. Pirson, F.A.; Hinsenveld, W.H.; Staals, J.; de Ridder, I.R.; van Zwam, W.H.; Schreuder, T.H.; Roos, Y.B.; Majoie, C.B.; van der Worp, H.B.; Uyttenboogaart, M.; et al. Peripheral Artery Disease in Acute Ischemic Stroke Patients Treated with Endovascular Thrombectomy; Results from the MR CLEAN Registry. Front. Neurol. 2020, 11, 560300. [Google Scholar] [CrossRef] [PubMed]
  65. Meves, S.H.; Diehm, C.; Berger, K.; Pittrow, D.; Trampisch, H.-J.; Burghaus, I.; Tepohl, G.; Allenberg, J.-R.; Endres, H.G.; Schwertfeger, M.; et al. Peripheral arterial disease as an independent predictor for excess stroke morbidity and mortality in primary-care patients: 5-year results of the getABI study. Cerebrovasc. Dis. 2010, 29, 546–554. [Google Scholar] [CrossRef] [PubMed]
  66. Millán, M.; Ramos-Pachón, A.; Dorado, L.; Bustamante, A.; Hernández-Pérez, M.; Rodríguez-Esparragoza, L.; Gomis, M.; Remollo, S.; Castaño, C.; Werner, M.; et al. Predictors of Functional Outcome After Thrombectomy in Patients with Prestroke Disability in Clinical Practice. Stroke 2022, 53, 845–854. [Google Scholar] [CrossRef] [PubMed]
  67. Seker, F.; Pfaff, J.; Schönenberger, S.; Herweh, C.; Nagel, S.; Ringleb, P.; Bendszus, M.; Möhlenbruch, M. Clinical Outcome after Thrombectomy in Patients with Stroke with Premorbid Modified Rankin Scale Scores of 3 and 4: A Cohort Study with 136 Patients. Am. J. Neuroradiol. 2018, 40, 283–286. [Google Scholar] [CrossRef] [PubMed]
  68. Brugnara, G.; Neuberger, U.; Mahmutoglu, M.A.; Foltyn, M.; Herweh, C.; Nagel, S.; Schönenberger, S.; Heiland, S.; Ulfert, C.; Ringleb, P.A.; et al. Multimodal Predictive Modeling of Endovascular Treatment Outcome for Acute Ischemic Stroke Using Machine-Learning. Stroke 2020, 51, 3541–3551. [Google Scholar] [CrossRef] [PubMed]
  69. Asdaghi, N.; Wang, K.; Gardener, H.; Jameson, A.; Rose, D.Z.; Alkhachroum, A.; Gutierrez, C.M.; Ying, H.; Mueller-Kronast, N.; Sur, N.B.; et al. Impact of Time to Treatment on Endovascular Thrombectomy Outcomes in the Early Versus Late Treatment Time Windows. Stroke 2023, 54, 733–742. [Google Scholar] [CrossRef] [PubMed]
  70. Ota, T.; Nishiyama, Y.; Koizumi, S.; Saito, T.; Ueda, M.; Saito, N. Impact of onset-to-groin puncture time within three hours on functional outcomes in mechanical thrombectomy for acute large-vessel occlusion. Interv. Neuroradiol. 2018, 24, 162–167. [Google Scholar] [CrossRef]
  71. Saver, J.L.; Goyal, M.; van der Lugt, A.; Menon, B.K.; Majoie, C.B.L.M.; Dippel, D.W.; Campbell, B.C.; Nogueira, R.G.; Demchuk, A.M.; Tomasello, A.; et al. Time to Treatment with Endovascular Thrombectomy and Outcomes from Ischemic Stroke: A Meta-analysis. JAMA 2016, 316, 1279–1289. [Google Scholar] [CrossRef]
  72. Farghaly, W.M.; Ahmed, M.A.; El-Bassiouny, A.; Hamed, A.A.; Shehata, G.A. Predictors of three months mortality after endovascular mechanical thrombectomy for acute ischemic stroke. Egypt. J. Neurol. Psychiatry Neurosurg. 2022, 58, 96. [Google Scholar] [CrossRef]
  73. Spiotta, A.M.; Vargas, J.; Turner, R.; Chaudry, M.I.; Battenhouse, H.; Turk, A.S. The golden hour of stroke intervention: Effect of thrombectomy procedural time in acute ischemic stroke on outcome. J. Neurointerv. Surg. 2014, 6, 511–516. [Google Scholar] [CrossRef] [PubMed]
  74. Alawieh, A.; Pierce, A.K.; Vargas, J.; Turk, A.S.; Turner, R.D.; Chaudry, M.I.; Spiotta, A.M. The golden 35 min of stroke intervention with ADAPT: Effect of thrombectomy procedural time in acute ischemic stroke on outcome. J. Neurointerv. Surg. 2018, 10, 213–220. [Google Scholar] [CrossRef] [PubMed]
Diagnostics 14 01531 g001
Figure 1. Summarized steps of the analysis. After a two-step feature selection process, with univariate feature selection (UFS) as the first step and Recursive Feature Elimination (RFE) as the second, models were trained, evaluated, and tested. After obtaining the classifiers’ results, features and individual level interpretations were performed for the best-performing classifiers in both models, using SHAP and LIME.
Figure 1. Summarized steps of the analysis. After a two-step feature selection process, with univariate feature selection (UFS) as the first step and Recursive Feature Elimination (RFE) as the second, models were trained, evaluated, and tested. After obtaining the classifiers’ results, features and individual level interpretations were performed for the best-performing classifiers in both models, using SHAP and LIME.
Diagnostics 14 01531 g001
Diagnostics 14 01531 g002
Figure 2. Estimated marginal means graphs of chosen features. Continuous (a) and categorical (b) features have been presented in the graph. Continuous features are represented with their true values, and categorical based on the analyzed categories. Estimated marginal means of continuous features are presented with a solid line, representing the predicted probability of in-hospital death based on the respective variable shown on the x-axis. The gray shaded areas depict the 95% confidence intervals around the estimated marginal means, with a wider gray area indicating more uncertainty and a narrower gray area indicating less uncertainty about the estimate. NIHSS, National Institutes of Health Stroke Scale; NLR, neutrophil-to-lymphocyte ratio; INR, international normalized ratio; OPT, onset-to-puncture time; PET, puncture-to-end time; PAD, peripheral arterial disease; pre-mRS, premorbid modified Rankin scale; MCA, middle cerebral artery; ICA, internal carotid artery.
Figure 2. Estimated marginal means graphs of chosen features. Continuous (a) and categorical (b) features have been presented in the graph. Continuous features are represented with their true values, and categorical based on the analyzed categories. Estimated marginal means of continuous features are presented with a solid line, representing the predicted probability of in-hospital death based on the respective variable shown on the x-axis. The gray shaded areas depict the 95% confidence intervals around the estimated marginal means, with a wider gray area indicating more uncertainty and a narrower gray area indicating less uncertainty about the estimate. NIHSS, National Institutes of Health Stroke Scale; NLR, neutrophil-to-lymphocyte ratio; INR, international normalized ratio; OPT, onset-to-puncture time; PET, puncture-to-end time; PAD, peripheral arterial disease; pre-mRS, premorbid modified Rankin scale; MCA, middle cerebral artery; ICA, internal carotid artery.
Diagnostics 14 01531 g002
Diagnostics 14 01531 g003
Figure 3. Evaluation of the predictive models. The receiver operating characteristic curves (ROCs) for pre-MT (a) and post-MT (c) models. Random guessing is represented by the black dashed line, with an AUC of 50% (0.5). The calibration plots with reliability curves for pre-MT (b) and post-MT (d) classifiers are also visualized, with the black dashed line representing perfect calibration.
Figure 3. Evaluation of the predictive models. The receiver operating characteristic curves (ROCs) for pre-MT (a) and post-MT (c) models. Random guessing is represented by the black dashed line, with an AUC of 50% (0.5). The calibration plots with reliability curves for pre-MT (b) and post-MT (d) classifiers are also visualized, with the black dashed line representing perfect calibration.
Diagnostics 14 01531 g003
Diagnostics 14 01531 g004
Figure 4. The SHAP analysis. The SHAP analysis graphs for in-hospital death prediction for the Pre-MT (a) and the Post-MT (b) models were visualized as absolute mean SHAP values. NIHSS, National Institutes of Health Stroke Scale; NLR, neutrophil-to-lymphocyte ratio; INR, international normalized ratio; PAD, peripheral arterial disease; pre-mRS, premorbid modified Rankin scale; OPT, onset-to-puncture time; PET, puncture-to-end time.
Figure 4. The SHAP analysis. The SHAP analysis graphs for in-hospital death prediction for the Pre-MT (a) and the Post-MT (b) models were visualized as absolute mean SHAP values. NIHSS, National Institutes of Health Stroke Scale; NLR, neutrophil-to-lymphocyte ratio; INR, international normalized ratio; PAD, peripheral arterial disease; pre-mRS, premorbid modified Rankin scale; OPT, onset-to-puncture time; PET, puncture-to-end time.
Diagnostics 14 01531 g004
Diagnostics 14 01531 g005
Figure 5. The LIME analysis. Local Interpretable Model-Agnostic Explanations (LIME) plot from the testing set of the GB (a) and XGB (b) models. True positive (upper examples) and true negative (lower examples) patients were chosen for the interpretation. NIHSS, National Institutes of Health Stroke Scale; INR, international normalized ratio; PAD, peripheral arterial disease; NLR, neutrophil-to-lymphocyte ratio; pre-mRS, premorbid modified Rankin scale; OPT, onset-to-puncture time; PET, puncture-to-end time.
Figure 5. The LIME analysis. Local Interpretable Model-Agnostic Explanations (LIME) plot from the testing set of the GB (a) and XGB (b) models. True positive (upper examples) and true negative (lower examples) patients were chosen for the interpretation. NIHSS, National Institutes of Health Stroke Scale; INR, international normalized ratio; PAD, peripheral arterial disease; NLR, neutrophil-to-lymphocyte ratio; pre-mRS, premorbid modified Rankin scale; OPT, onset-to-puncture time; PET, puncture-to-end time.
Diagnostics 14 01531 g005
Table 1. Whole sample analysis.
Table 1. Whole sample analysis.
VariablesAll Patients (N = 602)Patients with the Miserable Outcome
(N = 133)		Patients with the Favorable Outcome
(N = 469)		p Value
Sociodemographic data
Age, median (IQR)76 (65–83)81 (74–87)74 (62–82)<0.001
Sex, n (%) 0.113
Female328/602 (54%)81/133 (61%)247/469 (53%)
Male274/602 (46%)52/133 (39%)222/469 (47%)
Days to In-hospital death (IQR) 6 (2.5–14)
Vascular risk factors and comorbidities
Previous stroke, n (%)92/602 (15%)32/133 (24%)60/469 (13%)0.002
Peripheral arterial disease (PAD), n (%)33/602 (5%)19/133 (14%)14/469 (3%)<0.001
Tobacco smoking n (%)381 (63%)81 (61%)300 (64%)0.393
Atrial fibrillation, n (%)237/602 (39%)62/133 (47%)175/469 (37%)0.066
Diabetes mellitus, n (%)90/602 (15%)26/133 (20%)64/469 (14%)0.122
Arterial hypertension, n (%)433/602 (72%)103/133 (77%)330/469 (70%)0.135
Internal carotid artery (ICA) stenosis, n (%)86/602 (14%)24/133 (18%)62/469 (13%)0.206
Ischemic heart disease, n (%)110/602 (18%)31/133 (23%)79/469 (17%)0.115
Kidney failure, n (%)53/602 (9%)22/133 (17%)31/469 (6%)0.001
Heart failure, n (%)75/602 (12%)22/133 (17%)53/469 (11%)0.142
Statins usage, n (%)149/602 (25%)31/133 (23%)118/469 (25%)0.747
Antiplatelets drug usage, n (%)158/602 (26%)38/133 (29%)120/469 (26%)0.563
Anticoagulants usage, n (%)58/602 (10%)24/133 (18%)34/469 (7%)<0.001
Direct oral anticoagulants usage, n (%)57/602 (9%)20/133 (15%)37/469 (8%)0.02
Baseline laboratory values
C-reactive protein (CRP) (mg/dL), median (IQR)0.39 (0.18–0.91)0.42 (0.19–1.48)0.39 (0.17–0.81)0.008
Glycemia (mg/dL), median (IQR)120 (107–143)124 (112–157)119 (106–141)0.004
HbA1c (%), median (IQR)5.7 (5.5–5.9)5.7 (5.5–5.9)5.7 (5.4–5.9)0.167
Creatinine (mg/dL), median (IQR)0.89 (0.77–1.08)0.93 (0.82–1.25)0.88 (0.76–1.05)0.001
Cholesterol (mg/dL), median (IQR)153 (134–177)153 (132–158)153 (134–181)0.025
Triglycerides (mg/dL), median (IQR)100 (79–129)100 (81–122)100 (78–133)0.405
High-density lipoprotein (HDL) (mg/dL), median (IQR)46 (39–53)46 (40–49)46 (39–54)0.708
Low-density lipoprotein (LDL) (mg/dL), median (IQR)95 (78–112)95 (73–98)95 (78–116)0.002
White Blood Cells (WBC) (×109/L), median (IQR)8.59 (7.00–10.8)8.59 (7.08–11.89)8.59 (6.99–10.76)0.107
Neutrophils (×109/L), median (IQR)6.27 (4.58–8.23)6.27 (4.98–8.88)6.02 (4.52–8.11)0.005
Lymphocytes (×109/L), median (IQR)1.55 (1.10–2.10)1.40 (0.91–1.97)1.63 (1.15–2.11)0.001
Red Blood Cells (RBC) (×1012/L), median (IQR)4.41 (4.07–4.74)4.41 (4.01–4.81)4.41 (4.09–4.72)0.447
Hematocrit (HCT) (%), median (IQR)39.2 (36.0–42.2)39.20 (35.3–42.4)39.20 (36.20–42.20)0.566
Platelets (PLT) (×109/L), median (IQR)224 (188–273)224 (187–284)224 (188–269)0.368
Fibrinogen (mg/dL), median (IQR)341 (286–399)359 (315–446)341 (277–394)0.002
International normalized ratio (INR), median (IQR)1.20 (1.20–1.20)1.20 (1.20–1.21)1.20 (1.20–1.20)0.009
Neutrophil-to-lymphocyte ratio (NLR), median (IQR)4.06 (2.41–6.50)3.95 (2.86–8.67)3.95 (2.29–5.86)0.001
Systemic inflammatory index (SII), median (IQR)908 (511–1522)919 (618–1951)918 (494–1395)0.006
Clinical data
pre-mRS ≥ 2, n (%)91/602 (15%)32/133 (24%)59/469 (13%)0.002
Baseline NIHSS, median (IQR)18 (12–22)20 (16–24)17 (11–21)<0.001
Wake-up stroke, n (%)114/602 (19%)32/133 (24%)82/469 (17%)0.113
ASPECTS > 6, n (%)536/602 (89%)116/133 (87%)420/469 (90%)0.546
TOAST
Cardioembolic cause (CE), n (%)331/602 (55%)86/133 (65%)245/469 (52%)0.015
Large artery atherosclerosis (LAA), n (%)81/602 (13%)12/133 (9%)69/469 (15%)0.120
Other, or unknown cause, n (%)190/602 (32%)35/133 (26%)155/469 (33%)0.171
Thrombolysis, n (%)361/602 (60%)64/133 (48%)297/469 (63%)0.002
Occluded vessel and leptomeningeal collaterals
Vessel type, n (%)
Middle Cerebral Artery-M1 segment335/602 (56%)55/133 (41%)280/469 (60%)<0.001
Middle Cerebral Artery-M2 segment47/602 (8%)11/133 (8%)36/469 (8%)0.966
Internal Carotid Artery-I type93/602 (15%)20/133 (15%)73/469 (16%)0.990
Internal Carotid Artery-L type33/602 (5%)12/133 (9%)21/469 (4%)0.069
Internal Carotid Artery-T type94/602 (16%)35/133 (26%)59/469 (13%)<0.001
Vessel side, n (%) 0.567
Right287/602 (48%)60/133 (45%)227/469 (48%)
Left315/602 (52%)73/133 (55%)242/469 (52%)
Leptomeningeal collaterals (LC), n (%) 0.001
Good or Equal to the unaffected side454/602 (75%)85/133 (64%)369/469 (79%)
Absent148/602 (25%)48/133 (36%)100/469 (21%)
Endovascular procedure information
Onset-to-puncture time (OPT)192 (154–270)200 (152–329)195.0 (140–285)0.181
Puncture-to-end time (PET), median (IQR)52 (38–66)62 (54–90)55 (33–79)0.358
Onset to the procedure end time (OPET)264 (219–318)276 (237–389)269 (205–343)0.269
Number of steps, median (IQR)2 (1–3)3 (2–4)2 (1–4)0.011
Procedure type, n (%)
Failed/Abandoned thrombectomy, n (%)55/602 (9%)11/133 (8%)44/469 (9%)0.824
Aspiration only229/602 (38%)41/133 (31%)188/469 (40%)0.066
Stent-retriever only24/602 (4%)5/133 (4%)19/469 (4%)1.000
Aspiration and stent-retriever294/602 (49%)76/133 (57%)218/469 (46%)0.038
Procedure complications, n (%)
No complications586/602 (97%)129/133 (97%)457/469 (97%)1.000
Downstream complications11/602 (2%)3/133 (2%)8/469 (2%)0.959
Distal complications5/602 (1%)1/133 (1%)4/469 (1%)1.000
Vessel perforation45/602 (7%)8/133 (6%)37/469 (8%)0.590
Bleeding type, n (%)
No bleeding415/602 (69%)85/133 (64%)330/469 (70%)0.189
Symptomatic bleeding18/602 (3%)1/133 (1%)17/469 (4%)0.153
Any other bleeding169/602 (28%)47/133 (35%)122/469 (26%)0.045
Thrombolysis in Cerebral Infarction (TICI), n (%) 0.040
Incomplete recanalization (TICI = 0-2a)152/602 (25%)24/133 (18%)128/469 (27%)
Complete recanalization (TICI = 2b-3)450/602 (75%)109/133 (82%)341/469 (73%)
Internal carotid artery stenting, n (%)34/602 (6%)11/133 (8%)23/469 (5%)0.203
Osteoclastic decompressive craniectomy, n (%)29/602 (5%)8/133 (6%)21/469 (4%)0.616
IQR, interquartile range; mRS, modified Rankin scale; NIHSS, the National Institutes of Health Stroke Scale; ASPECTS, Alberta stroke program early CT score; TOAST, Trial of ORG 10,172 in Acute Stroke Treatment.
Table 2. Binomial logistic regression results of the chosen parameters associated with in-hospital death after mechanical thrombectomy for anterior large vessel occlusion stroke, N = 602.
Table 2. Binomial logistic regression results of the chosen parameters associated with in-hospital death after mechanical thrombectomy for anterior large vessel occlusion stroke, N = 602.
PredictorEstimate95% Confidence IntervalSEzp ValueOdds Ratio95% Confidence Interval
LowerUpperLowerUpper
Age0.0530.0340.0740.0105.289<0.0011.0611.0341.076
Baseline NIHSS0.0710.0370.1050.0174.104<0.0011.0861.0381.111
Glycemia0.0050.0000.0090.0022.3170.0201.0071.0011.009
NLR0.0550.0160.0950.0202.7360.0061.0711.0161.100
INR1.0500.3981.7100.3343.150<0.0012.8661.4885.517
PAD1.5050.7512.2590.3843.910<0.0014.5042.1209.569
pre-mRS0.5900.0741.1050.2632.2400.0251.8031.0773.019
OPT0.0000.0000.0010.0000.2580.7961.0010.9991.001
PET0.0020.0010.0060.0021.0490.2941.0020.9981.006
SE, standard error; NIHSS, National Institutes of Health Stroke Scale; NLR, Neutrophil-to-lymphocyte ratio; INR, international normalized ratio; PAD, peripheral arterial disease; pre-mRS, premorbid modified Rankin scale; OPT, onset-to-puncture time; PET, puncture-to-end time.
Table 3. Evaluation of the classifiers associated with in-hospital death after mechanical thrombectomy for anterior large vessel occlusion stroke, N = 602.
Table 3. Evaluation of the classifiers associated with in-hospital death after mechanical thrombectomy for anterior large vessel occlusion stroke, N = 602.
ClassifierAccuracyPrecisionRecallF1-ScoreAUC-ROCBrier Score
Pre-MT ModelLR0.72220.70830.68000.69390.78210.2205
RF0.72220.72730.64000.68090.76480.2058
GB0.66670.62070.72000.66670.79170.2009
XGB0.70370.68000.68000.68000.75030.2182
Post-MT ModelLR0.72220.72730.64000.68090.78480.1328
RF0.70730.68000.68000.68000.81450.1192
GB0.68520.66670.64000.65310.79310.1197
XGB0.74070.72000.72000.72000.83720.1194
MT, mechanical thrombectomy; LR, logistic regression; RF, random forest; GB, gradient boosting; XGB, extreme gradient boosting; AUC-ROC, the area under the receiver operating characteristic curve.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Petrović, I.; Broggi, S.; Killer-Oberpfalzer, M.; Pfaff, J.A.R.; Griessenauer, C.J.; Milosavljević, I.; Balenović, A.; Mutzenbach, J.S.; Pikija, S. Predictors of In-Hospital Mortality after Thrombectomy in Anterior Circulation Large Vessel Occlusion: A Retrospective, Machine Learning Study. Diagnostics 2024, 14, 1531. https://doi.org/10.3390/diagnostics14141531

AMA Style

Petrović I, Broggi S, Killer-Oberpfalzer M, Pfaff JAR, Griessenauer CJ, Milosavljević I, Balenović A, Mutzenbach JS, Pikija S. Predictors of In-Hospital Mortality after Thrombectomy in Anterior Circulation Large Vessel Occlusion: A Retrospective, Machine Learning Study. Diagnostics. 2024; 14(14):1531. https://doi.org/10.3390/diagnostics14141531

Chicago/Turabian Style

Petrović, Ivan, Serena Broggi, Monika Killer-Oberpfalzer, Johannes A. R. Pfaff, Christoph J. Griessenauer, Isidora Milosavljević, Ana Balenović, Johannes S. Mutzenbach, and Slaven Pikija. 2024. "Predictors of In-Hospital Mortality after Thrombectomy in Anterior Circulation Large Vessel Occlusion: A Retrospective, Machine Learning Study" Diagnostics 14, no. 14: 1531. https://doi.org/10.3390/diagnostics14141531

APA Style

Petrović, I., Broggi, S., Killer-Oberpfalzer, M., Pfaff, J. A. R., Griessenauer, C. J., Milosavljević, I., Balenović, A., Mutzenbach, J. S., & Pikija, S. (2024). Predictors of In-Hospital Mortality after Thrombectomy in Anterior Circulation Large Vessel Occlusion: A Retrospective, Machine Learning Study. Diagnostics, 14(14), 1531. https://doi.org/10.3390/diagnostics14141531

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop