Next Article in Journal
Classification of Chemotherapy-Related Subjective Cognitive Complaints in Breast Cancer Using Brain Functional Connectivity and Activity: A Machine Learning Analysis
Next Article in Special Issue
Using “Diraya” System as a Complementary Tool in Nursing Process Education: A Controlled Clinical Study
Previous Article in Journal
Clinical Applications of Artificial Intelligence—An Updated Overview
Previous Article in Special Issue
Early Virtual-Reality-Based Home Rehabilitation after Total Hip Arthroplasty: A Randomized Controlled Trial
 
 
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Prediction of Poststroke Depression Based on the Outcomes of Machine Learning Algorithms

1
Department of Rehabilitation Medicine, College of Medicine, Dankook University, Cheonan 31116, Korea
2
Welfare & Medical ICT Research Department, Electronics and Telecommunications Research Institute, Daejeon 34129, Korea
3
Department of Nanobiomedical Science & BK21 NBM Global Research Center for Regenerative Medicine, Dankook University, Cheonan 31116, Korea
4
Institute of Tissue Regeneration Engineering (ITREN), Dankook University, Cheonan 31116, Korea
*
Authors to whom correspondence should be addressed.
These authors contributed equally to this work.
J. Clin. Med. 2022, 11(8), 2264; https://doi.org/10.3390/jcm11082264
Received: 8 February 2022 / Revised: 15 April 2022 / Accepted: 16 April 2022 / Published: 18 April 2022
(This article belongs to the Special Issue Ehealth, Telemedicine and AI in Clinical Medicine)

Abstract

:
Poststroke depression (PSD) is a major psychiatric disorder that develops after stroke; however, whether PSD treatment improves cognitive and functional impairments is not clearly understood. We reviewed data from 31 subjects with PSD and 34 age-matched controls without PSD; all subjects underwent neurological, cognitive, and functional assessments, including the National Institutes of Health Stroke Scale (NIHSS), the Korean version of the Mini-Mental Status Examination (K-MMSE), computerized neurocognitive test (CNT), the Korean version of the Modified Barthel Index (K-MBI), and functional independence measure (FIM) at admission to the rehabilitation unit in the subacute stage following stroke and 4 weeks after initial assessments. Machine learning methods, such as support vector machine, k-nearest neighbors, random forest, voting ensemble models, and statistical analysis using logistic regression were performed. PSD was successfully predicted using a support vector machine with a radial basis function kernel function (area under curve (AUC) = 0.711, accuracy = 0.700). PSD prognoses could be predicted using a support vector machine linear algorithm (AUC = 0.830, accuracy = 0.771). The statistical method did not have a better AUC than that of machine learning algorithms. We concluded that the occurrence and prognosis of PSD in stroke patients can be predicted effectively based on patients’ cognitive and functional statuses using machine learning algorithms.

1. Introduction

Poststroke depression (PSD) is one of the most common psychiatric disorders in stroke patients [1,2]. Its incidence ranges from 10 to 52% according to subject selection or diagnostic criteria [2,3,4,5,6]. The pathophysiology of PSD is not obvious, although it might be associated with the secondary effects of psychological distress and cognitive impairment, not stroke itself [7]. The relationship between PSD and cognitive impairment has already been widely investigated. Cognitive impairment, in fact, is known to be one of the major predictors of PSD [6,8,9]; moreover, PSD is associated with a greater degree of cognitive impairment and has a negative impact on the activities of daily living (ADL) that are integral to recovery [4,5,10,11]. Based on the strong relationship between PSD and cognitive impairment, reductions in PSD symptoms might also enhance patients’ cognitive and functional recovery [12], although the treatment effect remains unclear. PSD is also associated with functional impairment, and rehabilitation is less effective; moreover, hospital stays are longer in PSD patients than in stroke patients without depression [13,14]. Some studies have reported that treatment for PSD also affects cognitive and functional improvements [4,12]. However, others have found no improvement in cognitive impairment, albeit significant reductions in depressive symptoms in stroke patients were noted [15,16]. Machine learning (ML) algorithms are widely used in the medical field for the prediction of disease diagnosis and treatments. Some recent studies reported that the ML method is more effective in predicting stroke outcomes than statistical methods or scoring systems [17,18,19]. In addition, various psychiatric disorders after stroke, including depression, were predictable using ML algorithms [20]. Some predictors for the treatment outcome of PSD were analyzed statistically in a previous study [21]; however, no studies have used an ML algorithm to predict PSD in stroke patients in combination with the treatment outcome of PSD based on comprehensive cognitive and functional analysis.
In this study, we aimed to use various ML algorithms to predict the occurrence and prognosis of PSD in stroke patients based on their cognitive and functional status and evaluated whether ML algorithms are superior to statistical methods.

2. Subjects and Methods

2.1. Subjects

A total of 623 patients who had a first-ever hemorrhagic or ischemic stroke were reviewed, and finally, 31 patients who were diagnosed with PSD on admission were included (Figure 1). PSD was confirmed by psychiatrists using the Diagnostic and Statistical Manual of Mental Disorders, fourth edition (DSM-IV) for major depressive disorder [22]. Any patients with any of the following conditions were excluded: preexisting major depressive disorder, dementia, Parkinson’s disease, any other brain lesions, no follow-up with a psychiatrists’ assessment for the progress of PSD, or no follow-up with a computerized neurocognitive test (CNT). Over the course of 4 weeks following their diagnosis, all PSD patients received psychiatric and medical treatments, including antidepressants (i.e., escitalopram, amitriptyline, or fluoxetine). A total of 35 age-matched patients who had a first-ever stroke without PSD were recruited as controls, and their cognitive and functional changes were compared with those of PSD patients.

2.2. Measurements

Basic characteristics including medical and family history, education period, and smoking habits were documented, and neurological, cognitive, and functional tests were performed on all subjects, including those in the control (n = 34) and PSD (n = 31) groups, at admission to a rehabilitation unit in the subacute stage following stroke and 4 weeks after initial assessments.

2.2.1. Neurological Assessment

A neurological assessment scored on the National Institutes of Health Stroke Scale (NIHSS) [23] was performed. In addition, the type of stroke, which was classified as hemorrhagic or ischemic, and the laterality of stroke were evaluated by magnetic resonance imaging.

2.2.2. Psychological Assessment

When subjects were admitted to a rehabilitation unit, the results of the Hamilton Rating Scale for Depression, a test commonly used to screen for and identify depression [24], were evaluated by a rehabilitation doctor, and the cutoff score was 10 [3]. Subsequently, a psychiatrist interviewed selected patients and diagnosed them with major depressive disorder according to the criteria for the DSM-IV (which specifies more than five different symptoms of depression from a list of nine, at least one of which was either depressed mood or loss of interest or pleasure for more than two weeks) [22,25]. All of the PSD patients were followed up 4 weeks later by the same psychiatrist to assess them for any improvements in their depressive symptoms. According to the psychiatrist’s follow-up results, PSD patients were divided into improved (Imp) and not improved (NoImp) groups (Figure 1).

2.2.3. Cognitive Assessments

The Korean version of the Mini-Mental Status Examination (K-MMSE) and CNT were assessed in all subjects at admission to a rehabilitation unit, and participants were followed up 4 weeks after the initial assessment. The K-MMSE was modified from the original MMSE [26,27] by Kang et al. [28] and consists of 11 questions in the following five categories: orientation to place and time, registration, recall, attention and calculation, and language and complex commands; the total score ranges from 0 to 30. The CNT consists of 20 items in 4 major subtests: visual memory, language memory, visual perception, and language perception [29]. Each item is scored on a scale ranging from T-scores of 27 to 80. The total score ranges from 108 to 1600.

2.2.4. Functional Assessments

The severity of impairment in ADL performance was evaluated using the Korean version of the Modified Barthel index (K-MBI) and functional independence measure (FIM) for all subjects at admission to a rehabilitation unit, and participants were followed up 4 weeks after the initial assessment. The K-MBI is composed of 10 items: hygiene (grooming), bathing, eating, toileting, stair-climbing, dressing, bowel control, bladder control, toilet transfer, and ambulation; scores range from 0 (completely dependent) to 100 (independent in basic ADL) [30,31]. The FIM is composed of 18 items and divided into six areas: self-care, sphincter control, transfer, locomotion, communication, and social. Scores range from 18 (completely dependent) to 126 (independent in basic ADL) [32].

2.3. ML Analysis

We used two sets of data: control vs. PSD and patients in the Imp vs. NoImp groups (Figure 2). For the process of feature selection, baseline characteristics and the initial cognitive and functional data were used initially. Input features were reduced repeatedly one by one, from the least important to the last, which is known as the wrapper method. This was done to select the best set of features for prediction based on feature importance; it turned out that performance was better when this approach was used. The prediction of PSD occurrence and prognosis was developed using 5 ML models of support vector machine linear (SVM_L), support vector machine with radial basis function (RBF) kernel (SVM_R), k-nearest neighbors (KNN), random forest (RF), and a voting ensemble (VE) algorithm from a retrospective study that included all subjects. Then, we assessed model performance using accuracy and receiver operating characteristic (ROC) curves. The average accuracy was improved from 53.21% to 60.61% in the SVM model by the wrapper method. The wrapper method was used to select the most important predictors, and data were analyzed by ML algorithms with 5- and 10-fold cross-validation. In 10-fold cross-validation, groups were randomly shuffled and partitioned into 10 groups, each of which was used as the test set, while the remaining nine were used for training. A total of 13 randomly selected test datasets were used for the prediction of PSD occurrence and prognosis. Decision tree classifiers are commonly used to provide a descriptive representation of a classifier. The inner nodes of the decision tree represent features, and the branches represent decision rules. In this paper, we used a decision tree classifier from the scikit-learn package with Gini impurity as a partitioning criterion [33].

2.4. Statistics

Statistical analyses were performed using PASW Statistics 18 for Windows (IBM Corp., New York, NY, USA). The Shapiro–Wilk test was performed to assess the normal distribution of all numerical data from each group. The likelihood ratio was obtained to analyze baseline categorical data, such as sex and the cause and laterality of stroke on initial assessment, among other aspects. The Mann–Whitney U test was used to compare numerical data of age, educational period, NIHSS score, and time since stroke between participants in the control and PSD groups and between participants with PSD who showed improvement in their symptoms and between participants with PSD without improvement in their symptoms. The Mann–Whitney U test was performed to compare the initial, follow-up, and gain values of the K-MMSE, CNT, K-MBI, and FIM total and subtest scores between participants in the control and PSD groups and between participants with PSD who showed improvement in their symptoms and participants with PSD without improvement in their symptoms. A Wilcoxon sum rank test was conducted to compare the initial values to the follow-up values of K-MMSE, CNT, FIM, and K-MBI total and subtest scores in the same subjects in the control and PSD groups. The Spearman rank correlation analysis was performed using statistically significant parameters that were determined by the Mann–Whitney U test between participants in the control and PSD groups and participants with PSD who showed improvement and participants with PSD without improvement in their symptoms. The logistic regression analysis was performed, and significant variables for the prediction of PSD (control vs. PSD groups) and PSD prognosis (Imp vs. NoImp) were determined using the forward Wald method. The predictive performance was considered based on the area under the ROC curve (AUC) with their 95% confidence intervals (CIs), sensitivity values, and specificity values. Statistical significance was set at p < 0.05.

3. Results

3.1. Baseline Characteristics

Among the 31 PSD patients, depression symptoms were improved in 13 patients (41.9%) 4 weeks after their initial assessment. The number of positive depressive symptoms among the nine-item depression module based on DSM-IV [34] showed no difference between NoImp and Imp groups initially (6.06 ± 1.39 and 6.69 ± 1.60 for NoImp and Imp groups, respectively), but the Imp group showed a smaller number of depressive symptoms at follow-up period (6.67 ± 1.53 and 2.08 ± 2.25 for NoImp and Imp groups, respectively, Table 1). We found that there were no differences in age, sex, educational period, onset from stroke to initial evaluation, type and laterality of stroke, NIHSS score, family history, mental disorder history, smoking year, history of diabetes mellitus or hypertension between participants in the control and PSD groups, and the educational period was different between participants in the NoImp and Imp groups (Table 1).

3.2. Cognitive and Functional Analysis Using Statistical Methods

Patients’ cognitive status was expressed as the total K-MMSE score (14.0 ± 8.4 and 14.1 ± 7.7 for participants in the control and PSD groups, respectively) and CNT score (419.1 ± 180.1 and 390.1 ± 127.8 for participants in the control and PSD groups, respectively) were not different between the groups initially (Table 2). Initial functional status based on the total scores of the K-MBI (25.7 ± 25.2 and 19.8 ± 15.5 for participants in the control and PSD groups, respectively) and FIM (46.3 ± 22.9 and 44.1 ± 15.5 for participants in the control and PSD groups, respectively) were also not different between participants in the control and PSD groups. During the 4-week follow-up period, the K-MMSE, K-MBI, and FIM total scores were improved for participants in both the control and PSD groups without differences between groups, but the CNT total score was not changed from the initial values (Table 2). The follow-up total K-MBI and FIM scores were lower for participants in the PSD group (40.1 ± 19.7 and 59.1 ± 17.5, respectively) than for participants in the control group (46.5 ± 28.6 and 64.7 ± 28.2, respectively). When comparing the PSD patients in the Imp group with those in the NoImp group, follow-up K-MMSE total scores were higher for patients in the Imp group (21.6 ± 6.8) than for those in the NoImp group (16.1 ± 8.2) (Table 2).
The detailed scores of the K-MMSE were divided into five categories. All categories except registration at the follow-up period showed improvement from initial values, but there was no difference between participants in the control and PSD groups (Table 3). When comparing participants with PSD in the Imp group with those in the NoImp group, the recall subscores of participants in the Imp group at the follow-up period (2.8 ± 2.2) were higher than those of participants in the NoImp group (0.9 ± 1.0) (Table 3).
Among the 12 subtests of the CNT, the initial visual attention omission error score of participants in the PSD group (30.5 ± 12.0) was lower than that of participants in the control group (38.2 ± 18.2), and the gain scores of auditory attention correct time standard deviation (SD) and visual attention commission error of participants in the PSD group (5.9 ± 12.9 and 5.4 ± 15.5, respectively) were different from those of participants in the control group (−7.5 ± 21.7 and 9.3 ± 13.9, respectively) (Table 4). The follow-up scores of most categories except auditory and visual attention omission error were improved for participants in the control group, whereas the follow-up scores of only four subtests, such as digit span backward (DSB) language memory, auditory attention correct response, and commission error, and visual attention correct response, improved for participants in the PSD group (Table 4).
Among the 10 items of the K-MBI, the initial subscore of dressing was significantly lower for participants in the PSD group (2.0 ± 1.9) than for participants in the control group (3.4 ± 2.7). In addition, follow-up subscores of bathing, toileting, stair-climbing, dressing, bladder control, transfer and ambulation, and gain subscores of bladder control were lower for participants in the PSD group than for participants in the control (Table 5). However, there was no difference in the initial, follow-up, or gain subscores of all items between participants with PSD in the Imp and NoImp groups (Table 5).
Among the six areas in the FIM, the follow-up subscores of self-care, transfer, and locomotion were significantly lower for participants in the PSD group (17.8 ± 7.3, 8.3 ± 3.9, and 4.3 ± 3.3, respectively) than for participants in the control group (23.2 ± 10.2, 12.2 ± 5.5 and 5.9 ± 3.7, respectively). In addition, all initial, follow-up, and gain subscores for participants in the Imp and NoImp groups were not significantly different (Table 6).
As a result of the correlation analysis between featured parameters, a strong correlation was observed between the total and subscores of the functional tests (K-MBI and FIM) in the control and PSD groups (Supplementary Table S1), and the total and subscores of the cognitive tests (K-MMSE and CNT) in the Imp and NoImp groups (Supplementary Table S2). Then, logistic regression analysis was performed, and two parameters, the initial subscore of auditory attention omission error on the CNT (CNT1_AA OE) and the initial subscore of bathing on the K-MBI (MBI1_Bat) for the comparison of participants in the control and PSD groups, and one parameter, educational period for the comparison of participants in the Imp and NoImp groups, were included. The AUC and accuracy used to classify the control and PSD groups were 0.706 and 0.696, respectively, and those used to classify the Imp and NoImp groups were 0.797 and 0.778, respectively (Table 7).

3.3. ML Analysis

The featured parameters were selected through a wrapper method, and each parameter was ranked by importance using support vector machine linear model-based recursive feature elimination (Table 8). The five featured parameters (MBI1_Amb, CNT1_AA OE, MMSE1_Rec, MBI1_Dre, FIM1_Loc) for the prediction of PSD (control vs. PSD groups) and five parameters (MBI1_Bla, MBI1_Bow, FIM1_Tra, CNT1_VM VSB, FIM1_Com) for the prediction of PSD prognosis (Imp vs. NoImp) were used for ML analysis.
When five ML algorithms (SVM_L, SVM_R, KNN, RF, and VE) with 5- or 10-fold cross-validation were compared, SVM_R with 10-fold cross-validation showed the best AUC for the prediction of PSD occurrence (0.711), and SVM_L with five-fold cross-validation showed the best AUC for the prediction of PSD prognosis (0.830) (Table 9). Accuracies of SVM_R with 10-fold cross-validation for the prediction of PSD occurrence and SVM_L with five-fold cross-validation for the prediction of PSD prognosis were 7.000 and 0.771, respectively, and which were analyzed using hyper-parameters (Supplementary Table S3A,B). SVM_L with 10-fold cross-validation showed the best sensitivity (0.775) for the prediction of PSD occurrence and showed the best sensitivity (0.650) and specificity (0.950) for the prediction of PSD prognosis (Table 9).
The ROC curves of each ML algorithm and logistic regression analysis are shown in Figure 3. The mean AUC of SVM_R with 10-fold cross-validation for the prediction of PSD occurrence was 0.71 ± 0.12, which was comparable to that of logistic regression analysis (0.71 ± 0.07). The mean AUC of SVM_L with five-fold cross-validation for the prediction of PSD prognosis was also higher than that of logistic regression analysis (0.83 ± 0.14 vs. 0.80 ± 0.09) (Figure 3).

3.4. Decision-Making Model for the Prediction of PSD Occurrence and Prognosis

Decision tree classification models for the prediction of PSD occurrence were created using initial values of featured parameters, as shown in Table 8 (Figure 4). From this analysis, we can recursively construct a tree structure in which the input featured parameters and their values can be precisely assigned a given label by generating an appropriate partition and final decision of PSD for clinical use.
We found that the initial subscore of auditory attention omission error on the CNT was identified as the first single discriminator for group determination between participants in the control and PSD groups (Figure 4). The initial ADL and locomotor functions such as dressing and transfer on the K-MBI were also important discriminators to predict PSD occurrence (Figure 4).

4. Discussion

In this study, we found that numerous cognitive and functional statuses were associated with the occurrence of PSD in stroke patients (Table 5 and Table 6). The recall and auditory attention omission error among the subitems of the MMSE and CNT, and dressing and locomotor function including ambulation among the subitems of the K-MBI and FIM were considered to be important features to predict the occurrence of PSD ((A) in Table 8). From decision-making models, we found that initial scores of visual and auditory attention were important for the prediction of PSD occurrence. A previous study revealed that the severity of depression and the decrease in visual and auditory tension tend to correlate to a weak degree in PSD patients [35], and depressive patients also showed impaired visual attention omission errors in a meta-analysis [36].
PSD is frequently seen in stroke patients, and it might worsen their cognitive and functional recovery and quality of life [37]. The prevention of PSD has been suggested in previous studies using various nonpharmacological modalities and antidepressants [38]; however, the evidence is very limited, and more clinical trials are needed to confirm effective prevention methods for PSD [39]. There is no doubt that early detection and treatment of PSD can help improve a patient’s prognosis; therefore, it is important to identify modifiable risk factors and their application to stroke patients. From previous studies, major risk factors from meta-analysis still debated according to researchers were identified as follows: previous history of mental disorders, including depression or anxiety; diabetes mellitus; cognitive impairment and functional deficits, including impairment in ADL; and other factors, such as old age, female sex, lesion location, and stroke type [40,41]. It should be noted here that risk factors, which are determined by statistical methods, cannot be used to develop a predictive model directly because risk factors contain results, course, or complications of target diseases, as well as causative factors and can be used as an explanatory model [42].
ML methods are a useful tool to overcome the limitations of statistical methods and help us to find predictive models [43]. Most previous ML studies on stroke patients have focused on the prediction of stroke occurrence or outcome [44], and only one study has revealed the relationship between stroke and mood disorders, including depression, apathy, and anxiety, using ML analysis [20]. In this study, we used various ML algorithms to predict PSD occurrence and prognosis for the first time, and the SVM linear and SVM with an RBF kernel were optimal to develop a predictive model for PSD. The RBF kernel is commonly used in SVM classification, has the advantages of the KNN algorithm, and overcomes challenges of using the RBF alone, such as the space complexity problem [45]. We also tried to apply other ML algorithms, including KNNs and RFs, which were effective in predicting stroke occurrence and prognosis in previous studies [44]. The VE algorithm is the combination of all ML models that were used in this study to improve model performance; however, its AUC and accuracy were not higher than those of the SVM linear or SVM with the RBF kernel (Table 9). The VE model uses multiple models for the analysis, which might contain pros and cons of each model, and individual algorithms, which are superior to other algorithms, can show better performance.
Whereas cognitive impairment and PSD are highly connected with each other, the effect of PSD treatment on cognitive improvement was not clear in previous studies. In one study, improved PSD patients showed greater cognitive improvement than nonimproved PSD patients showed [46], but in another study, PSD treatment only helped to improve attention [47]. Previous studies also revealed that motor recovery is obvious in treated PSD patients [48]. In our study, cognitive impairments such as visual memory visual span backward of the CNT, were closely related to improvements in patients with PSD ((B) in Table 8); however, further studies are needed to further confirm these relationships. We found that the educational period was strongly associated with recovery from PSD in the logistic regression analysis (AUC = 0.80). In previous studies, educational level might be associated with anxiety, depression [49], and poststroke cognitive impairment [50] and be a protective factor against PSD [51,52].
The statistical method of logistic regression analysis did not show a higher AUC than the optimal ML algorithm for the prediction of PSD occurrence and prognosis (Figure 3). During statistical processing of logistic regression, only a few parameters were included for the comparison of participants in the control and PSD groups or the Imp and NoImp groups, whereas ML algorithms included five parameters for the detection of AUC and accuracy. The sample size was important to reduce bias in regression coefficients [53], and a smaller sample size might have influenced the low AUC and accuracy of the regression analysis in this study. Nevertheless, ML algorithms in this study showed comparable performance to the existing statistical methods and might have been a suitable method to overcome a small sample size.
The SVM linear and SVM with RBF showed the best AUC among various ML algorithms for the prediction of PSD occurrence and prognosis; however, the specificity for the prediction of PSD occurrence and the sensitivity for the prediction of PSD prognosis was low, which means that these models might not accurately detect PSD among stroke patients or PSD patients whose symptoms improved depending on the cut-off selected. We studied 65 stroke patients, including controls without PSD, for the development of ML algorithm-based prediction models of PSD occurrence and prognosis; notably, many previous studies could not evaluate more than 60 stroke patients [16,54,55]. Small sample sizes can cause some problems, such as generalization, in the field of ML. In the medical field, the small sample size is due to an imbalance in which the number of people with the disease is smaller than the number of people without the disease. This imbalance problem can be addressed by introducing a method of oversampling relatively small data [56]. In this study, we performed a 5- and 10-fold stratified cross-validation set to compensate for the small sample size [57]. Although there are differences according to the ML algorithms, no significant difference was found between the two methods for the prediction of PSD occurrence and prognosis (Table 9). A prospective study can provide more accurate clinical information; therefore, it is possible to develop better predictive models. Pertinently, some prospective studies have been designed to predict PSD, but they were not replicable in an independent stroke group [6].
In this study, we used various ML algorithms to predict PSD occurrence and prognosis for the first time and showed better performance than the statistical method. However, further studies with larger sample sizes and longer follow-up periods are needed to ascertain applications for clinical use.

5. Conclusions

We concluded that the occurrence and prognosis of PSD in stroke patients can be predicted effectively based on cognitive and functional status using ML algorithms.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/jcm11082264/s1, Table S1: Correlation analysis between featured parameters in cognitive and functional tests for control and PSD patients; Table S2: Correlation analysis between featured parameters in cognitive and functional tests for PSD patients; Table S3: Accuracy analysis using various hyper-parameters for the prediction of PSD occurrence and prognosis with 5-fold cross-validation (A), and 10-fold cross-validation (B).

Author Contributions

Conceptualization, S.Y.K. and J.K.H.; methodology, Y.H.R., S.Y.K. and J.K.H.; software, S.J.P. and H.-Y.J.; validation, S.Y.K., S.J.P. and H.-Y.J.; formal analysis, Y.H.R., H.-Y.J. and J.K.H.; investigation, Y.H.R., H.-Y.J. and J.K.H.; resources, S.Y.K., T.U.K. and S.J.L.; data curation, S.Y.K., T.U.K. and S.J.L.; writing—original draft preparation, Y.H.R., S.Y.K., H.-Y.J. and J.K.H.; writing—review and editing, H.-Y.J. and J.K.H.; visualization, Y.H.R., H.-Y.J. and J.K.H.; supervision, H.-Y.J. and J.K.H.; project administration, H.-Y.J. and J.K.H.; funding acquisition, S.Y.K. and J.K.H. All authors have read and agreed to the published version of the manuscript.

Funding

This study was supported by grants (2019R1A6A1A11034536, 2020R1A2C2004764) through the National Research Foundation (NRF), Electronics and Telecommunications Research Institute (ETRI) grant funded by the Korean government [21YR2410, Development of image and medical intelligence core technology for rehabilitation diagnosis and treatment of brain and spinal cord diseases], and Ministry of Science and ICT (MSIT) and a Korea Medical Device Development Fund grant (202017D01) funded by the Korean government (MSIT, the Ministry of Trade, Industry and Energy, the Ministry of Health & Welfare, and the Ministry of Food and Drug Safety).

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Institutional Review Board of Dankook University Hospital (IRB No. 2013-02-007), approved date 26 March 2013.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available from the corresponding authors upon reasonable request.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Burvill, P.W.; Johnson, G.A.; Jamrozik, K.D.; Anderson, C.S.; Stewart-Wynne, E.G.; Chakera, T.M. Prevalence of depression after stroke: The Perth Community Stroke Study. Br. J. Psychiatry 1995, 166, 320–327. [Google Scholar] [CrossRef] [PubMed]
  2. D’Aniello, G.E.; Scarpina, F.; Mauro, A.; Mori, I.; Castelnuovo, G.; Bigoni, M.; Baudo, S.; Molinari, E. Characteristics of anxiety and psychological well-being in chronic post-stroke patients. J. Neurol. Sci. 2014, 338, 191–196. [Google Scholar] [CrossRef] [PubMed]
  3. Berg, A.; Lonnqvist, J.; Palomaki, H.; Kaste, M. Assessment of depression after stroke: A comparison of different screening instruments. Stroke 2009, 40, 523–529. [Google Scholar] [CrossRef][Green Version]
  4. Robinson, R.G. Poststroke depression: Prevalence, diagnosis, treatment, and disease progression. Biol. Psychiatry 2003, 54, 376–387. [Google Scholar] [CrossRef]
  5. Whyte, E.M.; Mulsant, B.H. Post stroke depression: Epidemiology, pathophysiology, and biological treatment. Biol. Psychiatry 2002, 52, 253–264. [Google Scholar] [CrossRef]
  6. Ayerbe, L.; Ayis, S.; Wolfe, C.D.; Rudd, A.G. Natural history, predictors and outcomes of depression after stroke: Systematic review and meta-analysis. Br. J. Psychiatry 2013, 202, 14–21. [Google Scholar] [CrossRef]
  7. Hellmann-Regen, J.; Piber, D.; Hinkelmann, K.; Gold, S.M.; Heesen, C.; Spitzer, C.; Endres, M.; Otte, C. Depressive syndromes in neurological disorders. Eur. Arch. Psychiatry Clin. Neurosci. 2013, 263 (Suppl. S2), S123–S136. [Google Scholar] [CrossRef]
  8. Allan, L.M.; Rowan, E.N.; Thomas, A.J.; Polvikoski, T.M.; O’Brien, J.T.; Kalaria, R.N. Long-term incidence of depression and predictors of depressive symptoms in older stroke survivors. Br. J. Psychiatry 2013, 203, 453–460. [Google Scholar] [CrossRef][Green Version]
  9. Jiang, X.G.; Lin, Y.; Li, Y.S. Correlative study on risk factors of depression among acute stroke patients. Eur. Rev. Med. Pharmacol. Sci. 2014, 18, 1315–1323. [Google Scholar]
  10. Astrom, M.; Adolfsson, R.; Asplund, K. Major depression in stroke patients. A 3-year longitudinal study. Stroke 1993, 24, 976–982. [Google Scholar] [CrossRef][Green Version]
  11. Kauhanen, M.; Korpelainen, J.T.; Hiltunen, P.; Brusin, E.; Mononen, H.; Maatta, R.; Nieminen, P.; Sotaniemi, K.A.; Myllyla, V.V. Poststroke depression correlates with cognitive impairment and neurological deficits. Stroke 1999, 30, 1875–1880. [Google Scholar] [CrossRef] [PubMed]
  12. Saxena, S.K.; Ng, T.P.; Koh, G.; Yong, D.; Fong, N.P. Is improvement in impaired cognition and depressive symptoms in post-stroke patients associated with recovery in activities of daily living? Acta. Neurol. Scand. 2007, 115, 339–346. [Google Scholar] [CrossRef] [PubMed]
  13. Schottke, H.; Gerke, L.; Dusing, R.; Mollmann, A. Post-stroke depression and functional impairments—A 3-year prospective study. Compr. Psychiatry 2020, 99, 152171. [Google Scholar] [CrossRef] [PubMed]
  14. Paolucci, S.; Iosa, M.; Coiro, P.; Venturiero, V.; Savo, A.; De Angelis, D.; Morone, G. Post-stroke Depression Increases Disability More Than 15% in Ischemic Stroke Survivors: A Case-Control Study. Front. Neurol. 2019, 10, 926. [Google Scholar] [CrossRef][Green Version]
  15. Andersen, G.; Vestergaard, K.; Riis, J.O.; Ingeman-Nielsen, M. Dementia of depression or depression of dementia in stroke? Acta Psychiatr. Scand. 1996, 94, 272–278. [Google Scholar] [CrossRef]
  16. Robinson, R.G.; Schultz, S.K.; Castillo, C.; Kopel, T.; Kosier, J.T.; Newman, R.M.; Curdue, K.; Petracca, G.; Starkstein, S.E. Nortriptyline versus fluoxetine in the treatment of depression and in short-term recovery after stroke: A placebo-controlled, double-blind study. Am. J. Psychiatry 2000, 157, 351–359. [Google Scholar] [CrossRef]
  17. Heo, J.; Yoon, J.G.; Park, H.; Kim, Y.D.; Nam, H.S.; Heo, J.H. Machine Learning-Based Model for Prediction of Outcomes in Acute Stroke. Stroke 2019, 50, 1263–1265. [Google Scholar] [CrossRef]
  18. Jang, S.K.; Chang, J.Y.; Lee, J.S.; Lee, E.J.; Kim, Y.H.; Han, J.H.; Chang, D.I.; Cho, H.J.; Cha, J.K.; Yu, K.H.; et al. Reliability and Clinical Utility of Machine Learning to Predict Stroke Prognosis: Comparison with Logistic Regression. J. Stroke 2020, 22, 403–406. [Google Scholar] [CrossRef]
  19. Lin, C.H.; Hsu, K.C.; Johnson, K.R.; Fann, Y.C.; Tsai, C.H.; Sun, Y.; Lien, L.M.; Chang, W.L.; Chen, P.L.; Lin, C.L.; et al. Evaluation of machine learning methods to stroke outcome prediction using a nationwide disease registry. Comput. Methods Programs Biomed. 2020, 190, 105381. [Google Scholar] [CrossRef]
  20. Hama, S.; Yoshimura, K.; Yanagawa, A.; Shimonaga, K.; Furui, A.; Soh, Z.; Nishino, S.; Hirano, H.; Yamawaki, S.; Tsuji, T. Relationships between motor and cognitive functions and subsequent post-stroke mood disorders revealed by machine learning analysis. Sci. Rep. 2020, 10, 19571. [Google Scholar] [CrossRef]
  21. Rauwenhoff, J.C.C.; Bronswijk, S.C.; Peeters, F.; Bol, Y.; Geurts, A.C.H.; van Heugten, C.M. Personalized predictions of treatment outcome in patients with post-stroke depressive symptoms. J. Rehabil. Med. 2020, 52, jrm00120. [Google Scholar] [CrossRef] [PubMed]
  22. First, M.B.; Spitzer, R.L.; Gibbon, M.W.J.B.; Williams, J.B. Structured Clinical Interview for DSM-IV Clinical Version (SCID-I/CV); American Psychiatric Press: Washington, DC, USA, 1997. [Google Scholar]
  23. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders; American Psychiatric Association: Washington, DC, USA, 1980; Volume 3. [Google Scholar]
  24. Kunitz, S.C.; Gross, C.R.; Heyman, A.; Kase, C.S.; Mohr, J.P.; Price, T.R.; Wolf, P.A. The pilot Stroke Data Bank: Definition, design, and data. Stroke 1984, 15, 740–746. [Google Scholar] [CrossRef] [PubMed][Green Version]
  25. Turner-Stokes, L.; Hassan, N. Depression after stroke: A review of the evidence base to inform the development of an integrated care pathway. Part 2: Treatment alternatives. Clin. Rehabil. 2002, 16, 248–260. [Google Scholar] [CrossRef] [PubMed]
  26. Hamilton, M. The Hamilton Depression Scale—accelerator or break on antidepressant drug discovery. Psychiatry 1960, 23, 56–62. [Google Scholar]
  27. Folstein, M.F.; Folstein, S.E.; McHugh, P.R. “Mini-mental state”: A practical method for grading the cognitive state of patients for the clinician. J. Psychiatr. Res. 1975, 12, 189–198. [Google Scholar] [CrossRef]
  28. Kang, Y.W.; Na, D.L.; Hahn, S.H. A validity study on the korean mini-mental state examination (K-MMSE) in dementia patients. J. Korean Neurol. Ass. 1997, 15, 300–308. [Google Scholar]
  29. Thomas Gualtieri, C. Computerized neurocognitive testing and its potential for modern psychiatry. Psychiatry 2004, 1, 29–36. [Google Scholar]
  30. Han, C.; Jo, S.A.; Jo, I.; Kim, E.; Park, M.H.; Kang, Y. An adaptation of the Korean mini-mental state examination (K-MMSE) in elderly Koreans: Demographic influence and population-based norms (the AGE study). Arch. Gerontol. Geriatr. 2008, 47, 302–310. [Google Scholar] [CrossRef]
  31. Katz, N.; Erez, A.B.-H.; Livni, L.; Averbuch, S. Dynamic Lowenstein Occupational Therapy Cognitive Assessment: Evaluation of potential to change in cognitive performance. Am. J. Occup. Ther. 2012, 66, 207–214. [Google Scholar] [CrossRef][Green Version]
  32. Keith, R.A.; Granger, C.V.; Hamilton, B.B.; Sherman, F.S. Advances in Clinical Rehabilitation; Springer: New York, NY, USA, 1987; Volume 2, pp. 6–18. [Google Scholar]
  33. Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; et al. Scikit-learn: Machine Learning in Python. J. Mach. Learn. Res. 2011, 12, 2825–2830. [Google Scholar]
  34. Mitchell, N.; Hewitt, C.; Adamson, J.; Parrott, S.; Torgerson, D.; Ekers, D.; Holmes, J.; Lester, H.; McMillan, D.; Richards, D.; et al. A randomised evaluation of CollAborative care and active surveillance for Screen-Positive EldeRs with sub-threshold depression (CASPER): Study protocol for a randomized controlled trial. Trials 2011, 12, 225. [Google Scholar] [CrossRef] [PubMed][Green Version]
  35. Barker-Collo, S.L. Depression and anxiety 3 months post stroke: Prevalence and correlates. Arch. Clin. Neuropsychol. 2007, 22, 519–531. [Google Scholar] [CrossRef] [PubMed][Green Version]
  36. Semkovska, M.; Quinlivan, L.; O’Grady, T.; Johnson, R.; Collins, A.; O’Connor, J.; Knittle, H.; Ahern, E.; Gload, T. Cognitive function following a major depressive episode: A systematic review and meta-analysis. Lancet Psychiatry 2019, 6, 851–861. [Google Scholar] [CrossRef][Green Version]
  37. Villa, R.F.; Ferrari, F.; Moretti, A. Post-stroke depression: Mechanisms and pharmacological treatment. Pharmacol. Ther. 2018, 184, 131–144. [Google Scholar] [CrossRef]
  38. Woranush, W.; Moskopp, M.L.; Sedghi, A.; Stuckart, I.; Noll, T.; Barlinn, K.; Siepmann, T. Preventive Approaches for Post-Stroke Depression: Where Do We Stand? A Systematic Review. Neuropsychiatr. Dis. Treat. 2021, 17, 3359–3377. [Google Scholar] [CrossRef]
  39. Allida, S.; Cox, K.L.; Hsieh, C.F.; House, A.; Hackett, M.L. Pharmacological, psychological and non-invasive brain stimulation interventions for preventing depression after stroke. Cochrane Database Sys. Rev. 2020, 5, CD003689. [Google Scholar] [CrossRef]
  40. Shi, Y.; Yang, D.; Zeng, Y.; Wu, W. Risk Factors for Post-stroke Depression: A Meta-analysis. Front. Aging Neurosci. 2017, 9, 218. [Google Scholar] [CrossRef]
  41. Robinson, R.G.; Jorge, R.E. Post-Stroke Depression: A Review. Am. J. Psychiatry 2016, 173, 221–231. [Google Scholar] [CrossRef][Green Version]
  42. Schooling, C.M.; Jones, H.E. Clarifying questions about “risk factors”: Predictors versus explanation. Emerg. Themes Epidemiol. 2018, 15, 10. [Google Scholar] [CrossRef]
  43. Bzdok, D.; Altman, N.; Krzywinski, M. Statistics versus machine learning. Nat. Methods 2018, 15, 233–234. [Google Scholar] [CrossRef]
  44. Sirsat, M.S.; Ferme, E.; Camara, J. Machine Learning for Brain Stroke: A Review. J. Stroke Cerebrovasc. Dis. 2020, 29, 105162. [Google Scholar] [CrossRef] [PubMed]
  45. Que, Q.; Belkin, M. Back to the Future: Radial Basis Function Network Revisited. IEEE Trans. Pattern Anal. Mach. Intell. 2020, 42, 1856–1867. [Google Scholar] [CrossRef] [PubMed][Green Version]
  46. Kimura, M.; Robinson, R.G.; Kosier, J.T. Treatment of cognitive impairment after poststroke depression: A double-blind treatment trial. Stroke 2000, 31, 1482–1486. [Google Scholar] [CrossRef] [PubMed]
  47. Lincoln, N.B.; Nicholl, C.R.; Flannaghan, T.; Leonard, M.; Van der Gucht, E. The validity of questionnaire measures for assessing depression after stroke. Clin. Rehabil. 2003, 17, 840–846. [Google Scholar] [CrossRef] [PubMed]
  48. Jorge, R.E.; Acion, L.; Moser, D.; Adams, H.P., Jr.; Robinson, R.G. Escitalopram and enhancement of cognitive recovery following stroke. Arch. Gen. Psychiatry 2010, 67, 187–196. [Google Scholar] [CrossRef] [PubMed][Green Version]
  49. Bjelland, I.; Krokstad, S.; Mykletun, A.; Dahl, A.A.; Tell, G.S.; Tambs, K. Does a higher educational level protect against anxiety and depression? The HUNT study. Soc. Sci. Med. 2008, 66, 1334–1345. [Google Scholar] [CrossRef]
  50. Baccaro, A.; Wang, Y.P.; Candido, M.; Conforto, A.B.; Brunoni, A.R.; Leite, C.D.C.; Busatto Filho, G.; Lotufo, P.A.; Bensenor, I.M.; Goulart, A.C. Post-stroke depression and cognitive impairment: Study design and preliminary findings in a Brazilian prospective stroke cohort (EMMA study). J. Affect. Disord. 2019, 245, 72–81. [Google Scholar] [CrossRef]
  51. Wang, Z.; Zhu, M.; Su, Z.; Guan, B.; Wang, A.; Wang, Y.; Zhang, N.; Wang, C. Post-stroke depression: Different characteristics based on follow-up stage and gender-a cohort perspective study from Mainland China. Neurol. Res. 2017, 39, 996–1005. [Google Scholar] [CrossRef]
  52. Whyte, E.M.; Mulsant, B.H.; Vanderbilt, J.; Dodge, H.H.; Ganguli, M. Depression after stroke: A prospective epidemiological study. J. Am. Geriatr. Soc. 2004, 52, 774–778. [Google Scholar] [CrossRef]
  53. Ogundimu, E.O.; Altman, D.G.; Collins, G.S. Adequate sample size for developing prediction models is not simply related to events per variable. J. Clin. Epidemiol. 2016, 76, 175–182. [Google Scholar] [CrossRef][Green Version]
  54. Fruehwald, S.; Gatterbauer, E.; Rehak, P.; Baumhackl, U. Early fluoxetine treatment of post-stroke depression--a three-month double-blind placebo-controlled study with an open-label long-term follow up. J. Neurol. 2003, 250, 347–351. [Google Scholar] [CrossRef] [PubMed]
  55. Wiart, L.; Petit, H.; Joseph, P.A.; Mazaux, J.M.; Barat, M. Fluoxetine in early poststroke depression: A double-blind placebo-controlled study. Stroke 2000, 31, 1829–1832. [Google Scholar] [CrossRef] [PubMed]
  56. Chawla, N.V.; Bowyer, K.W.; Hall, L.O.; Kegelmeyer, W.P. SMOTE: Synthetic minority over-sampling technique. J. Artif. Intell. Res. 2002, 16, 321–357. [Google Scholar] [CrossRef]
  57. Kohavi, R. A study of cross-validation and bootstrap for accuracy estimation and model selection. In Proceedings of the International Joint Conference on Artificial Intelligence, Montreal, QC, Canada, 20–25 August 1995; pp. 1137–1145. [Google Scholar]
Figure 1. Flow and grouping of patients and controls. Abbreviation: PSD = poststroke depression.
Figure 1. Flow and grouping of patients and controls. Abbreviation: PSD = poststroke depression.
Jcm 11 02264 g001
Figure 2. Flow of ML analysis. Abbreviations: PSD = poststroke depression; Imp = patients showing improvement in PSD symptoms; NoImp = patients showing no improvement in PSD symptoms; SVM_L = linear support vector machine; SVM_R = support vector machine with radial basis function kernel; KNN = k-nearest neighbors; RF = random forest; VE = voting ensemble; AUC = area under the curve; ROC = receiver operating characteristic.
Figure 2. Flow of ML analysis. Abbreviations: PSD = poststroke depression; Imp = patients showing improvement in PSD symptoms; NoImp = patients showing no improvement in PSD symptoms; SVM_L = linear support vector machine; SVM_R = support vector machine with radial basis function kernel; KNN = k-nearest neighbors; RF = random forest; VE = voting ensemble; AUC = area under the curve; ROC = receiver operating characteristic.
Jcm 11 02264 g002
Figure 3. Receiver operating characteristic (ROC) curve for support vector machine with radial basis function kernel function algorithm with 10-fold cross-validation in the prediction of poststroke depression (PSD) occurrence (control vs. PSD, (A)), support vector machine linear algorithm with five-fold cross-validation in the prediction of PSD prognosis (improved vs. not improved, (B)), and ROC curve for logistic regression analysis in the prediction of PSD occurrence (C) and PSD prognosis (D).
Figure 3. Receiver operating characteristic (ROC) curve for support vector machine with radial basis function kernel function algorithm with 10-fold cross-validation in the prediction of poststroke depression (PSD) occurrence (control vs. PSD, (A)), support vector machine linear algorithm with five-fold cross-validation in the prediction of PSD prognosis (improved vs. not improved, (B)), and ROC curve for logistic regression analysis in the prediction of PSD occurrence (C) and PSD prognosis (D).
Jcm 11 02264 g003
Figure 4. Decision tree models to classify controls and patients with PSD. Abbreviations: PSD = poststroke depression; CNT1_AA OE = Initial subscore of auditory attention omission error on the CNT; CNT1_VA OE = Initial subscore of visual attention omission error on the CNT; MBI1_Tra = Initial subscore of transfer on the K-MBI; MBI1_Dre = Initial subscore of dressing on the K-MBI. Black arrows = true, red arrows = false.
Figure 4. Decision tree models to classify controls and patients with PSD. Abbreviations: PSD = poststroke depression; CNT1_AA OE = Initial subscore of auditory attention omission error on the CNT; CNT1_VA OE = Initial subscore of visual attention omission error on the CNT; MBI1_Tra = Initial subscore of transfer on the K-MBI; MBI1_Dre = Initial subscore of dressing on the K-MBI. Black arrows = true, red arrows = false.
Jcm 11 02264 g004
Table 1. Baseline characteristics of controls and patients with PSD.
Table 1. Baseline characteristics of controls and patients with PSD.
ControlsPSD Patientsp Value 2
NoImpImpp Value 1All
No. of subjects3418130.79831
Age (years)64.6 ± 15.962.3 ± 11.563.5 ± 13.90.77962.8 ± 15.90.563
Female, no. (%)15 (44)7 (39)8 (61)0.28521 (58)0.546
Educational period (years)9.2 ± 5.07.0 ± 3.49.8 ± 4.00.027 *8.2 ± 3.90.406
Onset (days)74.3 ± 68.971.2 ± 41.369.5 ± 25.40.89070.5 ± 35.00.250
No. of depressive symptoms
 Initial-6.06 ± 1.396.69 ± 1.600.2606.32 ± 1.49-
 Follow-up-6.67 ± 1.532.08 ± 2.250.000 *4.74 ± 2.94-
Type of stroke, no. (%)
 Hemorrhagic stroke15 (44)3 (23)9 (50)0.83312 (39)0.661
 Ischemic stroke19 (56)10 (77)9 (50)0.83319 (61)0.661
Side of hemiplegia
(Rt:Lt:both)
14 (42):12 (35):8 (23)6 (33):7 (39):5 (28)3 (23):8 (61):2 (16)0.4629 (29):15 (48):7 (23)0.518
Family Hx. (medical disorders)12 (35.3)6 (33)6 (46)0.47012 (39)0.777
Family Hx. (mental disorders)0 (0)1 (5)0 (0)0.3881 (3)0.295
Smoking (years)8.2 ± 15.86.1 ± 14.49.6 ± 17.10.5677.5 ± 15.40.880
Diabetes mellitus8 (23)4 (22)6 (46)0.16010 (30)0.436
Hypertension10 (56)10 (55)6 (46)0.60516 (52)0.288
NIHSS score7.6 ± 5.67.3 ± 7.08.0 ± 4.30.5356.8 ± 3.40.826
Note: Values are presented as the number of subjects (%) or mean ± standard deviation. 1 p values between participants in the NoImp and Imp groups determined by the Mann–Whitney U test or the likelihood ratio, 2 p values between participants in the control and PSD groups determined by the Mann–Whitney U test or the likelihood ratio, * p < 0.05. Abbreviations: PSD = poststroke depression; NoImp = PSD patients with no symptom improvement; Imp = PSD patients with improvement in their symptoms; NIHSS = National Institutes of Health Stroke Scale.
Table 2. Initial and follow-up results of all cognitive and functional tests for controls and PSD patients.
Table 2. Initial and follow-up results of all cognitive and functional tests for controls and PSD patients.
ControlsPSD Patientsp Value 2
NoImpImpp Value 1All
K-MMSE
 Initial14.0 ± 8.412.4 ± 7.616.3 ± 7.50.20614.1 ± 7.70.787
 Follow-up19.3 ± 9.316.1 ± 8.221.6 ± 6.80.037 *18.7 ± 8.00.491
 Gain5.4 ± 6.84.3 ± 3.25.4 ± 3.10.3954.7 ± 3.10.535
p value 30.000 *0.001 *0.000 * 0.000 *
CNT
 Initial419.1 ± 180.2371.5 ± 114.4415.8 ± 145.10.603390.1 ± 127.80.427
 Follow-up449.5 ± 220.7422.9 ± 191.7492.3 ± 230.00.526449.9 ± 203.70.980
 Gain33.6 ± 70.949.1 ± 64.536.0 ± 57.70.61844.0 ± 60.50.580
p value 30.1190.2410.237 0.088
K-MBI
 Initial25.7 ± 25.219.8 ± 15.525.7 ± 25.20.71819.8 ± 15.50.103
 Follow-up46.5 ± 28.640.1 ± 19.746.5 ± 28.60.33040.1 ± 19.70.006 *
 Gain18.7 ± 12.720.2 ± 15.518.7 ± 12.70.70320.2 ± 15.50.276
p value 30.000 *0.001 *0.002 * 0.000 *
FIM
 Initial46.3 ± 22.944.1± 15.546.3 ± 22.90.90444.1 ± 15.50.285
 Follow-up64.7 ± 28.259.1 ± 17.564.7 ± 28.20.52559.1 ± 17.50.013 *
 Gain17.0 ± 11.916.5 ± 8.217.0 ± 11.90.92516.5 ± 8.20.132
p value 30.000 *0.000 *0.005 * 0.000 *
Note: Values are presented as the mean ± standard deviation. 1 p values between participants in the NoImp and Imp groups determined by the Mann–Whitney U test, 2 p values between participants in the control and PSD groups determined by the Mann–Whitney U test, 3 p values between initial and follow-up values determined by the Wilcoxon sum rank test, * p < 0.05. Abbreviations: PSD = poststroke depression; NoImp = PSD patients showing no symptom improvement; Imp = PSD patients showing improvement in their symptoms; K-MMSE = Korean version of the Mini-Mental Status Examination; CNT = computerized neurocognitive test; K-MBI = Korean version of the modified Barthel index; FIM = functional independence measure.
Table 3. Initial and follow-up results of K-MMSE categories for controls and PSD patients.
Table 3. Initial and follow-up results of K-MMSE categories for controls and PSD patients.
ControlsPSD Patientsp Value 2
NoImpImpp Value 1All
Orientation
 Initial5.1 ± 3.14.2 ± 3.55.5 ± 3.40.2244.7 ± 3.50.630
 Follow-up6.8 ± 3.45.3 ± 3.37.5 ± 2.60.1436.2 ± 3.20.355
 Gain1.7 ± 2.81.4 ± 2.11.4 ± 2.20.6881.4 ± 2.10.916
p value 30.003 *0.020 *0.077 0.003 *
Registration
 Initial2.3 ± 1.12.3 ± 1.22.9 ± 0.30.1382.6 ± 1.00.254
 Follow-up2.5 ± 1.22.6 ± 1.43.0 ± 0.00.7012.8 ± 1.10.200
 Gain0.2 ± 0.90.4 ± 1.00.1 ± 0.30.5440.3 ± 0.80.983
p value 30.1960.1310.343 0.084
Recall
 Initial0.9 ± 1.80.5 ± 0.61.2 ± 1.50.3640.8 ± 1.10.446
 Follow-up1.9 ± 2.10.9 ± 1.02.8 ± 2.20.027 *1.6 ± 1.80.894
 Gain0.9 ± 1.50.4 ± 0.71.6 ± 1.60.026 *0.9 ± 1.20.601
p value 30.003 *0.038 *0.011 * 0.002 *
Attention and calculation
 Initial1.1 ± 1.30.5 ± 0.81.2 ± 1.20.0630.8 ± 1.00.495
 Follow-up1.7 ± 1.31.4 ± 1.42.0 ± 0.90.3741.7 ± 1.20.946
 Gain0.5 ± 1.30.9 ± 1.00.7 ± 1.70.6950.8 ± 1.30.439
p value 30.013 *0.010 *0.226 0.007 *
Language and complex commands
 Initial4.6 ± 2.74.9 ± 2.75.5 ± 2.60.7625.2 ± 2.60.341
 Follow-up6.3 ± 3.05.8 ± 2.87.4 ± 1.50.0806.4 ± 2.50.696
 Gain1.4 ± 2.21.1 ± 1.11.6 ± 1.80.5711.3 ± 1.40.659
p value 30.000 *0.003 *0.022 * 0.000 *
Note: Values are presented as the mean ± standard deviation. 1 p values between participants in the NoImp and Imp groups determined by the Mann–Whitney U test, 2 p values between participants in the control and PSD groups determined by the Mann–Whitney U test, 3 p values between initial and follow-up values determined by the Wilcoxon sum rank test, * p < 0.05. Abbreviations: PSD = poststroke depression; NoImp = PSD patients showing no symptom improvement; Imp = PSD patients showing improvement in their symptoms.
Table 4. Initial and follow-up results of subtests of the CNT for controls and PSD patients.
Table 4. Initial and follow-up results of subtests of the CNT for controls and PSD patients.
ControlsPSD Patientsp Value 2
NoImpImpp Value 1All
Language memory
digit span forward
 Initial30.4 ± 5.129.2 ± 4.232.2 ± 7.30.09030.9 ± 5.90.869
 Follow-up31.5 ± 6.929.0 ± 3.235.1 ± 5.20.017 *31.5 ± 6.90.592
 Gain2.3 ± 3.30.5 ± 2.21.4 ± 5.20.7280.8 ± 3.30.105
p value 30.005 *0.4651.00 0.483
Language memory
digit span backward
 Initial29.9 ± 5.628.3 ± 2.530.4 ± 7.40.68929.2 ± 5.10.934
 Follow-up33.0 ± 11.229.0 ± 4.835.6 ± 8.20.013 *32.0 ± 7.81.000
 Gain3.9 ± 8.41.1 ± 2.87.0 ± 8.30.0903.4 ± 6.10.847
p value 30.005 *0.1800.046 * 0.017 *
Visual memory
visual span forward
 Initial30.7 ± 7.528.9 ± 3.033.5 ± 9.30.20130.8 ± 6.70.820
 Follow-up32.0 ± 7.829.6 ± 3.432.6 ± 7.60.49630.8 ± 5.40.684
 Gain2.7 ± 6.31.1 ± 2.9−0.9 ± 5.60.6190.3 ± 6.30.373
p value 30.016 *0.2251.00 0.440
Visual memory
visual span backward
 Initial30.6 ± 6.128.8 ± 3.032.9 ± 8.00.21530.5 ± 5.90.650
 Follow-up32.0 ± 5.929.8 ± 4.3 34.7 ± 10.60.23031.7 ± 7.50.593
 Gain2.5 ± 4.60.6 ± 4.81.4 ± 5.70.8140.9 ± 5.00.424
p value 30.012 *0.4390.465 0.323
Auditory attention
correct response
 Initial29.8 ± 6.627.8 ± 3.229.0 ± 3.20.39928.3 ± 4.10.320
 Follow-up31.9 ± 11.231.9 ± 7.036.1 ± 18.31.00033.3 ± 12.30.361
 Gain3.7 ± 7.73.6 ± 5.06.7 ± 12.10.8264.9 ± 8.40.465
p value 30.027 *0.0680.102 0.027 *
Auditory attention
commission error
 Initial29.8 ± 6.627.8 ± 3.229.0 ± 3.20.39928.3 ± 4.10.320
 Follow-up31.9 ± 11.231.9 ± 7.036.1 ± 18.31.00033.6 ± 12.60.361
 Gain3.7 ± 7.73.6 ± 5.06.7 ± 12.10.8264.9 ± 8.40.465
p value 30.027 *0.0680.102 0.027 *
Auditory attention
omission error
 Initial38.1 ± 17.234.5 ± 14.833.6 ± 13.70.72328.7 ± 5.70.481
 Follow-up36.0 ± 15.141.3 ± 21.336.3 ± 18.50.71131.2 ± 9.40.892
 Gain−5.9 ± 22.55.5 ± 10.55.7 ± 16.00.5851.4 ± 10.70.052
p value 30.2210.1090.414 0.075
Auditory attention
correct time SD
 Initial38.1 ± 17.234.5 ± 14.833.6 ± 13.70.77334.1 ± 14.10.232
 Follow-up36.1 ± 15.141.3 ± 21.336.3 ± 18.50.74039.2 ± 19.80.924
 Gain−7.5 ± 21.76.1 ± 10.95.7 ± 16.00.5215.9 ± 12.90.026 *
p value 30.026 *0.0680.414 0.085
Visual attention
correct response
 Initial33.2 ± 12.730.1 ± 9.935.4 ± 11.90.30831.4 ± 11.10.617
 Follow-up38.7 ± 14.537.7 ± 19.234.1 ± 16.00.66936.2 ± 17.50.202
 Gain9.0 ± 13.211.6 ± 19.2−2.4 ± 5.80.041 *5.4 ± 16.20.174
p value 30.006 *0.0780.197 0.026 *
Visual attention
commission error
 Initial33.4 ± 13.130.1 ± 10.732.4 ± 12.00.38131.1 ± 11.10.223
 Follow-up38.7 ± 14.537.7 ± 19.234.1 ± 16.00.60636.2 ± 17.50.072
 Gain9.3 ± 13.910.1 ± 18.7−1.3 ± 5.30.1385.4 ± 15.50.049 *
p value 30.006 *0.0780.414 0.288
Visual attention
omission error
 Initial38.2 ± 18.230.1 ± 12.530.9 ± 11.90.48130.5 ± 12.00.027 *
 Follow-up40.4 ± 20.531.0 ± 6.833.1 ± 16.30.39631.9 ± 11.20.303
 Gain1.0 ± 20.74.0 ± 6.8−1.1 ± 22.60.2531.9 ± 15.00.677
p value 30.6710.0660.655 0.344
Visual attention
correct time SD
 Initial29.6 ± 5.829.3 ± 4.130.6 ± 6.40.65129.9 ± 5.20.640
 Follow-up30.3 ± 4.629.5 ± 3.131.6 ± 6.30.62230.5 ± 4.80.885
 Gain3.8 ± 7.70.4 ± 6.31.3 ± 2.90.7630.8 ± 4.90.438
p value 30.010 *0.4680.257 0.205
Note: Values are presented as the mean ± standard deviation. 1 p values between participants in the NoImp and Imp groups determined by the Mann–Whitney U test, 2 p values between participants in the control and PSD groups determined by the Mann–Whitney U test, 3 p values between initial and follow-up values determined by the Wilcoxon sum rank test, * p < 0.05. Abbreviations: PSD = poststroke depression; NoImp = PSD patients showing no symptom improvement; Imp = PSD patients showing improvement in their symptoms; SD = standard deviation.
Table 5. Initial and follow-up results of subtests of the K-MBI for controls and PSD patients.
Table 5. Initial and follow-up results of subtests of the K-MBI for controls and PSD patients.
ControlsPSD Patientsp Value 2
NoImpImpp Value 1All
Hygiene
 Initial2.1 ± 1.81.8 ± 1.51.6 ± 2.00.5071.7 ± 1.70.369
 Follow-up3.3 ± 1.62.6 ± 1.53.2 ± 1.30.4142.9 ± 1.40.166
 Gain1.2 ± 1.70.9 ± 1.51.6 ± 1.70.2771.2 ± 1.60.628
p value 30.001 *0.027 *0.017 * 0.001 *
Bathing
 Initial1.1 ± 1.20.6 ± 0.50.3 ± 0.50.1010.5 ± 0.50.075
 Follow-up2.4 ± 1.61.4 ± 1.21.7 ± 1.50.5671.5 ± 1.30.030 *
 Gain1.4 ± 1.60.7 ± 1.01.4 ± 1.40.1641.0 ± 1.20.324
p value 30.000 *0.015 *0.016 * 0.001 *
Eating
 Initial3.7 ± 3.53.0 ± 3.03.3 ± 3.90.9503.1 ± 3.30.543
 Follow-up6.6 ± 2.95.2 ± 3.15.7 ± 3.40.6625.4 ± 3.20.146
 Gain2.9 ± 3.42.1 ± 2.62.5 ± 3.00.6582.3 ± 2.70.531
p value 30.000 *0.011 *0.024 * 0.001 *
Toileting
 Initial2.9 ± 3.51.2 ± 2.22.3 ± 3.50.5261.6 ± 2.80.097
 Follow-up5.5 ± 3.73.4 ± 2.84.3 ± 3.80.6793.8 ± 3.20.047 *
 Gain2.8 ± 3.92.6 ± 2.82.0 ± 3.10.4552.4 ± 2.90.773
p value 30.001 *0.006 *0.058 0.001 *
Stair-climbing
 Initial0.3 ± 1.40.0 ± 0.00.4 ± 1.40.2390.2 ± 0.90.613
 Follow-up3.2 ± 3.70.3 ± 1.21.5 ± 3.20.2730.8 ± 2.30.002 *
 Gain2.9 ± 3.60.3 ± 1.21.0 ± 2.50.3140.6 ± 1.80.003 *
p value 30.001 *0.3170.180 0.109
Dressing
 Initial3.4 ± 2.72.2 ± 1.71.6 ± 2.10.2302.0 ± 1.90.030 *
 Follow-up6.1 ± 3.23.9 ± 2.54.9 ± 3.10.4534.3 ± 2.80.029 *
 Gain 2.8 ± 3.21.9 ± 2.13.0 ± 3.30.3882.3 ± 2.60.694
p value 30.000 *0.008 *0.028 * 0.001 *
Bowel control
 Initial5.3 ± 4.44.6 ± 4.54.8 ± 5.10.8314.7 ± 4.70.448
 Follow-up8.4 ± 3.37.4 ± 3.76.7 ± 4.20.7347.1 ± 3.90.190
 Gain3.2 ± 4.23.1 ± 4.12.0 ± 3.20.5582.6 ± 3.80.921
p value 30.001 *0.013 *0.039 * 0.002 *
Bladder control
 Initial4.8 ± 4.53.1 ± 4.14.9 ± 4.90.1693.9 ± 4.50.291
 Follow-up8.2 ± 3.45.8 ± 4.36.5 ± 4.50.5486.1 ± 4.30.047 *
 Gain3.4 ± 4.32.5 ± 4.21.6 ± 2.80.7242.1 ± 3.60.314
p value 30.000 *0.031 *0.066 0.007 *
Transfer
 Initial6.2 ± 5.03.0 ± 3.75.4 ± 5.90.4084.0 ± 4.80.050
 Follow-up10.1 ± 4.46.9 ± 4.07.5 ± 5.70.6977.1 ± 4.60.013 *
 Gain4.1 ± 4.54.5 ± 4.12.2 ± 3.50.1373.6 ± 4.00.630
p value 30.000 *0.003 *0.058 0.001 *
Ambulation
 Initial2.4 ± 4.10.3 ± 0.51.1 ± 2.40.6690.6 ± 1.60.058
 Follow-up6.8 ± 5.72.6 ± 3.53.4 ± 4.90.7702.9 ± 4.00.009 *
 Gain4.5 ± 5.02.4 ± 3.42.1 ± 3.20.7492.3 ± 3.30.129
p value 30.000 *0.006 *0.042 * 0.001 *
Note: Values are presented as the mean ± standard deviation. 1 p values between participants in the NoImp and Imp groups determined by the Mann–Whitney U test, 2 p values between participants in the control and PSD groups determined by the Mann–Whitney U test, 3 p values between initial and follow-up values determined by the Wilcoxon sum rank test, * p < 0.05. Abbreviations: PSD = poststroke depression; NoImp = PSD patients showing no symptom improvement; Imp = PSD patients showing improvement in their symptoms.
Table 6. Initial and follow-up results of the FIM subtests for controls and PSD patients.
Table 6. Initial and follow-up results of the FIM subtests for controls and PSD patients.
ControlsPSD Patientsp Value 2
NoImpImpp Value 1All
Self-care
 Initial15.4 ± 8.012.4 ± 5.011.6 ± 6.00.46812.1 ± 5.40.102
 Follow-up23.2 ± 10.217.0 ± 6.219.0 ± 8.80.58817.8 ± 7.30.033 *
 Gain7.8 ± 8.84.9 ± 3.96.5 ± 6.00.7595.5 ± 4.80.783
p value 30.000 *0.000 *0.005 * 0.000 *
Sphincter control
 Initial7.8 ± 5.16.1 ± 4.27.6 ± 5.70.6066.7 ± 4.90.350
 Follow-up11.4 ± 4.29.9 ± 4.29.6 ± 5.30.9219.8 ± 4.60.200
 Gain3.6 ± 4.44.0 ± 4.02.0 ± 2.90.0903.2 ± 3.70.847
p value 30.000 *0.001 *0.017 * 0.000 *
Transfer
 Initial8.3 ± 5.25.2 ± 3.17.2 ± 4.50.2966.0 ± 3.80.052
 Follow-up12.2 ± 5.57.9 ± 2.58.9 ± 5.40.7578.3 ± 3.90.004 *
 Gain3.9 ± 4.43.3 ± 2.21.7 ± 2.80.1102.7 ± 2.50.530
p value 30.000 *0.001 *0.074 0.000 *
Locomotion
 Initial3.2 ± 2.52.3 ± 0.82.8 ± 1.50.4972.5 ± 1.10.231
 Follow-up5.9 ± 3.74.1 ± 2.84.6 ± 4.20.5414.3 ± 3.30.032 *
 Gain2.7 ± 3.21.8 ± 2.81.7 ± 2.90.3151.8 ± 2.80.159
p value 30.000 *0.012 *0.066 0.002 *
Communication
 Initial7.9 ± 3.38.1 ± 3.57.7 ± 3.90.7777.9 ± 3.60.925
 Follow-up10.2 ± 3.48.8 ± 3.09.6 ± 3.40.6689.1 ± 3.10.169
 Gain2.3 ± 2.31.0 ± 1.81.9 ± 2.00.1941.4 ± 1.9,138
p value 30.000 *0.034 *0.016 * 0.001 *
Social
 Initial10.2 ± 5.110.1 ± 4.79.2 ± 5.30.6279.7 ± 4.90.808
 Follow-up13.7 ± 5.411.5 ± 4.813.0 ± 5.50.65412.1 ± 5.10.177
 Gain3.5 ± 4.11.6 ± 2.73.3 ± 2.80.1522.3 ± 2.80.370
p value 30.000 *0.000 *0.000 * 0.000 *
Note: Values are presented as the mean ± standard deviation. 1 p values between participants in the NoImp and Imp groups determined by the Mann–Whitney U test, 2 p values between participants in the control and PSD groups determined by the Mann–Whitney U test, 3 p values between initial and follow-up values determined by the Wilcoxon sum rank test, * p < 0.05. Abbreviations: PSD = poststroke depression; NoImp = PSD patients showing no symptom improvement; Imp = PSD patients showing improvement in their symptoms.
Table 7. Parameters of the logistic regression and their assessment.
Table 7. Parameters of the logistic regression and their assessment.
ClassificationF-Valuep-ValueEffect SizeAUCAccuracySensitivitySpecificity
Control vs. PSD12.9750.002CNT1_AA OE0.9330.7060.6960.9670.419
MBI1_Bat0.426
Imp vs. NoImp13.2960.001Edu_Per1.6160.7970.7780.6920.833
Abbreviations: PSD = poststroke depression; Imp = PSD patients showing improvement in their symptoms; NoImp = PSD patients showing no symptom improvement; AUC = area under the curve; CNT1_AA OE = the initial subscore of auditory attention omission error on the CNT; MB1_Bat = the initial subscore of bathing on the K-MBI; Edu_Per = educational period.
Table 8. Lists of feature selection for the prediction of PSD (control vs. PSD, A) and the prognosis of PSD (Imp vs. NoImp, B).
Table 8. Lists of feature selection for the prediction of PSD (control vs. PSD, A) and the prognosis of PSD (Imp vs. NoImp, B).
RankParameter Name
(A) Control vs. PSD groups
1Initial subscore of ambulation on the K-MBI (MBI1_Amb)
2Initial subscore of auditory attention omission error on the CNT (CNT1_AA OE)
3Initial subscore of recall on the K-MMSE (MMSE1_Rec)
4Initial subscore of dressing on the K-MBI (MBI1_Dre)
5Initial subscore of locomotion on the FIM (FIM1_Loc)
(B) Imp vs. NoImp groups
1Initial subscore of bladder control on the K-MBI (MBI1_Bla)
2Initial subscore of bowel control on the K-MBI (MBI1_Bow)
3Initial subscore of transfer on the FIM (FIM1_Tra)
4Initial subscore of visual memory visual span backward on the CNT (CNT1_VM VSB)
5Initial subscore of communication on the FIM (FIM1_Com)
Abbreviations: PSD = poststroke depression; Imp = PSD patients showing improvement in their symptoms; NoImp = PSD patients showing no symptom improvement.
Table 9. AUC, accuracy, sensitivity, and specificity of each ML algorithm.
Table 9. AUC, accuracy, sensitivity, and specificity of each ML algorithm.
ML Models5-Fold Cross-Validation10-Fold Cross-Validation
AUCAccuracySensitivitySpecificityAUCAccuracySensitivitySpecificity
Control vs. PSD
SVM_L0.6900.6000.7480.5570.7060.6360.7750.542
SVM_R0.7080.6460.6810.4950.7110.7000.7420.517
KNN0.6590.5380.7430.3520.6810.5790.7420.425
RF0.6850.5380.6190.5570.6960.5600.7670.600
VE0.6750.6150.6760.5520.6460.6500.7080.517
Imp vs. NoImp
SVM_L0.8300.7710.6000.8830.7970.7750.6500.950
SVM_R0.4960.6480.2670.8170.7220.7080.3000.800
KNN0.6350.6810.3000.9500.6740.7420.3000.850
RF0.7600.7430.4670.8670.6240.7170.5000.950
VE0.7840.7430.5330.8670.7470.7330.4500.90
Abbreviations: PSD = poststroke depression; Imp = PSD patients showing improvement in their symptoms; NoImp = PSD patients showing no symptom improvement; SVM_L = linear support vector machine; SVM_R = support vector machine with radial basis function kernel function; KNN = k-nearest neighbors; RF = random forest; VE = voting ensemble; AUC = area under the curve.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Ryu, Y.H.; Kim, S.Y.; Kim, T.U.; Lee, S.J.; Park, S.J.; Jung, H.-Y.; Hyun, J.K. Prediction of Poststroke Depression Based on the Outcomes of Machine Learning Algorithms. J. Clin. Med. 2022, 11, 2264. https://doi.org/10.3390/jcm11082264

AMA Style

Ryu YH, Kim SY, Kim TU, Lee SJ, Park SJ, Jung H-Y, Hyun JK. Prediction of Poststroke Depression Based on the Outcomes of Machine Learning Algorithms. Journal of Clinical Medicine. 2022; 11(8):2264. https://doi.org/10.3390/jcm11082264

Chicago/Turabian Style

Ryu, Yeong Hwan, Seo Young Kim, Tae Uk Kim, Seong Jae Lee, Soo Jun Park, Ho-Youl Jung, and Jung Keun Hyun. 2022. "Prediction of Poststroke Depression Based on the Outcomes of Machine Learning Algorithms" Journal of Clinical Medicine 11, no. 8: 2264. https://doi.org/10.3390/jcm11082264

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop