Utilization of Decision Tree Algorithms for Supporting the Prediction of Intensive Care Unit Admission of Myasthenia Gravis: A Machine Learning-Based Approach

Myasthenia gravis (MG), an acquired autoimmune-related neuromuscular disorder that causes muscle weakness, presents with varying severity, including myasthenic crisis (MC). Although MC can cause significant morbidity and mortality, specialized neuro-intensive care can produce a good long-term prognosis. Considering the outcomes of MG during hospitalization, it is critical to conduct risk assessments to predict the need for intensive care. Evidence and valid tools for the screening of critical patients with MG are lacking. We used three machine learning-based decision tree algorithms, including a classification and regression tree, C4.5, and C5.0, for predicting intensive care unit (ICU) admission of patients with MG. We included 228 MG patients admitted between 2015 and 2018. Among them, 88.2% were anti-acetylcholine receptors antibody positive and 4.7% were anti-muscle-specific kinase antibody positive. Twenty clinical variables were used as predictive variables. The C5.0 decision tree outperformed the other two decision tree and logistic regression models. The decision rules constructed by the best C5.0 model showed that the Myasthenia Gravis Foundation of America clinical classification at admission, thymoma history, azathioprine treatment history, disease duration, sex, and onset age were significant risk factors for the development of decision rules for ICU admission prediction. The developed machine learning-based decision tree can be a supportive tool for alerting clinicians regarding patients with MG who require intensive care, thereby improving the quality of care.


Introduction
Myasthenia gravis (MG) is an acquired autoimmune neuromuscular disorder presenting as muscle weakness in the eye, bulbar, limbs, and respiratory muscles that worsens with repeated muscle motion [1,2]. The pathogenesis of MG is caused by antibodies against post-synaptic proteins in the neuromuscular junction, the most common being

Participant and Study Design
The data of 513 hospital admissions of patients with MG who were admitted to the Shin-Kong Wu Ho-Su Memorial Hospital in Taipei, Taiwan, between December 2015 and October 2018, were retrospectively analyzed. The inclusion criteria for patients with MG were (1) admission due to MG symptoms deterioration and (2) admission for MG-related management, including thymectomy or for immunotherapy. The exclusion criteria were (1) incomplete data and (2) admission that unrelated to myasthenia gravis. A total of 188 hospital admissions were excluded because they were not due to MG and 13 were excluded due to data loss, respectively. After cleaning the data, data from 200 patients with 312 hospital admissions were used for the analyses. The data in cases where the same patient had been hospitalized for the same reason were merged. Finally, a total number of 228 hospitalizations (including 200 patients) were used for the analysis (Figure 1). The protocol of this study was evaluated and deemed acceptable by the Research Ethics Review Committee of the Shin Kong Wu Ho-Su Memorial Hospital (No. 20190109R).

Participant and Study Design
The data of 513 hospital admissions of patients with MG who were admitted to t Shin-Kong Wu Ho-Su Memorial Hospital in Taipei, Taiwan, between December 2015 a October 2018, were retrospectively analyzed. The inclusion criteria for patients with M were (1) admission due to MG symptoms deterioration and (2) admission for MG-relat management, including thymectomy or for immunotherapy. The exclusion criteria we (1) incomplete data and (2) admission that unrelated to myasthenia gravis. A total of 1 hospital admissions were excluded because they were not due to MG and 13 were e cluded due to data loss, respectively. After cleaning the data, data from 200 patients w 312 hospital admissions were used for the analyses. The data in cases where the sam patient had been hospitalized for the same reason were merged. Finally, a total numb of 228 hospitalizations (including 200 patients) were used for the analysis (Figure 1). T protocol of this study was evaluated and deemed acceptable by the Research Ethics R view Committee of the Shin Kong Wu Ho-Su Memorial Hospital (No. 20190109R).

Data Collection and Clinical Measurement
A retrospective review of medical records, including information on the age, sex, a at diagnosis, disease duration, autoantibodies present, medications used, maximum do age of corticosteroid before admission, thymic histology, history of thymectomy, tre ment during hospitalization, and length of ICU admission, was conducted. Disease sev ity was graded according to the classification of the Myasthenia Gravis Foundation America (MGFA) classification. Twenty clinical variables were collected. Table 1 li twenty clinical factors (variables X1-X20) associated with patient with MG that may aff ICU admission.
The inclusion criteria for patients with MG were (1) Myasthenia Gravis Foundati of America (MGFA) class II and III, and (2) no medication adjustment in the last 6 mont The exclusion criteria were (1) unstable MG symptoms and (2) history of intensive i muno-modulation therapy, including immunoglobulins, high dose intravenous cortic steroid, or plasmapheresis, 6 months before enrollment-because the use of these sh action immunotherapy means that the patient has a life-threatening phenomenon and u stable symptoms. Patients were eligible if they were diagnosed with MG based on t MGFA criteria. Briefly, the diagnosis of MG was based on fluctuating muscle weakne with fatigability, decreased symptom severity after use of acetylcholinesterase inhibito decremental changes in repetitive nerve stimuli on repetitive nerve stimulation test, presence of anti-AchR autoantibodies.
The MGFA clinical classification was based on previous reviews that represented t clinical severity of the patient upon admission [24]. The maximum daily oral steroid do

Data Collection and Clinical Measurement
A retrospective review of medical records, including information on the age, sex, age at diagnosis, disease duration, autoantibodies present, medications used, maximum dosage of corticosteroid before admission, thymic histology, history of thymectomy, treatment during hospitalization, and length of ICU admission, was conducted. Disease severity was graded according to the classification of the Myasthenia Gravis Foundation of America (MGFA) classification. Twenty clinical variables were collected. Table 1 lists twenty clinical factors (variables X1-X20) associated with patient with MG that may affect ICU admission.
The inclusion criteria for patients with MG were (1) Myasthenia Gravis Foundation of America (MGFA) class II and III, and (2) no medication adjustment in the last 6 months. The exclusion criteria were (1) unstable MG symptoms and (2) history of intensive immunomodulation therapy, including immunoglobulins, high dose intravenous corticosteroid, or plasmapheresis, 6 months before enrollment-because the use of these short action immunotherapy means that the patient has a life-threatening phenomenon and unstable symptoms. Patients were eligible if they were diagnosed with MG based on the MGFA criteria. Briefly, the diagnosis of MG was based on fluctuating muscle weakness with fatigability, decreased symptom severity after use of acetylcholinesterase inhibitors, decremental changes in repetitive nerve stimuli on repetitive nerve stimulation test, or presence of anti-AchR autoantibodies. The MGFA clinical classification was based on previous reviews that represented the clinical severity of the patient upon admission [24]. The maximum daily oral steroid dose before admission was recorded from the dosages during outpatient visits conducted within 1 month before hospitalization. Disease duration was defined as the time from the onset to the first visit after 1 December 2015.
The history of thymectomy was divided into three categories, as follows: (1) the patient had never undergone thymectomy; (2) the patient had undergone thymectomy during this admission; (3) thymectomy had been performed previously. The treatment during hospitalization included plasmapheresis, intravenous corticosteroid administration, intravenous immunoglobulin administration, and rituximab administration. Treatment with plasmapheresis was divided into three categories, as follows: (1) the patient did not undergo plasmapheresis; (2) the patient underwent five sessions; (3) the patient underwent more than five sessions. The serology status of MG autoantibodies included anti-AChR antibody and anti-MuSK antibody positivity or negativity, as well as double seronegativity.
As per the protocol of our hospital, patients with MG who are hospitalized for thymectomy must be observed in the ICU for 1 day after the operation. This distinguishes which groups of MG patients require ICU admission and then further divides them into two categories: ICU admission more than 1 day and less than 1 day. Most patients with MG are treated regularly in the ward, and those who require thymectomy are required for admission to the ICU for 1 day. Therefore, ICU admission in our study was defined as greater than 1 day.
In total, 228 patients included in the study ( Figure 1) along with 20 clinical variables. Patient demographics are presented in Table 2. The average age at admission was 49.1 years with female predominance (61.4%). The average disease duration was 68.75 months. The average age at onset of MG symptoms was 43.2 years. Of the patients, 88.2% showed anti-AChR antibody positivity, 4.8% showed anti-MuSK antibody positivity, and 7.5% showed double seronegativity. A total of 12.7% of the patients were admitted to the ICU for more than 1 day.

Machine Learning-Based Decision Tree Analysis
ML-based decision tree algorithms are popular and effective approaches for clinical/healthcare classification problems that visually represent the decision rules of generated predictions using a tree-shaped figure [25][26][27]. A decision tree is composed of nodes that are the optimum spilt of each feature, calculated by Gini or Entropy [28]. Gini measures the probability that any element of the dataset will be mislabeled when it is randomly labeled. Similar to Gini, Entropy measures information that indicates the disorder of the features with the target. Thus, this study utilized three most used tree-based algorithms including the following: CART, a decision tree based on the Gini method [29]; C4.5, a decision tree based on the Entropy method [30]; C5.0, an upgraded version of C4.5 that adds several facilities, such as variable misclassification costs [31]. The logistic regression (LR) is used as a baseline for performance comparison as it is a classic regression algorithm that focused on binary classification problems by calculating the natural logarithm of an odds ratio (logit). It predicted the logit of the dependent variable, that is, the ratio of probabilities of the dependent variable occurring from the logit of independent variables [32]. Figure 2 shows the overall flowchart of the proposed scheme. In the proposed scheme, we first collected patients with MG and identified the subjects to prepare the dataset for model construction, then the dataset was randomly split into 80% training dataset for model building and 20% testing dataset for out of sample testing. Twenty clinical variables (X1 to X20) were used as predictor variables and ICU admission (Y) was applied as the target variable.
During the training phase, the hyperparameters of CART, C4.5, and C5.0 methods needed to be tuned for constructing a relatively well performed model, so we applied a 10-fold cross-validation technique for tuning the hyperparameters of each method. The training dataset was further randomly divided into 10 equal-sized folds, then 9 folds were used as a training dataset to build the model with a different set of hyperparameters, the remaining 1 fold was used as a validation dataset for model validation. The 10-fold crossvalidation was repeated 10 times, by changing the fold of the validation dataset, thus the best hyperparameters set, which generated the best validation performance, based on the area under the receiver operating characteristic (ROC) curve (AUC) metric for each decision tree method, which was then used to construct the best CART, C4.5, and C5.0 models.
In the model evaluation phase, the testing dataset was used to evaluate the predictive performance of the best CART, C4.5, and C5.0 models. The metrics used for performance comparison were accuracy, sensitivity, specificity, AUC, and F1 score. In order to provide a more robust comparison, the process mentioned above was randomly repeated 10 times, and the averaged metrics were used to find the best decision model among the best CART, C4.5, and C5.0 models.
Finally, based on the best decision tree model, the decision rules could be developed, and the important clinical variables were identified. The rules were then discussed to improve the early recognition and care of patients with MG who need intensive care. During the training phase, the hyperparameters of CART, C4.5, and C5.0 methods needed to be tuned for constructing a relatively well performed model, so we applied a 10-fold cross-validation technique for tuning the hyperparameters of each method. The training dataset was further randomly divided into 10 equal-sized folds, then 9 folds were used as a training dataset to build the model with a different set of hyperparameters, the remaining 1 fold was used as a validation dataset for model validation. The 10-fold crossvalidation was repeated 10 times, by changing the fold of the validation dataset, thus the best hyperparameters set, which generated the best validation performance, based on the area under the receiver operating characteristic (ROC) curve (AUC) metric for each decision tree method, which was then used to construct the best CART, C4.5, and C5.0 models.
In the model evaluation phase, the testing dataset was used to evaluate the predictive performance of the best CART, C4.5, and C5.0 models. The metrics used for performance comparison were accuracy, sensitivity, specificity, AUC, and F1 score. In order to provide a more robust comparison, the process mentioned above was randomly repeated 10 times, and the averaged metrics were used to find the best decision model among the best CART, C4.5, and C5.0 models. All methods were implemented in the R software with the 3.6.2 version. LR was constructed by the "blorr" package with the 0.3.0 version [33]; CART was constructed by the "rpart" package with the 4.1.15 version [34]; C4.5 was constructed by the "RWeka" package with the 0.4.42 version [35]; C5.0 was constructed by the "C50" package with the 0.1.5 version [36]. The hyperparameters of all the methods used were tuned by the "caret" package with the 6.0.88 version [37].

Results
According to the proposed scheme, for modeling effective CART, C4.5, and C5.0 models, the hyperparameters of each method were tuned and evaluated. The LR method without hyperparameters tuning-the baseline method-was constructed by using the proposed scheme. The values of the hyperparameters which generated the best CART, C4.5, and C5.0 models with the highest AUC values are listed in Table 3. Figure 3 uses confusion matrices to demonstrate the predicted results of LR, CART, C4.5, and C5.0 methods. From this figure, it can be observed that the best C5.0 method generated the best positive and negative predicted results compared with that of the best LR, CART, and C4.5 methods. Table 3. Summary of the values of the hyperparameters for the best CART, C4.5, and C5.0 models.

Hyperparameters Value Meaning
CART minispilt 20 The minimum number of observations that must exist in a node for a split to be attempted. Finally, based on the best decision tree model, the decision rules could be developed, and the important clinical variables were identified. The rules were then discussed to improve the early recognition and care of patients with MG who need intensive care.
All methods were implemented in the R software with the 3.6.2 version. LR was constructed by the "blorr" package with the 0.3.0 version [33]; CART was constructed by the "rpart" package with the 4.1.15 version [34]; C4.5 was constructed by the "RWeka" package with the 0.4.42 version [35]; C5.0 was constructed by the "C50" package with the 0.1.5 version [36]. The hyperparameters of all the methods used were tuned by the "caret" package with the 6.0.88 version [37].

Results
According to the proposed scheme, for modeling effective CART, C4.5, and C5.0 models, the hyperparameters of each method were tuned and evaluated. The LR method without hyperparameters tuning-the baseline method-was constructed by using the proposed scheme. The values of the hyperparameters which generated the best CART, C4.5, and C5.0 models with the highest AUC values are listed in Table 3. Figure 3 uses confusion matrices to demonstrate the predicted results of LR, CART, C4.5, and C5.0 methods. From this figure, it can be observed that the best C5.0 method generated the best positive and negative predicted results compared with that of the best LR, CART, and C4.5 methods.  The performance of the LR, CART, C4.5, and C5.0 methods with 10 repetitions is shown in Table 4, with the average and standard deviation (SD) of the 5 metrics used in this study. As shown in the table, the CART, C4.5, and C5.0 methods have better AUC performance than the classic LR. Among all three decision tree algorithms, C5.0 had the highest average AUC (0.814), followed by CART and C4.5. The C5.0 method also performed best in terms of the accuracy (0.942), sensitivity (0.994), and F1 score (0.967). The ROC curves, as well as the one SD of the mean AUCs of all methods, are shown in Figure  4. The figure shows that C5.0 is the best predictive model in this study.  Table 4, with the average and standard deviation (SD) of the 5 metrics used in this study. As shown in the table, the CART, C4.5, and C5.0 methods have better AUC performance than the classic LR. Among all three decision tree algorithms, C5.0 had the highest average AUC (0.814), followed by CART and C4.5. The C5.0 method also performed best in terms of the accuracy (0.942), sensitivity (0.994), and F1 score (0.967). The ROC curves, as well as the one SD of the mean AUCs of all methods, are shown in Figure 4. The figure shows that C5.0 is the best predictive model in this study.  As C5.0 has the best AUC and outperforms the four competing methods, the important clinical factors and decision rules generated and suggested by the best C5.0 model are discussed. Table 3. Summary of the values of the hyperparameters for the best CART, C4.5, and C5.0 models.

Methods Hyperparameters Value Meaning
CART minispilt 20 The minimum number of observations that must exist in a node for a split to be attempted.  As C5.0 has the best AUC and outperforms the four competing methods, the important clinical factors and decision rules generated and suggested by the best C5.0 model are discussed.

Discussion
This is the first study to use the ML decision tree method for predicting ICU admission in patients with MG. The C5.0 method generated the best and most promising classification results and provided an output of six clinical features that were critical for determining the risk of ICU admission. Figure 5 shows the decision rules for the prediction of ICU admission in MG patients based on the six important clinical factors of the best model-the C5.0 model. Table 5 summaries decision rules of combinations of clinical factors from Figure 5. The rules in Figure 4 and Table 5 are then discussed to improve the early recognition and care of patients with MG who need intensive care.

Discussion
This is the first study to use the ML decision tree method for predicting ICU admission in patients with MG. The C5.0 method generated the best and most promising classification results and provided an output of six clinical features that were critical for determining the risk of ICU admission. Figure 5 shows the decision rules for the prediction of ICU admission in MG patients based on the six important clinical factors of the best model-the C5.0 model. Table 5 summaries decision rules of combinations of clinical factors from Figure 5. The rules in Figure 4 and Table 5 are then discussed to improve the early recognition and care of patients with MG who need intensive care.   The MGFA classification and the presence of thymoma were two of the important physiological indices. The MGFA clinical classifications are used to identify the different clinical features and severity of patients with MG [38]. The higher the class of MGFA, the more severe the symptoms. The severity at the onset of MG constituted a grave risk in our patients, and the MGFA class at admission reflected the severity of MG upon admission. The association of disease severity with a high risk of death could be explained by the frequent involvement of the bulbar and respiratory muscles in these individuals. An MGFA score of 4 indicates severe MG symptoms or an acute crisis that may require ventilator support or intensive care [38]. Another physiological index for MG that influences ICU admission is the presence of thymoma, which was found in approximately 15%-60% of MG cases [39][40][41]. The presentation of thymoma is caused by an immune response in thymoma cells [42]. Studies showed controversial outcomes and disease severities regarding different thymic pathologies [43]. Zhang et al. also demonstrated that MG patients with thymoma had a poorer prognosis than patients without thymoma [39] due to serious disease manifestations. Our results are in line with those of previous studies that show the important role of thymoma in ICU admission among patients with MG.
In addition, this study demonstrated that some treatments have influenced ICU admission in patients with MG, including treatment with oral azathioprine (AZA). According to the international guidelines for the management of MG, immunosuppressive therapy is used in patients with MG who have poor response to pyridostigmine alone [5]. AZA was added early if the patient had comorbidities, such as diabetes, significant depression (with steroids potentially exacerbating their mood), osteoporosis, and leg ulcerations and could not tolerate steroid treatment [39,44]. Therefore, patients used AZA due to comorbidities and side effects that cause physicians to change their medications. This also meant that such patients are likely to have other comorbid diseases that influence their need for ICU admission.
The onset age and disease duration also influenced ICU admission in patients with MG. Our results showed that late-onset MG, defined as MG at an onset age >50 years, had a negative prediction for ICU admission. Previous studies indicated that patients with late-onset MG are likely to have a thymoma and a severe disease that is difficult to treat [45,46]. An observational cross-sectional multicenter study showed that patients with late-onset MG may present with more severe symptoms than younger patients [47]. Other large cohort studies suggested that late-onset MG patients are prone to increased disease severity, and the mortality rate increased in the elderly [15,48]. This is because elderly patients with MG tend to have comorbidities and complications, such as sepsis, resulting in long hospital stays and high costs [49]. Old onset age of MG could be associated with an increased susceptibility to autoimmune diseases, due to immune dysregulation and increased inflammatory background that can cause a high production of autoantibodies [49]. Although the elderly have a more severe presentation than younger patients, they require a low dosage of medications, have better prognosis to management, and a short weaning time in the ICU after a myasthenic crisis [48]. This may explain why elderly patients were less likely to be admitted in ICU in our study.
Our findings found that disease duration shorter than 41 months was a factor that could influence ICU admission in patients with MG [50]. Many reports have not concluded that disease duration is closely associated with the prognosis in patients with MG [51]. A large retrospective study, that is very similar to ours, demonstrated that the majority of deaths occurred 5-10 years after the onset of the disease and rarely within 5 years; however, the risk of death tended to decrease after 15 years of the prevalence of the disease [52]. Since it is an autoimmune disease, proper medical intervention helps stabilize the symptoms significantly [52]. This may be because the longer the course of the disease, the more stable the drug treatment and the better the psychological adaptation of the patient to the disease, resulting in a lower the rate of hospitalization requiring ICU admission. Our findings, distinguished from other studies, defined an absolute point-that disease duration of 41 months can predict ICU admission. To the best of our knowledge, no previous study has defined how long the duration of the disease may affect the outcomes of MG, and our studies provide new insights into the clinical intensive care of MG.
Management of MG upon admission would benefit from a good understanding of the disease course. In this study, many of the risk factors associated with the prognosis of hospitalized MG patients requiring ICU admission were identified using the C5.0 algorithm. C5.0 is a promising ML-based decision tree algorithm and has been successfully used in clinical/healthcare issues [26,53]. Previously published predictive factors of the prognosis of MG in MG deterioration, severity, and hospital stay have used multivariable logistic regression analyses [49,54,55]. This research found that, compared with LR, C5.0 and the other two ML-based decision tree algorithms can generate a better accuracy. The results confirmed that ML algorithms are important for disease detection and risk assessment in several autoimmune diseases, such as rheumatoid arthritis [56,57]. ML can help clinicians detect and process clinically useful information in small patient samples, gain a good understanding of disease courses, adapt treatments earlier, and find the best management plan [56]. Most importantly, the natural course of an MG phenotype is highly variable and impacts the disease clinical course and prognosis significantly, with no good prediction target [58]. Our study tried to use the ensemble learning method to construct a decision tree model for predicting ICU admission in MG patients during hospitalization for future studies.
Some investigators have evaluated the potential risk factors of ICU care in patient with MG. A retrospective study showed that higher MG-ADL scores with bulbar involvement and higher MGFA classification were associated more frequently in the ICU group. Better outcomes may be obtained with early intensive care management if the patient presents with those factors [18]. Liu et al. suggested that CO2 level before intubation and the score on MG-ADL at onset may be associated with prolonged ICU stays [20]. Some respiratory management had been found for prevention of intubation, and prolonged ventilator use in patients with MG had also been reported, including bilevel positive airway pressure (BiPAP) [13], and hypercapnia could also be a predicting factor for early intubation under BiPAP [12]. However, there was still no valid guidance that could identify the need for intensive care in MG early.
This study had several clinical implications. First, the variable phenotype of MG makes it difficult to determine the optimal management plan and prognosis; therefore, physicians can use this decision tree model to identify patients likely to have ICU admission during hospitalization. Second, this study may provide a possible tool for clinical guidance for ICU clinicians to improve intensive care quality. This is because patients with MG who are likely to stay in the ICU have a high mortality, require a ventilator, and suffer severe disability. Due to the improvement in management and development of intensive care techniques, the fatality rate of MG has been less than 5% in the recent years [15]. An early diagnosis of MG patients who need ICU can improve survival and outcomes. Third, as ML expands its access to health care for patients with rheumatic disease, this study, using ML methods to assess ICU admission, revealed a new point of view, and the results showed that ML methods can provide a predictive accuracy. Using this method could improve the quality of care of patients with MG. The model can be combined with other clinical parameters, including the respiratory rate, difficulty with phonation, weak neck muscles, oxygenation, and advanced electronic functionalities, such as monitoring of partial pressure of carbon dioxide and testing of vital capacity. This study recommends the model as a primary benchmarking tool to be used in the evaluation of MG patients during hospitalization.
There are some limitations to this study. First, the usefulness of the model is probably restricted to our hospital because of inter-hospital differences that impact the model of ICU strategy in MG. Second, the data were acquired from chart reviews, and the specific details of each patient, such as the quantitative MG score, which can quantify the severity and outcomes of treatment, were not available for analysis. Third, due to the heterogeneity of MG symptoms, our model failed to account for additional clinical factors known to influence ICU admission. These include the partial pressure of carbon dioxide at admission, the activities of daily living score at myasthenia crisis onset, and nosocomial infection [18,20]. Finally, these models were chosen based on clinical data. Other variables, such as corticosteroid dose, treatment period, and previous underlying diseases, including chronic obstructive pulmonary disease and diabetes, were not included in our analysis. Multicenter studies should be performed to ensure that the results are not due to artifacts in ML systems. In addition, multicenter studies may complete the framework of this study.

Conclusions
This study uses an ML-based decision tree approach to predict ICU admission in patients with MG. Decision tree methods are promising tools that can build a predictive model of ICU admission in patients with MG, and they can provide physicians with information to evaluate the potential risk of ICU admission in patients with MG. Due to the varying clinical presentation of patients with MG, the model produced can be used for performance benchmarking and as a supportive tool for alerting clinicians regarding the patients with MG who require intensive care and management; therefore, enabling clinicians to provide timely and effective treatment, improving care quality and patient outcomes.