Toward Evaluating Critical Factors of Extubation Outcome with XCSR-Generated Rules

Predicting the correct timing for extubation is pivotal for critically ill patients with mechanical ventilation support. Evidence suggests that extubation failure occurs in approximately 15–20% of patients, despite their passing of the extubation evaluation, necessitating reintubation. For critically ill patients, reintubation invariably increases mortality risk and medical costs. The numerous parameters that have been proposed for extubation decision-making, which constitute the key predictors of successful extubation, remains unclear. In this study, an extended classifier system capable of processing real-value inputs was proposed to select features of successful extubation. In total, 40 features linked to clinical information and variables acquired during spontaneous breathing trial (SBT) were used as the environmental inputs. According to the number of “don’t care” rules in a population set, Probusage, the probability of the feature not being classified as above rules, can be calculated. A total of 228 subjects’ results showed that Probusage was higher than 90% for minute ventilation at the 1st, 30th, 60th, and 90th minutes; respiratory rate at the 90th minute; and body weight, indicating that the variance in respiratory parameters during an SBT are critical predictors of successful extubation. The present XCSR model is useful to evaluate critical factors of extubation outcomes. Additionally, the current findings suggest that SBT duration should exceed 90 min, and that clinicians should consider the variance in respiratory variables during an SBT before making extubation decisions.


Introduction
In the intensive care unit (ICU), intubation with mechanical ventilators necessarily saves the lives of critically ill patients immediately fighting for their lives and gaining extra time to treat and recover. However, clinical reports provide evidence that prolonged mechanical ventilation can lead to the impairment of diaphragmatic or lung function [1][2][3]. Similarly, respiratory failure requiring reintubation occurs in approximately 15-20% of patients after extubation [4]. Delayed extubation or reintubation tends to increase mortality risk in critically ill patients. Therefore, it is critical to accurately determine whether patients Bioengineering 2022, 9, 701 2 of 13 still require mechanical ventilation support. The timing of weaning from mechanical ventilation also constitutes a crucial concern.
Patients who are intubated and undergoing machine ventilation support require frequent physician visits and checkups. After the stabilization of original cause, there is a procedure for determining the patient's health status called the ventilator weaning protocol. The first step is to observe the patient's daily assessment of readiness to wean, and to meet and check the weaning profiles, including the rapid shallow breathing index (RSBI), maximal inspiratory pressure (Pimax), cuff leak test, etc. [5][6][7][8]. The second step was to conduct a spontaneous breathing trial (SBT) with 120 min of low pressure support ventilation. The third step of airway protection and cough function assessment was performed. After all of them have passed, the decision to turn off mechanical ventilation and remove the endotracheal tube is made by a trained and certified physician. The strictly refined process still produces 15~20% extubation failure rate; therefore, many studies have introduced multivariate analysis, artificial intelligence, and other methods to find the appropriate factors. During SBT, respiratory distress may occur [9] and changes in the respiratory variables may also be associated with successful extubation [10]. It is, however, still not possible to completely exclude the possibility of failed extubation. The search for other key variables, the development of novel features, and the establishment of accurate predictive models for extubation based on physician decision rules are the major research and development in this study. A learning classifier system (LCS), which is a rule-based machine learning method, interacts with the environment to generate an optimal policy, modifying the classifier parameter values according to the environmental feedback to formulate more effective rules. A popular type of LCS is the extended classifier system (XCS), for which numerous applications have been proposed [11,12]. The input data for the original XCS must be encoded into binary format. However, most problems are more easily addressed by encoding with real values. Therefore, Wilson modified the XCS for the adoption of realvalue inputs; an XCS that can process such inputs is called an XCSR [13]. One feature of XCS is the "don't care" bits in the rules. This feature enables users to determine which variables are less important in some situations but more important in other situations. A study applied this feature to identify the critical variables for detecting Internet addiction in patients [14]. Thus, the XCSR is suitable for the present objective: determining and evaluating SBT-related variables are more important for successful extubation than others. The aim of this study was not only to construct a model for predicting extubation outcomes but also to evaluate the critical SBT effect with other variables.

Subjects Description
This retrospective study of the electronic medical records (EMR) database has been approved by the Institutional Review Board of Kaohsiung Medical University Chung-Ho Memorial Hospital (the approval number: KUMHIRB-F(1)-20200033). A total of 262 deidentified patient data were obtained from the above hospital. The recruited patient is on a ventilator for the first time and for more than 48 h and has stable vital signs before extubation. Since the literature has indicated that SBT with limited pressure support is more effective than T-tubes [15,16], the data of a 6 cmH 2 O pressure-supported SBT is collected in this study. Figure 1 presents the extubation procedure used in the ICU of the internal medicine department. The intensivist decides to stop ventilation support and execute the extubation procedure after each index meting the relevant criteria. Within 48 h of extubation, the patient who experienced respiratory failure or died is defined as the extubation failure group, and the patient who did not is defined as the extubation success group. Descriptions of the variables obtained from the EMR are presented in the following section. group, and the patient who did not is defined as the extubation success group. Descriptions of the variables obtained from the EMR are presented in the following section.

Variable
For evaluating critical factors, 40 variables for each patient were selected as the input variables. Table 1 lists the selected 19 variables of clinical information, consisting of basic patient information, two severity scales, mechanical ventilator-related information, hemodynamics situation before doing SBT, maximal inspiratory pressure, and disease history (Table 1). Table 2 presents another 21 variables of acquired information when doing SBT, including the hemodynamics and ventilation level at the 1st, 30th, 60th, and 90th minute. Regarding the output variable, the patient in the extubation failure group is encoded as 0 and the one in the success group is encoded as 1.

Variable
For evaluating critical factors, 40 variables for each patient were selected as the input variables. Table 1 lists the selected 19 variables of clinical information, consisting of basic patient information, two severity scales, mechanical ventilator-related information, hemodynamics situation before doing SBT, maximal inspiratory pressure, and disease history (Table 1). Table 2 presents another 21 variables of acquired information when doing SBT, including the hemodynamics and ventilation level at the 1st, 30th, 60th, and 90th minute. Regarding the output variable, the patient in the extubation failure group is encoded as 0 and the one in the success group is encoded as 1.

XCS and XCSR
The learning classification system (LCS) is a rule-based machine learning method that combines exploration components (evolutionary computation, EC) and learning components (reinforcement learning, RL). The development of LCS has been decades-long, and the most famous descendant in the field of the LCS is the extended classifier system (XCS). The XCS retains the main components of the LCS, but still achieves similar performance or even better. The streamlined architecture makes the XCS a benchmark for future development in the LCS field.
The architecture of the XCS is a standard RL. The XCS can be regarded as an agent interacting with the environment to achieve the desired goal. The environment can be a database to be explored or a problem to be solved. The XCS uses a set of cooperative rules to express the solutions it has learned. The rules are easily interpretable, and the expression of the rules is "IF condition THEN action". In addition, each rule also contains 3 parameters to evaluate the quality of the rule: (1) prediction, predict the reward after the action is executed; (2) prediction error, calculate the error between predicted and actual reward; (3) fitness, evaluate the quality and reliability of rule. The reward received from the environment is used to update the rule parameters using Q-learning and the Widrow-Hoff delta rule. Introducing EC can help the XCS reorganize and explore better rules. The XCS has also been proven to be able to find the most general and accurate rules.
The operation process of the XCS is described as follows: (1) Environmental input is converted to binary encoding through the detector. The set of rules is called population [P] in the XCS. The condition part of the rule will correspond to the environmental input and use ternary-alphabet {0, 1, #} for encoding. The # symbol is treated as "don't care", which means it can match regardless of whether the corresponding environmental input is 0 or 1. The action part of the rule adopts binary encoding. The XCS will search for all the rules in [P] and move the rules whose condition part matches the environmental input to the matching set [M]. If [M] is empty, the XCS will start the covering mechanism. In the covering mechanism, the XCS will generate a new rule that matches the condition part with the environmental input and randomly give a binary code to the action part. In the condition part, there is a certain probability to replace the regular bit with the # symbol. The remaining parameters are given random initial values. The generated rule will be placed in [ The XCS has two modes to select action for execution, namely exploration and exploitation. In the exploration mode, the XCS adopts the roulette-wheel method, which means that the larger the expected reward, the easier to be picked. In the exploitation mode, the XCS directly selects the action with the most expected reward to execute. The selected action will be executed through the effector, and the environment will give reward. The rules in [M] whose action is the same as the selected action will be moved to action set [A]; (3) The reward obtained from the environment will be used to update the prediction, prediction error, and fitness parameters of rules in [A] through Q-learning and the Widrow-Hoff delta rule; (4) The parameters of the rule corresponding to [A] in [P] will be updated; (5) The EC method in the XCS is to use genetic algorithm (GA). When the average time of the rule in [A] from the last GA execution is greater than the threshold, GA will execute. GA will use fitness as the probability to select two rules for crossover and mutation to generate two new rules. In the XCS, the alleles of condition in the two rules will be exchanged with probability χ, and the alleles will be randomly flipped and mutated with the probability µ; (6) The XCS also has subsumption mechanism to assist in learning the most general and most accurate rules. The XCS will determine two rules corresponding to the same environmental input, and whether the more general rule has enough experience and reliability to accommodate another specific rule; (7) The XCS will repeat the above steps until it converges or reaches the maximum number of iterations.
The variables in real-world data are often continuous values. A new encoding is proposed in the condition part of the rule to allow the XCS to handle input data other than binary. The modified XCS version is called XCS with real-valued input (XCSR) [13]. The schematic of the XCSR and the corresponding operation procedure are shown in Figures 2 and 3, respectively. The XCSR mainly converts the encoding of the condition of the rules from ternary-alphabet to interval predicates. The expression of interval predicate is int i = (c i , s i ), where int i represents the i-th interval predicate corresponding to the i-th input variable, c i represents the center, and s i represents the spread. When an i-th input variable x i can match the i-th interval predicate int i , it means (c i − s i ≤ x i < c i + s i ). The operating mechanism of XCSR is the same as XCS, except that the mechanism is slightly changed in covering and GA. When covering generates a new rule, it will generate int i corresponding to the x i in the condition of the rule, where the value of c i is set to x i, and the value of s i is set to a random number ranging from 0 to s r , where s r is defined by the user. In the part of GA, the int i of the two rules will be exchanged with probability χ. c i and s i will randomly add or subtract the value of m i with probability µ, where m i is defined by the user. Bioengineering 2022, 9, x FOR PEER REVIEW 6

Data Analysis
This study will import clinical data of 228 patients with 40 variables as input da the XCSR. The rule condition of the XCSR contains 40 interval predicates, and the ou action is the success or failure of extubation. Table 3 shows the parameter settings of X used in this study. The detailed description of XCSR can be referred to [13]. After th ues of the 40 variables of the 228 patients were input into the XCSR, it generated 154 for extubation outcome prediction after the model reached stability. Therefore, accor to the number of "don't care" bits of each variable in the rules, the usage probability busage can be calculated. A larger Probusage in the ith variable indicates that more m rules use this variable to determine the extubation outcomes. The Probusage of the ith able is calculated as follows:

Data Analysis
This study will import clinical data of 228 patients with 40 variables as input data the XCSR. The rule condition of the XCSR contains 40 interval predicates, and the ou action is the success or failure of extubation. Table 3 shows the parameter settings of XC used in this study. The detailed description of XCSR can be referred to [13]. After the ues of the 40 variables of the 228 patients were input into the XCSR, it generated 154 r for extubation outcome prediction after the model reached stability. Therefore, accord to the number of "don't care" bits of each variable in the rules, the usage probability P busage can be calculated. A larger Probusage in the ith variable indicates that more mo rules use this variable to determine the extubation outcomes. The Probusage of the ith v able is calculated as follows:

Data Analysis
This study will import clinical data of 228 patients with 40 variables as input data for the XCSR. The rule condition of the XCSR contains 40 interval predicates, and the output action is the success or failure of extubation. Table 3 shows the parameter settings of XCSR used in this study. The detailed description of XCSR can be referred to [13]. After the values of the 40 variables of the 228 patients were input into the XCSR, it generated 154 rules for extubation outcome prediction after the model reached stability. Therefore, according to the number of "don't care" bits of each variable in the rules, the usage probability Prob usage can be calculated. A larger Prob usage in the ith variable indicates that more model rules use this variable to determine the extubation outcomes. The Prob usage of the ith variable is calculated as follows: Prob usage (i) = total rules number − don't care bit number(i) total rules number × 100% (1) We constructed the model 10 times to verify the stability of the Prob usage of each variable. Therefore, the results are presented as means ± standard deviations of Prob usage .

Statistical Analysis
Because the two groups were of unequal size, the nonparametric Mann-Whitney U test was performed to examine the distribution between extubation success and failure. The continuous variables are expressed as means ± standard deviations, and the discrete variables are expressed as numbers. A p value of <0.01 indicated a significant difference between two groups in two-tailed tests. The area under the receiver operating characteristic (ROC) curve (AUC) was used to test the discriminatory ability of each variable, with results expressed as the AUC values. The analyses were conducted and the XCSR was coded using the LabVIEW platform (National Instruments Corporation, Austin, TX, USA).

Results and Discussion
After exclusions were made for 34 of the 262 patients because of missing data, the data of 228 patients remained for analysis. Table 4 presents the clinical information of the final sample. Height and ideal body weight (IBW) differed significantly between the two groups. The AUC of these two variables were both 0.63.   Table 5 lists the variables measured during the SBTs of the 228 patients. A significant between-group difference was only observed for the heart rate (HR) in the 60th minute. The AUC of this variable was 0.63.  Figure 4 presents the performance of one of the XCSR models (a total of 10 models were built, one of them for an example). A problem refers to an iteration of a prediction that has been created. For the model to approach 100% accuracy, only approximately 125 problems were required. Though the possibility of over fitting of the XCSR model cannot be excluded, Bioengineering 2022, 9, 701 9 of 13 the analysis of the rules created by the XCSR can still be useful information for clinical doctors to make a decision. Figure 4 presents the performance of one of the XCSR models (a total of 10 models were built, one of them for an example). A problem refers to an iteration of a prediction that has been created. For the model to approach 100% accuracy, only approximately 125 problems were required. Though the possibility of over fitting of the XCSR model cannot be excluded, the analysis of the rules created by the XCSR can still be useful information for clinical doctors to make a decision.  Figure 5 presents the "don't care" numbers of each variable in all rules after sorting. The first six variables had significantly fewer "don't care" numbers, meaning that they had higher usage probabilities. Table 6 lists both these high-performing variables and some others with lower rankings. Variables ranked 1-6 had a substantially higher Probusage than did other model variables. The Probusage of all variables ranked ≥ 7 was lower than 36%.
Although height, IBW, and HR in the 60th minute typically differ significantly between the extubation failure and success groups in relevant studies, the AUC values remain lower than 0.63. Few studies using the XCSR have applied the ROC analysis. However, in clinical research, ROC analysis is the most common method for comparing different models and variables. Therefore, in the present study, the rewards in the prediction  Figure 5 presents the "don't care" numbers of each variable in all rules after sorting. The first six variables had significantly fewer "don't care" numbers, meaning that they had higher usage probabilities. Table 6 lists both these high-performing variables and some others with lower rankings.
Bioengineering 2022, 9, x FOR PEER REVIEW 9 of 13 Figure 4 presents the performance of one of the XCSR models (a total of 10 models were built, one of them for an example). A problem refers to an iteration of a prediction that has been created. For the model to approach 100% accuracy, only approximately 125 problems were required. Though the possibility of over fitting of the XCSR model cannot be excluded, the analysis of the rules created by the XCSR can still be useful information for clinical doctors to make a decision.  Figure 5 presents the "don't care" numbers of each variable in all rules after sorting. The first six variables had significantly fewer "don't care" numbers, meaning that they had higher usage probabilities. Table 6 lists both these high-performing variables and some others with lower rankings. Variables ranked 1-6 had a substantially higher Probusage than did other model variables. The Probusage of all variables ranked ≥ 7 was lower than 36%.
Although height, IBW, and HR in the 60th minute typically differ significantly between the extubation failure and success groups in relevant studies, the AUC values remain lower than 0.63. Few studies using the XCSR have applied the ROC analysis. However, in clinical research, ROC analysis is the most common method for comparing different models and variables. Therefore, in the present study, the rewards in the prediction Variables ranked 1-6 had a substantially higher Probusage than did other model variables. The Probusage of all variables ranked ≥7 was lower than 36%.
Although height, IBW, and HR in the 60th minute typically differ significantly between the extubation failure and success groups in relevant studies, the AUC values remain lower than 0.63. Few studies using the XCSR have applied the ROC analysis. However, in clinical research, ROC analysis is the most common method for comparing different models and variables. Therefore, in the present study, the rewards in the prediction array were applied to the XCSR model for the ROC analysis. The AUC of 0.993 was significantly higher than that of individual variables. Moreover, the accuracy of the XCSR model in predicting extubation outcomes approached 100% after training. For comparison with the XCSR model, the accuracy of height, IBW, and HR in the 60th minute was determined as 59%, 65%, and 65%, respectively, by using the cutoff point from the ROC analysis to predict extubation outcomes. The prediction accuracy for individual variables was relatively low because the ratio between the number of extubation failures and successes was approximately 0.83, meaning that the accuracy would be 83% if the model predicted that all extubation outcomes would be successful. Nevertheless, these results demonstrate how powerful the present XCSR model is. In the population rule set, 130 and 24 rules predicted extubation success and failure, respectively. Overall, the rules predicted the extubation outcomes with high accuracy. Table 6. Usage probabilities of the top 10 variables. The first 6 variables have significant higher probabilities.

Ranking
Variable Prob usage (%) After the model was constructed 10 times, although the Prob usage rank of some variables differed, the first six positions remained filled by the same variables. Moreover, the Prob usage of these six variables was higher than 95%, with small standard deviations. Overall, the results indicate that the respiratory rate (RR) and minute ventilation (VE) measured during the SBTs, as well as body weight (BW), were more important than the other variables. In particular, VE was critical at all-time points during the SBT. Studies have demonstrated that some SBT-related respiratory variables are risk factors for extubation failure [10,[17][18][19]. One study noted that VE is a parameter that is easy to measure during SBTs that can guide decisions regarding the discontinuation of mechanical ventilation [17]. Another study reported that including all variables (especially the variables in 90th minutes) assessed during SBTs in a training model resulted in higher performance [20]. The current results also suggest that VE during SBT is a key predictor of successful extubation.
Patients who are able to complete an SBT are likely to have more stable breathing than those who are not. This may explain why respiratory rate in the 90th minute constitutes a major factor in extubation decision-making. Notably, BW can also be a critical factor for successful extubation. Because the rules generated by the XCSR are presented in a range, the values may not be continuous. However, the reason BW is typically used for rules in XCSR models remains unclear. We speculate that patients may feel more comfortable and breathe more stably if their BW is regulated within a certain range. Further research is warranted to confirm this supposition.
Some studies have also applied machine learning to predict extubation success or failure. Zeggwagh used multivariable logistic regression (MLR) and three variables (vital capacity, RSBI, and maximal expiratory pressure) to create a prediction model [21]. Mueller (2004) observed that a model created using an artificial neural network (ANN) outperformed one created using MLR [22] in predicting extubation outcomes in infants. Another study revealed that an ANN model more accurately predicted extubation failure [23]. Mueller (2013) tested five machine learning methods, namely MLR, ANNs, boosted decision trees, naïve Bayesian classifiers, and support vector machines, to predict extubation outcomes in a large sample of premature infants [24]. Two common statistical models, MLR and partial least squares (PLS) regression, were also used to test the same dataset of present study. The AUCs of MLR (0.660) and PLS (0.812) were significantly lower than that of the XCSR model (0.993). Table 7 presents a summary of the AUC results obtained in previous studies and the current study. Because each study applied different database and variables for model training, the summary of Table 7 is just for reference instead of comparison. Interpreting the nonlinear models and determining which variables were more important than others are challenging, especially when ANNs are used. Relevant studies have typically employed the trial-and-error approach to fine-tune models and variables. Though some method had been proposed for NN model interpreting, the XCSR model is easier to interpret. In the present study, each rule created by the XCSR was readable (each variable in a rule would be a range of value or "don't care"). In general, interpreting an XCSR model is easier than a model formulated using other nonlinear machine learning methods. The usage probabilities calculated in the present study may be useful for identifying the importance of each variable. In addition, the distributions of each variable in different rules also can be useful information for finding the physiological meaning of patients with fail extubation and patients with successful extubation. These rules may serve as a valuable reference for further research.
Recently, a study also proposed a method to visualize rule-based machine learning [25]. In future work, it is possible to apply the method for identifying the importance of each variable to compare with present study and to explore detailed information from the XCSR. In addition, the original XCSR had been proposed for almost two decades. Many modified versions of the XCSR were proposed in these years. To study which one of the modified XCSRs is more suitable for the purpose of this study is also a future work. The present XCSR system is already built in the system of Kaohsiung Medical University Chung-Ho Memorial Hospital for providing the extubation suggestion. In the future work, the prospective research of this XCSR system for extubation suggestion will be investigated.

Conclusions
The XCSR model trained in the present study had a prediction accuracy of almost 100%. We identified critical factors of extubation success and failure from among 40 variables, observing that the respiratory variables assessed during SBTs were more important. The generated rules used over 95% of the VE measured in the 1st, 30th, 60th, and 90th minutes, as well as the respiratory rate in the 90th minute and BW. These parameters, especially VE, are the key to determining extubation success or failure. These findings suggest that the duration of an SBT should be longer than 90 min, and that extubation decisions should be made in consideration of the variability in such respiratory parameters during an SBT. Overall, relative to those of other methods, the AUC of the XCSR model was higher, facilitating the evaluation of the importance of each variable. Furthermore, each generated rule may constitute a useful reference for future investigations. Informed Consent Statement: Patient consent was waived due to only deidentified patient data were obtained.

Data Availability Statement:
The data presented in this study are available on request from the corresponding author. The data are not publicly available due to participates' privacy.