Integrated Machine Learning Decision Tree Model for Risk Evaluation in Patients with Non-Valvular Atrial Fibrillation When Taking Different Doses of Dabigatran

The new generation of nonvitamin K antagonists are broadly applied for stroke prevention due to their notable efficacy and safety. Our study aimed to develop a suggestive utilization of dabigatran through an integrated machine learning (ML) decision-tree model. Participants taking different doses of dabigatran in the Randomized Evaluation of Long-Term Anticoagulant Therapy trial were included in our analysis and defined as the 110 mg and 150 mg groups. The proposed scheme integrated ML methods, namely naive Bayes, random forest (RF), classification and regression tree (CART), and extreme gradient boosting (XGBoost), which were used to identify the essential variables for predicting vascular events in the 110 mg group and bleeding in the 150 mg group. RF (0.764 for 110 mg; 0.747 for 150 mg) and XGBoost (0.708 for 110 mg; 0.761 for 150 mg) had better area under the receiver operating characteristic curve (AUC) values than logistic regression (benchmark model; 0.683 for 110 mg; 0.739 for 150 mg). We then selected the top ten important variables as internal nodes of the CART decision tree. The two best CART models with ten important variables output tree-shaped rules for predicting vascular events in the 110 mg group and bleeding in the 150 mg group. Our model can be used to provide more visualized and interpretable suggestive rules to clinicians managing NVAF patients who are taking dabigatran.


Introduction
Non-valvular atrial fibrillation (NVAF) is a growing health issue in the aging society [1,2]. Elderly individuals and those with comorbidities have an increased risk of ischemic stroke and vascular events. Although new-generation nonvitamin K antagonists (NOACs) have demonstrated notable convenience and safety in the prevention of cardioembolic strokes [3], clinicians still struggle to achieve intensive medical control in NVAF patients with stroke risk [4][5][6][7][8]. Dabigatran etexilate, a direct thrombin inhibitor, is one NOAC that showed superior efficacy in vascular prevention compared to the traditional vitamin K antagonist, warfarin, when patients take high-dose dabigatran (150 mg twice daily) based on the results of the Randomized Evaluation of Long-Term Anticoagulant Therapy (RE-LY) trial [9]. Another approved low dabigatran dose (110 mg twice daily), which was non-inferior to warfarin regarding efficacy, had safety benefits, with lower rates of major bleeding. Patients with specific physical conditions and comorbidities have different risks of vascular events and bleeding [10,11]. Achieving optimal personalized medical prevention is desirable to balance the benefits and side effects of anticoagulant therapy based on patients' features. According to a post-hoc analysis of the RE-LY trial data, European label-simulated dabigatran treatment, which recommends adjusting dabigatran

Study Population
Our research adopted an ML method for secondary analysis based on the dataset of the RE-LY trial. The Fu Jen Catholic University Hospital's Research Ethics Review Committee examined and approved this study (IRB No. FJUH111180). Since the data solely contained de-identified content, informed permission was waived.
Dabigatran 110 or 150 mg twice daily or an adjusted dose of warfarin were the three treatment options in the RE-LY trial. There were 18113 patients with newly diagnosed arrhythmia and evidence suggesting secondary prevention randomly assigned to receive one kind of anticoagulant with a median follow-up time of approximately two years. A history of severe valvular heart disease, recent stroke, and renal failure was listed as exclusion criteria. A stroke or systemic embolism was the primary outcome, as major bleeding was the primary safety outcome [9]. The RE-LY trial was a multi-center, randomized trial with 99.9% of participants completing follow-up. Data from patients who received dabigatran and finished the RE-LY trial's follow-up were gathered for this study. Two study subgroups of patients receiving dabigatran 150 mg and 110 mg twice daily were designed.

Method
We proposed an integrated multi-step ML scheme (Figure 1) to construct a decisiontree model for risk evaluation in patients with NVAF taking different doses of dabigatran. Our protocol applied four ML algorithms: naive Bayes (NB), CART, random forest (RF), and extreme gradient boosting (XGBoost), which have been widely used in various medical informatics applications to select important variables [36][37][38][39][40].
LGR is a classic classification method that uses data fitting to a logistic function to estimate the likelihood of an event occurring [41]. The multivariate LGR model, which uses numerical or categorical predictor variables, was commonly used in medical research and as the benchmark method for comparing the performance of these ML methods. CART was used as the ML decision-tree algorithm to create our rules in the second step.
tree model for risk evaluation in patients with NVAF taking different doses of dabigatran. Our protocol applied four ML algorithms: naive Bayes (NB), CART, random forest (RF), and extreme gradient boosting (XGBoost), which have been widely used in various medical informatics applications to select important variables [36][37][38][39].
LGR is a classic classification method that uses data fitting to a logistic function to estimate the likelihood of an event occurring [40]. The multivariate LGR model, which uses numerical or categorical predictor variables, was commonly used in medical research and as the benchmark method for comparing the performance of these ML methods. CART was used as the ML decision-tree algorithm to create our rules in the second step.
CART is a binary splitting decision tree method based on the Gini index, which could be a classifier or regression predictive tool with an optimal tree size by applying a crossvalidation (CV) procedure [41]. NB is a well-known probabilistic classifier based on Bayes' theorem with strong independence assumptions between individual features [42]. RF is an ensemble classification method based on a decision tree [43]. Each tree grows to its best set of explanatory variables by the bootstrap resampling technique. The majority vote of these trees decide the model's final prediction. XGBoost is another tree-based learning algorithm with an interrelated base classifier [44]. Unlike RF, it adjusts the imperfections or inadequacies of the previous model in building new models to accelerate tree construction and prevent overfitting. First step: Define variables and classify the study subgroups. Our study referred to the recommendations of the American Heart Association and the European Society of Cardiology [1,2]. The personal features, medical history and biochemical data of the patients were defined as predictor variables. The target variable in the 110 mg group was vascular events (stroke, systemic embolism, and vascular death) and the target variable in CART is a binary splitting decision tree method based on the Gini index, which could be a classifier or regression predictive tool with an optimal tree size by applying a crossvalidation (CV) procedure [42]. NB is a well-known probabilistic classifier based on Bayes' theorem with strong independence assumptions between individual features [43]. RF is an ensemble classification method based on a decision tree [44]. Each tree grows to its best set of explanatory variables by the bootstrap resampling technique. The majority vote of these trees decide the model's final prediction. XGBoost is another tree-based learning algorithm with an interrelated base classifier [45]. Unlike RF, it adjusts the imperfections or inadequacies of the previous model in building new models to accelerate tree construction and prevent overfitting.
First step: Define variables and classify the study subgroups. Our study referred to the recommendations of the American Heart Association and the European Society of Cardiology [1,2]. The personal features, medical history and biochemical data of the patients were defined as predictor variables. The target variable in the 110 mg group was vascular events (stroke, systemic embolism, and vascular death) and the target variable in the 150 mg group was bleeding. The description of our predictive variables (X1-X17) and two target variables (P1, P2) are shown in Supplementary Table S1.
Second step: Construct an ML model to identify important variables of vascular events and bleeding in the two study subgroups. In our study, the dataset was randomly divided into an 80% training dataset for constructing the acceptable model with the best hyperparameter set and a 20% testing dataset for model testing. Our protocol applied a 10-fold CV method to enhance stability for tuning the hyperparameters for the best performance of NB, RF, CART, and XGBoost models based on the area under the receiver operating characteristic (ROC) curve (AUC) values. Tenfold CV involves the modifying the validation dataset ten times to generate the adequate validation performance [46]. The performance of the model was compared according to the AUC value superior to the result of LGR. We built each method with R software (http://www.R-project.org; accessed on 27 October 2022; https://www.rstudio.com/products/rstudio/; accessed on 27 October 2022) with the required installed packages. NB, RF, CART, and XGBoost were implemented, respectively. The "caret" R package version 6.0-90 was used for each method to generate the importance values of individual variables. Averaging the importance values of predictive variables in the selected models, we chose the ten most important variables for vascular events and bleeding in the 110 mg and 150 mg groups, respectively, as nodes in the decision tree model.
Third step: Develop decision rules using CART based on the selected important variables. Using the same model construction process in the previous step, the 10-fold CV was used to train the CART model. The CART decision tree model with the best performance was used to establish tree-shaped rules composed of the 10 selected variables for predicting vascular events and bleeding in the 110 mg and 150 mg groups, respectively.
In the last step, the rules derived from the important variables in our scheme were used to clarify what clinicians should avoid when selecting the dabigatran dose for NVAF patients. The discussion of these rules may provide us with graphical information on the precise dose of dabigatran.

Results
Our study recruited 5869 patients who took 110 mg dabigatran twice daily and 5933 patients who took 150 mg dabigatran twice. Table 1 shows the demographic data of the 110 mg and 150 mg groups. Vascular events and bleeding occurred in 185 (3.15%) and 1189 (20.04%) patients within the first year of follow-up in the 110 mg and 150 mg groups, respectively.  Table 2 shows the performances of LGR, NB, RF, CART, and XGBoost methods in predicting vascular events and bleeding in each group with their best hyperparameters. From the table, RF and XGBoost presented advanced predictive performance for vascular events in the 110 mg group and bleeding in the 150 mg group according to their AUC values. The ROC curves of the five methods in predicting vascular events and bleeding are shown in Figure 2. RF and XGBoost were selected as the modes because NB and CART showed inferior performance to LGR. Figure 3 shows the average importance values of variables in the RF and XGBoost models derived from the 10-fold CV. The top ten important variables in the two groups based on the average values in RF and XGBoost models are presented in Table 3.  The top ten important variables were used as inputs for rebuilding the CART decisiontree models. Figure 4 shows the performance of the CART model after adjusting for variables. The reconstructed CART model with 10 important variables generated better AUC values predicting vascular events (0.624) and bleeding (0.717) than that of the CART model with all 17 variables. After removing some variables, we obtained a more balanced performance and better AUC values in the CART models after adjusting for variables according to our feature selection scheme.   The top ten important variables were used as inputs for rebuilding the CART decision-tree models. Figure 4 shows the performance of the CART model after adjusting for variables. The reconstructed CART model with 10 important variables generated better AUC values predicting vascular events (0.624) and bleeding (0.717) than that of the CART model with all 17 variables. After removing some variables, we obtained a more balanced performance and better AUC values in the CART models after adjusting for variables according to our feature selection scheme. Interpreting the CART-generated rules in the 110 mg group, patients with a combination of a history of congestive heart failure (CHF), MI, diabetes mellitus (DM), kidney function abnormality, ethnicity, body mass index (BMI), and age feature had a tendency towards vascular events ( Figure 5). Patients with a combination of the concomitant use of drugs, kidney function abnormality, smoking habit, history of MI, BMI, and age features had a high risk of bleeding in the 150 mg group (Figure 6). Details regarding the decision rules predicting vascular events in the dabigatran 110 mg group and bleeding in the dabigatran 150 mg group are provided in Tables 4 and 5, respectively. The accuracy (ACC) demonstrated in these tables and figures is defining as below. Interpreting the CART-generated rules in the 110 mg group, patients with a combination of a history of congestive heart failure (CHF), MI, diabetes mellitus (DM), kidney function abnormality, ethnicity, body mass index (BMI), and age feature had a tendency towards vascular events ( Figure 5). Patients with a combination of the concomitant use of drugs, kidney function abnormality, smoking habit, history of MI, BMI, and age features had a high risk of bleeding in the 150 mg group (Figure 6). Details regarding the decision rules predicting vascular events in the dabigatran 110 mg group and bleeding in the dabigatran 150 mg group are provided in Tables 4 and 5

Discussion
To the best of our knowledge, our study is the first to apply an ML-based decision-tree model to NVAF patients taking dabigatran. Clinicians need to determine the balance point between preventing vascular events and avoiding bleeding. Although NOACs possess prior safety than warfarin, approximately 9-39% of patients had off-label low-dose NOACs in real-world research for concern about risk of bleeding or other side effects [27,47,48]. Many studies have identified risk factors and evaluation systems according to the LGR results [1,2]. An integrated ML scheme helped to identify important risk factors for vascular events and bleeding by analyzing the RE-LY trial data [25]. This kind ML scheme had more promising performance when we targeted vascular events in the dabigatran 110 mg group and bleeding in the dabigatran 150 mg group. RF and XGBoost were the best ML models in our analysis (Table 3).
Among ML techniques, decision tree models are more able to demonstrate clear rules. Like in most clinical settings, this method recursively partitions the data into subsets based on the values of the input features. The final work generates a tree-like model that can be used to make predictions by traversing the tree from the root to a leaf node, and the decision rules are the conditions at each node that determine which branch to follow. This allows the establishment of personalized suggestions and protocols to achieve precision medicine. CART is a popular tool for creating tree-shaped medical rules with realistic visualization and interpretability. It is a classifier and decision tree algorithm, but it showed inferior prediction performance in the second step of our study. We improved its AUC value by selecting fewer important variables with our integrated ML feature selecting scheme. Thus, we obtained explainable tree rules with acceptable predictive values. Figure 5 shows the decision rules for predicting vascular events, with five layers of nodes and 11 leaves. Figure 6 shows the decision rules for predicting bleeding, and it had five layers of nodes and 16 leaves. According to these suggestive medical rules based on important variables, clinicians may have a more detailed view and assistance in making decisions.
In the tree-shaped rule, the most important variables may be used to classify patients into high-risk and low-risk groups. We may notice that it is challenging to predict patients who may be free from vascular events in the next year when taking dabigatran 110 mg. Most rules indicating freedom from events demonstrated accuracy of 60-90%. However, we may surmise that 110 mg dabigatran was a relatively acceptable dose in patients without a history of MI, CHF, and kidney function abnormality. The risk of stroke and vascular events in patients with NVAF is determined by coexisting systemic diseases and age. The annual stroke risk was approximately 2-3% in low-risk patients based on the CHA 2 DS 2 -VASc score. In many national cohort studies, most clinicians prescribed off-label reduced doses of NOACs in consideration of age, body weight, kidney function abnormality, and medical history. Our research offers another viewpoint that may help in the prescription of reduceddose NOACs to patients with a low risk of vascular events. However, more critically, the decision tree demonstrated that patients with certain variables had an extremely high risk of vascular events while taking 110 mg dabigatran twice daily.
MI and CHF are important comorbidities and risk factors in NVAF patients, which greatly influence stroke and vascular death. This might be the result of different vascular risk factors, such as hypertension, DM, smoking, and dyslipidemia. In our tree model, NVAF patients with a history of MI and CHF had a higher risk of vascular events when taking dabigatran 110 mg twice daily. Furthermore, they had a greater risk of vascular events when they had low BMI < 18.5 kg/m 2 or kidney function abnormality (EGRF < 50 mL/min/1.73 m 2 ). For those without a history of MI, a combination of CHF, history of DM, and kidney function abnormality was associated with a high risk of vascular events when on dabigatran 110 mg, especially in underweight persons (BMI < 18.5 kg/m 2 ). Thus, applying intensive medical control and a standard dose of dabigatran 150 mg, if tolerable, is a more reasonable choice for clinicians.
When we consider safety with dabigatran 150 mg, Figure 6 shows the rules related to the high risk of bleeding. The concomitant use of certain drugs and older age (age ≥75 years) were the top factors that increased the risk of bleeding, especially when patients were underweight and smoker. For patients aged 65-74 years, dabigatran 150 mg might be risky if there is the concomitant use of certain drugs and kidney function abnormality (EGRF < 30 mL/min/1.73 m 2 ). The combination of kidney function abnormality (EGRF < 30 mL/min/1.73 m 2 ) and smoking habits also increased bleeding risk, regardless of BMI. Otherwise, the rule also indicated that elderly patients (age ≥75 years) with smoking habits and underweight stature tended to bleed when taking 150 mg dabigatran. Adjusting the dose of dabigatran may be considered in these patients.
The primary prevention of stroke in NVAF patients has been a prevalent issue in recent years due to the progression of NOACs. Intensive risk-factor control and the early application of anticoagulants in patients prone to vascular events are emphasized. This study provides an integrated multi-step ML scheme to assist in selecting the appropriate dose of dabigatran. In general, it is crucial to first determine if the patients have a history of MI, CHF, or kidney function abnormality with regard to the risk of vascular events, and if there is the contaminant use of drugs, kidney function abnormality, older age, and smoking habit with regard to the risk of bleeding. To confirm the most appropriate dose of dabigatran for the patient, clinicians may check the detailed classification information in our tree model.
Our model might be extended to more medical issues with its good prediction ability and convenience. However, dose adjustments and patient selection guidelines may take time before confirmation according to cohort studies and expert opinion. Our integrated ML scheme with adequate detailed data should provide clinicians with personalized decision support and recommendations. Moreover, ML techniques have the advantage of remodeling and may improve their prediction efficacy with new dataset. In this way, experts may reinspect and utilize our model to construct medical guidelines.

Limitation
There are some limitations to this study. First, our study was a post-hoc analysis of the RE-LY trial and excluded patients with recent ischemic stroke, and the participants' CHA 2 DS 2 -VASc scores were 2.1 ± 1.1, which might be different from the real-world situation. Our results might be inappropriate for patients in the acute stage of stroke or other extremely high-risk conditions. Second, we conducted a subgroup analysis to evaluate bleeding and vascular event risk. This procedure decreased the dataset size, and we did not perform CV to confirm the influence of different choices. However, the prediction performance value remained consistent in the two subgroups, and we performed 10-fold nested CV to enhance stability and avoid overfitting. The application of anticoagulant agents in NVAF patients with risk of vascular events and the choice of NOACs as first-line drugs have been the consensus management modality. We only focused on the dose selection for dabigatran but did not compare the doses with warfarin. However, the limited dataset and lack of external validation with real-world information restrict the consistency of our decision tree rules. Further remodeling our model with broad information is indicated before the utilization.

Conclusions
This study used an integrated ML-based decision tree scheme to predict vascular events and bleeding when NVAF patients took different doses of dabigatran. According to the integrated ML feature-selection tool, we utilized the performance of the CART method, which should provide clinicians with detailed and interpretable information. Our treeshaped rules should demonstrate the interaction between factors and the most risk-prone combinations of variables. Appropriate dose selection of NOACs for balancing benefits and side effects is an art in medicine. Our results may help clinicians achieve the optimal treatment for NVAF with different characteristics.

Informed Consent Statement:
The requirement for informed consent was waived, since the data contain only de-identified information.
Data Availability Statement: The data are available through application to Boehringer-Ingelheim on the research data sharing platform (https://vivli.org/; accessed on 28 July 2021). Restrictions apply to the availability of these data, which were used under license for this study. Analysis codes are available on request to the corresponding author due to privacy/ethical restrictions.