Feature Importance of Acute Rejection among Black Kidney Transplant Recipients by Utilizing Random Forest Analysis: An Analysis of the UNOS Database

Background: Black kidney transplant recipients have worse allograft outcomes compared to White recipients. The feature importance and feature interaction network analysis framework of machine learning random forest (RF) analysis may provide an understanding of RF structures to design strategies to prevent acute rejection among Black recipients. Methods: We conducted tree-based RF feature importance of Black kidney transplant recipients in United States from 2015 to 2019 in the UNOS database using the number of nodes, accuracy decrease, gini decrease, times_a_root, p value, and mean minimal depth. Feature interaction analysis was also performed to evaluate the most frequent occurrences in the RF classification run between correlated and uncorrelated pairs. Results: A total of 22,687 Black kidney transplant recipients were eligible for analysis. Of these, 1330 (6%) had acute rejection within 1 year after kidney transplant. Important variables in the RF models for acute rejection among Black kidney transplant recipients included recipient age, ESKD etiology, PRA, cold ischemia time, donor age, HLA DR mismatch, BMI, serum albumin, degree of HLA mismatch, education level, and dialysis duration. The three most frequent interactions consisted of two numerical variables, including recipient age:donor age, recipient age:serum albumin, and recipient age:BMI, respectively. Conclusions: The application of tree-based RF feature importance and feature interaction network analysis framework identified recipient age, ESKD etiology, PRA, cold ischemia time, donor age, HLA DR mismatch, BMI, serum albumin, degree of HLA mismatch, education level, and dialysis duration as important variables in the RF models for acute rejection among Black kidney transplant recipients in the United States.

Recent investigations have demonstrated that machine learning approaches are superior to traditional statistical methods in various clinical scenarios [29,30]. Random forest (RF) is a widely used machine learning approach that effectively predicts outcomes [31] by utilizing a combination of tree predictors [32]. The RF algorithm randomly generates bootstrapped datasets that can be used to train an ensemble of decision trees, which determine an outcome by a majority "vote" [32]. As a type of robust nonparametric model, RF can simulate complex relationships and does not depend on the data distribution as is the case in logistic regression [31]. Whereas most traditional statistical approaches, such as linear regression and logistic regression, indicate which variables are significant with measures such as p-value and t-statistics, variable importance by RF is determined by how much each variable decreases the node impurity (gini decrease), number of nodes, accuracy, mean minimal depth, and times_a_root (total number of trees in which X j is used for splitting the root node) [31,33]. Recently, RF has increasingly been applied to medicine, including solid organ transplantation [34][35][36], and there is great potential to use the RF approach to improve outcomes among Black kidney transplant recipients. Furthermore, the feature interaction network analysis framework of RF may also provide an understanding of the interaction among multiple features in order to design strategies to prevent acute rejection and explore the mechanisms of variable interactions influencing acute rejection among Black kidney transplant recipients [37].
In this study of the UNOS/OPTN database from 2015 through 2019, we aimed to assess the risk factors and feature importance of acute rejection among Black kidney transplant recipients by utilizing RF vs. traditional multivariable logistic regression analysis.

Data Source and Study Population
The Organ Procurement and Transplantation Network (OPTN)/United Network for Organ Sharing database (UNOS) database was used for analysis of this study. The OPTN/UNOS database contains patient-level data of all transplant events in the United States. All adult (age ≥18 years) end-stage kidney disease patients who received kidneyonly kidney transplants from 2015 to 2019 were screened. Only Black patients were included in this study. If patients had multiple kidney transplants during the study period, the first kidney transplant was selected for analysis. This study was approved by the Mayo Clinic Institutional Review Board (#21-007698) and the UNOS/OPTN data is publicly available and de-identified.

Data Collection
Comprehensive recipient-, donor-, and transplant-related variables in the OPTN/UNOS database were extracted. All the extracted variables had less than 10% of missing data.
Missing data was imputed through multivariable imputation by the chained equation (MICE) method [38].
The primary outcome was acute rejection reported by transplant centers to the OPTN/UNOS within 1 year after kidney transplant. The UPTN/UNOS database did not specify the date of acute rejection occurrence.

Machine Learning Variable Importance Analysis
Variable importance was performed using the "randomForest" package and interpreted and visualized by the "randomForestExplainer" [39] packages in R 4.0.2. Random forests are ensemble classifiers that aggregate the results of many individual decision trees. We used the 'randomForest' R package [40] with two hyperparameters: the number of training trees (nTree) and the number of predictors to consider at each split point (mTry). The default settings of nTree = 500 and mTry as the square root of the number of predictor variables were used in this study. To avoid the bias of analysis of variable importance, various indicators (number of nodes, accuracy decrease, Gini decrease, times_a_root (total number of trees in which X j is used for splitting the root node), p value, and mean minimal depth) were selected to represent different perspectives and to comprehensively evaluate the importance of features [31,33]. The Gini impurity measures the frequency at which any element of the dataset will be mislabeled when it is randomly labeled. The minimum value of the Gini Index is 0. This happens when the node is pure, indicating that all the contained elements in the node are of one unique class. "Gini_decrease" indicates the decrease in the Gini impurity index, and "accuracy_decrease" refers to the mean decrease of prediction accuracy after the corresponding predictor was permuted [39].
The importance of each variable can be expressed using other metrics, such as mean minimal depth, times_a_root, accuracy decrease, and Gini decrease.

Statistical Analysis
Continuous variables were presented as mean ± standard deviation (SD) for normally distributed data, or median with interquartile range (IQR) for non-normally distributed data. Categorical variables were presented as number with percentage. The difference in clinical characteristics between patients with and without rejection were tested using the student's t-test or Wilcoxon's rank sum test as appropriate for continuous variables, and Chi-squared test for categorical variables. For traditional analysis to identify independent predictors for rejection, backward stepwise multivariable logistic regression with inclusion of variables whose p-value in univariable analysis <0.05 was performed.

Results
A total of 22,687 black kidney transplant recipients were eligible for analysis. Of these, 1330 (6%) had acute rejection within 1 year after kidney transplant. Table 1 compared the recipient-, donor-, and transplant-related characteristics between patients with and without rejection. Patients with rejection were younger, more likely to have a glomerular kidney disease etiology, have longer dialysis duration, and be HIV-seropositive. They were less likely to be diabetic or receive a living donor kidney transplant and more likely to have delayed graft function. They were also more likely to be kidney retransplants, have a higher PRA, have a higher total number of HLA mismatches, and carry public insurance. With regard to immunosuppression, patients with rejection were less likely to receive depleting induction (e.g., thymoglobulin, alemtuzumab) and were more likely to receive basiliximab. They were also more likely to be on cyclosporine, mycophenolate, azathioprine, and mTOR inhibitors for maintenance immunosuppression.  (7) 82 (6) 1470 (7) HLA mismatch

Traditional Analysis
The multi-collinearity of continuous variables was assessed by a correlation matrix, which demonstrated no significant multi-collinearity ( Figure 1). Using traditional analysis with backward stepwise multivariable logistic regression (Table 2), the independent predictors for increased acute rejection risk included kidney retransplantion, dialysis duration ≥1 years, a PRA of 81-100, HIV infection, ECD deceased donor utilization, a higher total number of HLA mismatches, delayed graft function, basiliximab induction, and the use of cyclosporine, azathioprine, and mTOR inhibitors for maintenance immunosuppression. In contrast, the independent predictors for decreased rejection risk included older recipient age and the use of thymoglobulin and alemtuzumab for induction.  Figure 2 demonstrates variables based on the minimum depth locations between the tree and the number of trees. The minimal depth for a variable in a tree is equal to the depth of the node which splits on that variable and is the closest to the root of the tree. If it is low, then a number of observations are divided into groups on the basis of this variable. From the top 10 variables with the smallest mean value of minimal depth plotted in Figure 2, recipient age, cold ischemia time, BMI, PRA, and serum albumin are the top 5 variables used to split trees at the root. The RF model built 500 trees with no limit to the maximum number of terminal nodes in a tree. It is evident that trees were split until a depth of 11.

Multi-way Importance Plot
The multi-way importance plot reveals the relation between the 3 measures of importance and labels 10 variables that scored best when it comes to these 3 measures.
The first multi-way importance plot ( Figure 3) centers on three important measures acquired from the structure of trees in the forest, including (1) mean depth of the first split on the variable, (2) number of trees in which the root is split on the variable, and (3) the total number of nodes in the forest that split on that variable. These top 10 relative variables of importance in the RF models for acute rejection are based on the minimum average depth and the number of nodes, and the times to root include age, cause of ESKD, PRA, education level, cold ischemia time, HLA DR mismatch, BMI, serum albumin, degree of HLA mismatch, and donor age. The second multi-way importance plot (Figure 4) reveals the important measures that emerge from the role of the variables in the prediction of acute rejection, including a decrease in accuracy and a decrease in Gini, with additional information on the p-value based binomial distribution of the number of nodes split on the variable implying that the variables are randomly drawn to form splits. After combining the mean decrease in Gini, decrease in accuracy, and p values of these features, the top variables for acute rejection include cold ischemia time, recipient age, donor age, BMI, serum albumin, PRA, degree of HLA mismatch, causes of ESKD, dialysis duration, and HLA-DR mismatch.  Figure 5 exhibits the bilateral relations between the rankings of variables according to the selected importance measures. It demonstrates that the RF parameters of importance are ascertained to have correlations among each other, thereby implying the reliability of each of these parameters to rank the variable importance. The top correlations among themeasures of importance include decrease in gini:number of nodes, decrease in gini:mean minimal depth, and mean minimal depth: number of nodes.

Variable Interactions
Feature interactions with the most frequent occurrences in the RF classification run between correlated and uncorrelated pairs. Figure 6 outlines the 30 top interactions of the variables according to the mean of conditional minimal depth, a generalization of minimal depth that measures the depth of the second variable in a tree of which the first variable is a root (a subtree of a tree from the forest).
To be comparable to the normal minimal depth, 1 is subtracted so that 0 is the minimum. Smaller values of the mean conditional depth with associated higher unconditional depth, as well as increased occurrences, indicate interaction effects ( Table 4). The interactions considered are those with the following variables as first (root variables): cold ischemia time, recipient age, donor age, BMI, PRA, cause of ESKD, serum albumin, total number of HLA mismatches, dialysis duration, HLA-DR mismatches, allocation type, education level, CMV status, HLA-B mismatch, HLA-A mismatch, and all plausible values of the second variable. The three most frequent interactions consist of two variables, including recipient age:donor age, recipient age:serum albumin, and recipient age:BMI, respectively.

Discussion
In this study, using tree-based RF feature importance and the feature interaction network analysis framework, we were able to demonstrate important variables in the RF models for acute rejection among Black kidney transplant recipients using the number of nodes, accuracy decrease, gini decrease, times_a_root, p value, and mean minimal depth. These identified risk factors for rejection included recipient age, cause of ESKD, PRA, cold ischemia time, donor age, HLA DR mismatch, BMI, serum albumin, degree of HLA mismatch, education level, and dialysis duration.
By comparison, traditional multivariable logistic regression analysis showed that younger recipient age; kidney retransplantation; ECD deceased donor utilization; dialysis duration; PRA; recipient HIV seropositivity; degree of HLA mismatch; DGF; basiliximab induction; and cyclosporine, azathioprine, and mTOR inhibitor-based immunosuppression were independent risk factors for acute rejection among Black kidney transplant recipients. Whereas some important variables from traditional logistic regression analysis are not listed as important variables in the RF approach, these factors are still variables that were used to predict acute rejection outcomes among Black kidney transplant recipients (as shown in Figure 7). RF is an ensemble of decision trees. Many trees produced in a particular "random" way build a RF [41]. Each tree is constructed from a diverse sample of rows, and at individual nodes, a different sample of features is chosen for splitting. Each of the trees produces its own individual prediction. These predictions are subsequently averaged to generate a single result. Averaging strengthens a RF to be better than a single decision tree and thereby increases its accuracy and lessens overfitting. The average of these models evens out the variance, resulting in an error reduction that is both low in bias and low in variance. This nonparametric and nonlinear machine learning RF method can resist noise and is expected to build accurate prediction models using aggregated data. In addition, RF works well on large datasets, especially when there are many categorical independent variables and unbalanced data [41], as in our OPTN/UNOS dataset. Conversely, a logistic regression analysis approach, which uses a generalized linear equation and the stepwise variable selection method, is based on the likelihood ratio test to describe the directed dependencies among a set of variables. To do so, a number of statistical assumptions must be met. Common concerns include overfitting (rule of 10) as well as outliers. As a result, logistic regression inherently has bias and low variance due to the rigid nature of the shape of the line.
Previous studies have demonstrated higher PRA, longer cold ischemia time, increased HLA mismatches, HLA DR mismatches, and longer dialysis duration as important generalizable risk factors for acute rejection among kidney transplant recipients. From our current study using RF, we demonstrate that some of these established variables, such as cold ischemia time and HLA DR mismatch, are not listed as independent predictors for acute rejection among Black kidney transplant recipients in traditional logistic regression analysis. Furthermore, we also found that BMI, cause of ESKD, serum albumin, donor age, and education level are important variables in RF, but not in traditional analysis. Given that the RF algorithm has increasingly been applied in medicine and transplantation [34][35][36], it is important to recognize these unique RF model variables for acute rejection among Black kidney transplant recipients. A good prediction model begins with a great feature selection process. Understanding these variables in our study using a national database will help each transplant center to develop their individualized RF model, prognosticate the risk of rejection among Black kidney transplant recipients, and develop strategies to prevent these serious events.
In addition to the identification of feature importance for acute rejection in Black kidney transplant recipients in the RF model, we also conducted feature interaction network analysis. A great benefit of the tree structure is the understanding of the interaction between variables. For example, if the split in a parent is by one variable and by another variable in the daughter node, it can be concluded that there is an interaction between these two variables [14,24,25]. Interactions also become apparent as common occurrences of variable combinations. Thus, both the pair frequency and the associated distances are informative with regard to the interaction effects. From the findings of our study, the three most frequent interactions consist of two numerical variables, including recipient age:donor age, recipient age:serum albumin, and recipient age:BMI, respectively. These findings from the feature interaction network analysis may help to determine the important effect modifiers of acute rejection risk among Black kidney transplant recipients.
Our study has several limitations. Given the nature of the UNOS database, we do not have details on the factors leading to acute rejection such as immunosuppression levels, medication non-adherence, donor-specific antibodies, or infection prior to episodes of acute rejection. Thus, we aimed to investigate RF feature importance with feature interaction network analysis as an initial step to create an RF prediction model. Additional investigations are needed to incorporate the findings of our study into other important variables such as crossmatch results, the presence of DSA, and follow-up data to construct a RF prediction model with a high predictive performance for acute rejection among Black kidney transplant recipients. In addition, future studies assessing tree-based RF feature importance and the feature interaction network analysis framework for acute rejection among the general kidney transplant recipient populations are needed.

Conclusions
In conclusion, the application of tree-based RF feature importance and the feature interaction network analysis framework identified the recipient age, ESKD etiology, PRA, cold ischemia time, donor age, HLA DR mismatch, BMI, serum albumin, degree of HLA mismatch, education level, and dialysis duration as important variables in the RF models for acute rejection among Black kidney transplant recipients in the United States.