A Novel Prediction Tool for Endoscopic Intervention in Patients with Acute Upper Gastro-Intestinal Bleeding

(1) Background: Predicting which patients with upper gastro-intestinal bleeding (UGIB) will receive intervention during urgent endoscopy can allow for better triaging and resource utilization but remains sub-optimal. Using machine learning modelling we aimed to devise an improved endoscopic intervention predicting tool. (2) Methods: A retrospective cohort study of adult patients diagnosed with UGIB between 2012–2018 who underwent esophagogastroduodenoscopy (EGD) during hospitalization. We assessed the correlation between various parameters with endoscopic intervention and examined the prediction performance of the Glasgow-Blatchford score (GBS) and the pre-endoscopic Rockall score for endoscopic intervention. We also trained and tested a new machine learning-based model for the prediction of endoscopic intervention. (3) Results: A total of 883 patients were included. Risk factors for endoscopic intervention included cirrhosis (9.0% vs. 3.8%, p = 0.01), syncope at presentation (19.3% vs. 5.4%, p < 0.01), early EGD (6.8 h vs. 17.0 h, p < 0.01), pre-endoscopic administration of tranexamic acid (TXA) (43.4% vs. 31.0%, p < 0.01) and erythromycin (17.2% vs. 5.6%, p < 0.01). Higher GBS (11 vs. 9, p < 0.01) and pre-endoscopy Rockall score (4.7 vs. 4.1, p < 0.01) were significantly associated with endoscopic intervention; however, the predictive performance of the scores was low (AUC of 0.54, and 0.56, respectively). A combined machine learning-developed model demonstrated improved predictive ability (AUC 0.68) using parameters not included in standard GBS. (4) Conclusions: The GBS and pre-endoscopic Rockall score performed poorly in endoscopic intervention prediction. An improved predictive tool has been proposed here. Further studies are needed to examine if predicting this important triaging decision can be further optimized.


Introduction
Acute upper gastro-intestinal bleeding (UGIB) is a common and urgent medical condition, usually requiring hospital admission with reported annual incidences in the range of 48 to 172 per 100,000 [1]. Current guidelines recommend that following hemodynamic resuscitation, patients with UGIB should undergo an esophagogastroduodenoscopy (EGD) within 12-24 h of presentation [2,3]. Endoscopic therapy in the setting of acute UGIB is indicated when recent bleeding stigmata is observed via EGD, leading to a reduction in the rates of mortality, recurrent bleeding, and surgical intervention [4][5][6]. Several guidelines support utilization of pre-endoscopic assessment scores to stratify patients to low 2 of 11 and high-risk groups [2,7]. The Glasgow-Blatchford score (GBS) is a risk assessment tool, synthesizing clinical and laboratory parameters to identify the likelihood of a patient to require medical intervention such as endoscopy, blood transfusion or surgery [8]. The pre-endoscopic Rockall score is an assessment tool which was designed to predict mortality among UGIB patients [9]. In 2017, a prospective study by Stanley and colleagues suggested that only the GBS, and not the other accepted scores (i.e., Rockall and AIM 65), is a reliable score for a composite outcome of transfusion, hemostatic intervention, or death. However, when assessing the performance of the scores for an endoscopic intervention solely, the area under the receiver operating characteristic curve (AUROC) is less than 0.80 for each of the scores, including the GBS [10]. Hence, the clinical utility of the current scores for this crucial outcome is limited. Prediction of endoscopic therapy among acute UGIB patients is complex and has significant implications in patient management. The purpose of this study was to identify parameters in correlation with endoscopic intervention and to create a dedicated predictive model for endoscopic intervention exclusively, in patients with acute UGIB. The new model is based on the GBS and other parameters found significantly associated with endoscopic therapy.

Study Design and Patient Selection
This was a retrospective study including consecutive adult patients who were hospitalized at the Sheba medical center due to acute UGIB, melena or hematemesis between 2012-2018, and had an EGD during their respective hospitalization. Data were collected from an electronic repository including demographic, hemodynamic and laboratory variables at presentation, necessary to calculate GBS and pre-endoscopic Rockall score. Other clinical variables, laboratory results, medical history, free-text physician records and endoscopy reports were collected as well.

Data Extraction
The following data were collected from the electronic health record of the Sheba Medical Center: -Demographic factors-age in years and sex. -Comorbidities-including hypertension (HTN), diabetes mellitus (DM), cardio-vascular disease (i.e., ischemic heart disease (IHD), arrythmias, valvular disorder, stroke), pulmonary disease (i.e., asthma, chronic obstructive pulmonary disease (COPD)), deep vein thrombosis (DVT), pulmonary embolism (PE) and cirrhosis. -Chronic treatment of anticoagulants and anti-platelets-including aspirin, P2Y 12 inhibitors (clopidogrel, ticagrelor, prasugrel), warfarin and direct oral anticoagulants (DOACs, i.e., dabigatran, rivaroxaban, apixaban). -Hemodynamic, laboratory variables-all necessary variables at presentation for GBS calculation were collected. Other laboratory results including C-reactive protein (CRP), white blood count (WBC), platelets count, international normalization ratio (INR) and albumin were collected as well. -Medications-date regarding blood transfusion during hospital stay and treatment with tranexamic acid (TXA, i.e., Hexakapron), proton pump inhibitors (PPI), erythromycin and intravenous (IV) fluids prior to endoscopy were collected. -Endoscopic data-data regarding endoscopic diagnosis and endoscopic intervention were collected. Endoscopic intervention was defined as the use of at least one or a combination of the following interventions-band ligation, adrenaline injection, hemoclip application and/or coaptive coagulation. All EGD were performed by a trained gastroenterologist or physicians in their GI training under the supervision of a senior gastroenterologist.

Study Aim
The main study goal was to assess the possible correlation between baseline parameters and endoscopic intervention. Furthermore, we assessed the ability of the GBS and the pre-endoscopic Rockall score to predict endoscopic intervention and to create a dedicated model for intervention prediction. The main study outcome was endoscopic intervention as defined in the data extraction section.

Data Analysis
Random forest models were trained to predict the two study outcomes. Data preprocessing included median imputation of missing values. Threefold cross validation splits were employed in each experiment. The testing-folds results were averaged to achieve pooled metrics. Single feature analysis was used to establish the optimal features to be used in the models. The final models' metrics included area under the receiver operator curve (AUC), sensitivity, specificity, PPV, NPV, and F1. For the explainability of variables in the random forest models, we used feature importance and Shapley additive explanations (SHAP) values. Machine learning programming was carried out with Python (Version 3.7, 64 bits).

Statistical Analysis
Categorical variables were presented as frequency and percentage and as medians and intra-quartile ranges (IQR) for continuous variables. Statistical significance for comparison of continuous variables was evaluated using the Student's t-test/Kruskal-Wallis test and Chi-square test/Fisher's exact test for categorical variables. Pearson coefficient evaluated linear correlation between the features in the models. All statistical tests were two-sided, and a p value < 0.05 was considered statistically significant. Statistical analysis was performed using with Python (Version 3.7 64 bits).

Study Ethics and Patient Consent
This study was carried out in accordance with the ethical guidelines of the Declaration of Helsinki. Since this was a retrospective analysis, no informed consent was obtained.

Results
This section may be divided by subheadings. It should provide a concise and precise description of the experimental results, their interpretation, as well as the experimental conclusions that can be drawn.

Patient Characteristics
A total of 883 patients were included in this study. A total of 52 (62.5%) were male, the median patient's age was 69.0 (IQR 58.0-79.0). A total of 579 patients (65.5%) had endoscopic intervention and/or blood transfusion. Endoscopic intervention without blood transfusion was performed in 145 (16.4%) patients and 434 patients were treated with blood transfusion with no endoscopic intervention. The etiology of bleeding for which endoscopic treatment was performed are presented in supplementary Table S1. In-hospital mortality rate was 4.3% (n = 38). All patients underwent EGD within 120 h from admission. The median GBS was 9.0 (IQR 6.0-12.0) and median time to endoscopy was 16 h (IQR 5.7-24.03). All demographic and clinical data of endoscopic intervention and in-hospital treatment are depicted in Table 1.

Background Diagnosis and Clinical Features and Endoscopy Timing
Several parameters correlated with increased risk for endoscopic intervention as presented in Table 1. Age, gender, and comorbidities except cirrhosis, did not correlate with endoscopic intervention. Cirrhosis (9.0% vs. 3.8%, p = 0.01) and syncope at presentation (19.3% vs. 5.4%, p < 0.01) were found to correlate with a higher rate of endoscopic intervention. Higher GBS (11 vs. 9, p < 0.01) and higher pre-endoscopic Rockall score (4.7 vs. 4.1, p < 0.01) were significantly associated with endoscopic therapy. Patients who underwent EGD earlier were more likely to be treated endoscopically (median 6

GBS and Pre-Endoscopic Rockall Score Performance for Prediction of Endoscopic Intervention
GBS and pre-endoscopic Rockall score performance in predication of endoscopic intervention was examined. AUC of 0.54 for GBS and 0.56 for pre-endoscopic Rockall score were demonstrated ( Table 2). When assessing the scores' discriminative ability for the composite outcome of endoscopic intervention and packed blood cell transfusion, the GBS was found to be superior to the pre-endoscopic Rockall score (AUC of 0.70 vs. 0.56, respectively, Table 3).  Evaluating the GBS by score groups showed an obvious correlation with the composite outcome of endoscopic intervention and red blood cells (RBCs) transfusion; this correlation was almost completely lost when using the GBS for endoscopic intervention alone. However, the pre-endoscopic Rockall score validity was partially kept when evaluated by score groups for both the composite outcome and endoscopic intervention alone (Table 3, Figure 1a-d).
Evaluating the GBS by score groups showed an obvious correlation with the composite outcome of endoscopic intervention and red blood cells (RBCs) transfusion; this correlation was almost completely lost when using the GBS for endoscopic intervention alone. However, the pre-endoscopic Rockall score validity was partially kept when evaluated by score groups for both the composite outcome and endoscopic intervention alone (Table 3, Figure 1a-d).

Predictive Model for Endoscopic Intervention
The mean AUC of the new model for endoscopic intervention was 0.68 (Table 2, Figure 2a). When assessing the feature importance of all variables included in the model for endoscopic intervention, syncope at presentation, GBS and erythromycin treatment were found to score the most important (Figure 3a). The SHAP importance plot of the model for endoscopic intervention shows variable findings regarding the GBS (i.e., there is no clear correlation between the score and the model prediction) (Figure 3b). The plot does show clear evidence that syncope, cirrhosis and erythromycin use are correlated positively with the risk of intervention and interestingly that DOAC use is negatively correlated with that risk.

Predictive Model for Endoscopic Intervention
The mean AUC of the new model for endoscopic intervention was 0.68 (Table 2, Figure 2a). When assessing the feature importance of all variables included in the model for endoscopic intervention, syncope at presentation, GBS and erythromycin treatment were found to score the most important (Figure 3a). The SHAP importance plot of the model for endoscopic intervention shows variable findings regarding the GBS (i.e., there is no clear correlation between the score and the model prediction) (Figure 3b). The plot does show clear evidence that syncope, cirrhosis and erythromycin use are correlated positively with the risk of intervention and interestingly that DOAC use is negatively correlated with that risk.

Predictive Model for Endoscopic Intervention and Blood Transfusion
A random forest model for the composite outcome of endoscopic intervention and RBCs transfusion was used. Figure 2b shows the receiver operating characteristic (ROC) curves of the validation folds of the model and mean, for the composite outcome. The mean AUC for the composite outcome was 0.86 (Table 3)

Predictive Model for Endoscopic Intervention and Blood Transfusion
A random forest model for the composite outcome of endoscopic intervention and RBCs transfusion was used. Figure 2b shows the receiver operating characteristic (ROC) curves of the validation folds of the model and mean, for the composite outcome. The mean AUC for the composite outcome was 0.86 (Table 3) compared with 0.70 of the GBS. Feature importance analysis of the model for the composite outcome showed hemoglobin level to have the greatest influence on the model with more than threefold the weight of the next feature.

Discussion
Acute upper GI bleeding still poses a burden on healthcare systems around the world [11,12]. Several scores have been developed to predict clinically relevant outcomes [8,9,13]. However, since these scores were originally published, major changes occurred

Predictive Model for Endoscopic Intervention and Blood Transfusion
A random forest model for the composite outcome of endoscopic intervention and RBCs transfusion was used. Figure 2b shows the receiver operating characteristic (ROC) curves of the validation folds of the model and mean, for the composite outcome. The mean AUC for the composite outcome was 0.86 (Table 3) compared with 0.70 of the GBS. Feature importance analysis of the model for the composite outcome showed hemoglobin level to have the greatest influence on the model with more than threefold the weight of the next feature.

Discussion
Acute upper GI bleeding still poses a burden on healthcare systems around the world [11,12]. Several scores have been developed to predict clinically relevant outcomes [8,9,13]. However, since these scores were originally published, major changes occurred

Discussion
Acute upper GI bleeding still poses a burden on healthcare systems around the world [11,12]. Several scores have been developed to predict clinically relevant outcomes [8,9,13]. However, since these scores were originally published, major changes occurred in the incidence of UGIB, features of patients, management, outcomes [14], and there are different characteristics in UGIB cases worldwide [14][15][16]. Moreover, none of these scores was designed to predict the likelihood of endoscopic intervention exclusively, which is a significant part of the management of UGIB, as previous studies demonstrated correlation between endoscopic intervention and reduced morbidity and mortality [6,17].
The aim of this study was to assess the possible correlation between baseline parameters and endoscopic intervention and to evaluate the performance of the GBS and the pre-endoscopic Rockall score in predicting endoscopic intervention.
The GBS, first published in 2000, is a well-established score used to define patients with UGIB who may be managed safely as outpatients 8 . The score was originally designed to predict a composite outcome including the risk of a blood transfusion, intervention to control bleeding, rebleeding, or death [8]. However, the decision to administer a blood transfusion is based on clinical evaluation and hemoglobin levels. Therefore, the value of predicting this decision is negligible in comparison to the need to predict endoscopic intervention. Testing the predictive performance of the GBS in our study, for endoscopic intervention alone, and for a composite outcome including blood transfusion, we found it to be low.
The pre-endoscopic Rockall is another well-established score, designed for mortality risk assessment only [9]. In our study, like several studies before [10,18], the ability of the pre-endoscopic Rockall score, like the GBS, to predict endoscopic therapy exclusively, is modest.
In a univariate analysis we found the history of cirrhosis, syncope at presentation, pre-endoscopic erythromycin and TXA treatment, and earlier endoscopy time to correlate significantly with endoscopic intervention.
Erythromycin is a macrolide antibiotic with prokinetic activity. In 2016, a meta-analysis by Rahman et al. demonstrated that erythromycin infusion prior to upper endoscopy significantly improved gastric mucosa visualization and reduced the need for a "second-look" endoscopy. However, correlation between erythromycin administration and endoscopic intervention was not assessed [19].
The American College of Gastroenterology recommends the use of erythromycin prior to endoscopy [20]. While acknowledging lack of evidence for erythromycin benefit in reducing further bleeding and mortality, it does provide meaningful reductions in repeat endoscopies and length of hospitalization. Considering its relatively low cost and ease of administration, the panel published a conditional recommendation for its use [20]. On the other hand, the European Society of Gastrointestinal Endoscopy (ESGE) recommends administration of erythromycin only in select patients with severe or ongoing bleeding [2].
In our study, only a minority of patients with UGIB were treated with erythromycin prior to endoscopy, yet a significant correlation was demonstrated between its use and endoscopic therapy. This is probably due to its prokinetic qualities, expelling blood clots distally out of the stomach and proximal duodenum, rendering them clearer to careful visualization and respective endoscopic intervention. On the other hand, its use may be interpreted as a marker for more severe patients, evaluated by the treating physician to have more significant bleeding, and hence treated with erythromycin, to improve the endoscopy outcomes. In both cases, the strong positive correlation herein found reinforces the importance of utilizing this medication among patients with UGIB before endoscopy is performed.
TXA is an antifibrinolytic agent, widely used for several indications, including postpartum hemorrhage, menorrhagia, trauma-associated hemorrhage and surgical bleeding [21]. However, previous studies demonstrated that the efficacy of TXA in patients with upper gastrointestinal bleeding is poor and carries a risk for thromboembolic events [22]. One third of the patients in this study have been treated with TXA and a significant positive correlation between its use and endoscopic intervention was demonstrated. We might hypothesize that bleeding cessation affords better visualization and higher rates of endoscopic intervention. Alternatively, bleeding cessation affords better hemodynamics, for continuation and prolongation of the upper endoscopy, creating a better opportunity to locate and treat a bleeding site. Nevertheless, mortality and other clinical outcomes including thromboembolic events were not assessed in our cohort, hence we cannot recommend its use, based upon our data.
We observed a negative trend between DOAC's treatment and endoscopic intervention. It can be assumed that although patients receiving this class of anticoagulants tend to bleed, they suffer from bleeding sources that often do not require endoscopic intervention, as has already been reported for dabigatran, which may cause a longitudinal esophageal mucosal injury in approximately 20% of patients [23]. In addition, it can be assumed that due to low availability of an antidote during the study years, a concern to perform endoscopic intervention in these patients led to the avoidance of endoscopic intervention.
The GBS and the pre-endoscopic Rockall score performances in prediction of endoscopic intervention in this tertiary center-based cohort were found to be low. Aiming to build a better prediction tool, random forest models were trained and validated. The performance of the new model was better than the performance of the GBS and the preendoscopic Rockall score (AUC of 0.68 vs. 0.54 and 0.56, respectively), yet far from perfect. In evaluating the reasons for the limited performance of the new model, its relative low sensitivity of 0.54 stands out. The specificity of the model was reasonable, much better than the GBS (0.71 vs. 0.28), but certainly not optimal and less than the pre-endoscopic Rockall score with specificity of 0.88. It should be considered that in a predictive model of this kind, where intervention and non-intervention have major risks associated, both the sensitivity and specificity are important for the potential physician using the tool, and so both should be improved in order for this tool to become practical, as should be also represented by a higher AUC.
It should be mentioned that the mortality rate and severe morbidity such as cirrhosis was low in our cohort (4.3% and 4.6% in accordance) compared to data from previous studies worldwide [24][25][26][27]. These differences, indicating different morbidity among different populations may explain the low level of accuracy relative to past studies of the models tested.
Our study, like any study designed to predict endoscopic intervention, has the inherent limitation of an outcome which is subjected to the endoscopist consideration. It is obvious that different physicians may decide differently when encountering similar lesions, based on their training and experience. Their decision may also be affected by the conventions in their units and by the clinical setup, such as timing of procedure, and the assisting nursing team. It should be mentioned that at the Sheba Medical Center, the endoscopist does not receive a fee for endoscopic intervention and hence this factor cannot influence the discretion of the performing endoscopist. This subjective aspect of the outcome limits the generalizability of the study, like any other study in this area. It also limits the potential prediction performance of the model.

Conclusions
Prediction of endoscopic intervention in UGIB is complex. In this study, we have demonstrated several parameters that significantly correlate with endoscopic intervention. The GBS and pre-endoscopic Rockall score performed poorly in endoscopic intervention prediction, compared with previous studies, which may reflect differences between populations. An improved model has been proposed here; however, its accuracy for prediction of endoscopic intervention was modest. Further research is required to improve this model's performance, and to examine it in a prospective manner, to make it practical for use in clinical settings. Optional means to improve this model are by expanding the cohort used for the random forest models training, include previous endoscopic evaluation and interventions, include non-invasive imaging raw data using image recognition methods, and possibly incorporating natural language processing tools to analyze free text from the electronic medical record. There is a reasonable chance that using all these methods together will improve the model, and provide a better prediction performance in future versions of it.