External Validation of the ImAgeS Risk Score for Mortality in Hospitalized Kidney Transplant Recipients with COVID-19: A Retrospective Observational Study

: Background: Timely recognition of high-risk individuals with novel Coronavirus disease (COVID-19) is important. Yet, validated risk scores for kidney transplant recipients with COVID-19 are lacking. The present study aimed to externally validate the novel ImAgeS risk score in this population. Methods: A retrospective analysis of 65 kidney transplant recipients with COVID-19 was conducted. A robust external validation of the novel ImAgeS risk score with respect to 30-day all-cause mortality was performed using regression analysis, discrimination and calibration methods. Results: An overall mortality rate during the study follow-up was 18.5% (N = 12). The ImAgeS risk score showed a statistically signiﬁcant association with 30-day all-cause mortality (HR 1.04 95% CI 1.00–1.08, p = 0.040). This risk score demonstrated a modest, statistically signiﬁcant discrimination of all-cause mortality (AUC of 0.679 (95% CI 0.519–0.840, p = 0.027). The calibration of the model was acceptable with a Hosmer-Lemeshow value of 3.74, Harrell’s C concordance index of 0.699 and Somers’ D of 0.397. Conclusions: The ImAgeS risk score demonstrated a signiﬁcant association with 30-day all-cause mortality in kidney transplant recipients with COVID-19. The model showed modest discrimination and satisfactory calibration, conﬁrming the ﬁndings from the computational study. Further studies are needed to determine the utility of the ImAgeS score in this high-risk population.


Introduction
The novel Coronavirus disease (COVID-19) has had a major impact on global health, leading to high morbidity and mortality, particularly in patients with chronic diseases [1][2][3]. Solid organ transplant recipients, such as kidney transplant recipients, represent a delicate population with complex pathophysiologic interactions and dependence on routine healthcare [1][2][3][4]. The importance of timely recognition of high-risk individuals in this setting is emphasized to guide management and prevent adverse outcomes [4].
Previous studies have reported immense COVID-19-associated mortality in kidney transplant recipients [3][4][5][6][7][8][9] because of various underlying mechanisms [10][11][12][13][14][15][16][17], warranting measures to improve the clinical decision-making process. Risk stratification is the method of evaluating the risk factors and patient characteristics to guide their management. In the context of kidney transplant recipients with COVID-19, risk stratification is particularly important because these individuals are at increased risk of severe disease. Furthermore, this population is quite heterogenous warranting measures to recognize individuals with increased risk of adverse outcomes. Despite this, validated and specific risk scores for kidney transplant recipients with COVID-19 are sparsely available in the literature.
Recently, de Andrade et al. have developed a novel clinical risk score (ImAgeS) using the data from a Brazilian cohort of kidney transplant recipients with COVID-19. This score proved to have satisfactory discrimination and prediction of mortality [4], but external validation has been warranted. This study aimed to expand the existing evidence by externally validating a novel ImAgeS risk score with respect to 30-day mortality in kidney transplant recipients with COVID-19.

Study Design and Patients
This observational retrospective study included 65 eligible kidney transplant recipients who were treated for COVID-19 at the University Hospital of Split in the period from August 2020 to March 2022. All included patients underwent reverse transcriptionpolymerase chain reaction (RT-PCR) testing (from nasopharyngeal and oropharyngeal swabs) to confirm acute COVID-19. Prior to study inclusion, all candidates underwent a detailed assessment of the study inclusion and exclusion criteria.
Exclusion criteria were age < 18 years, incomplete medical documentation, and loss of follow-up. The initial patient screening included a total of 67 patients, and due to incomplete medical documentation, the study eligibility was confirmed in a total of 65 patients (Supplementary Figure S1). The patients that were found not eligible were assessed after the end of the study follow-up, and there were no fatal events in these two patients. This observational study was reported according to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) recommendations (Supplementary File S2).

Ethical and Institutional Considerations
The study was performed in agreement with the principles of Good Clinical Practice, following the ethical standards and amendments of the Declaration of Helsinki. The study protocol was approved by the Medical Research Ethical Committee of the University Hospital of Split, Croatia (No. 2181-147-01/06). Informed consent was obtained from all patients before enrolment in the study.

ImAgeS Risk Score and Data Collection
The novel ImAgeS risk score was recently developed by de Andrade et al. in a large cohort of Brazilian kidney transplant recipients with COVID-19 [4]. This pivotal study evaluated the predictors of 28-day mortality to develop an online multi-component calculator (i.e., ImAgeS risk score) that consists of different clinical variables. It includes baseline patient characteristics (age, body mass, body height, sex, race, smoking history, creatinine, specific comorbidities), chronic immunosuppression therapy (steroids, antimetabolites and mammalian target of rapamycin (mTOR) inhibitors), and COVID-19-related symptoms (anosmia, headache, coryza, dyspnea and duration of symptoms). Specific comorbidities from the risk score are pre-existent diabetes mellitus, arterial hypertension and cardiovascular disease.
The ImAgeS risk score was initially developed in a cohort of kidney transplant recipients and was thereafter internally validated in a similar cohort, during which it demonstrated good discrimination and calibration. Finally, an easy-to-use web application was computed. A total score was expressed as the probability (%) and relative risk of 28day mortality.

Data Collection
Baseline characteristics were collected at the time of hospital admission, including age, mean blood pressure (MBP), peripheral oxygen blood saturation (SpO2), comorbidities, laboratory parameters (white blood cell, hemoglobin, platelets, C-reactive protein, estimated glomerular filtration rate, D-dimer), transplantation-related data (transplant duration, immunosuppressive therapy) and COVID-19-related data (clinical picture, vaccination status, therapy). It was collected using the electronic database and hospital reports. MBP was calculated using the sum of 1/3 systolic blood pressure and 2/3 diastolic blood pressure. Laboratory analysis was performed under routine hospital laboratory standards.

Aims and Outcomes
A primary outcome was an association between the ImAgeS score and 30-day mortality, including discrimination and calibration. The secondary outcome included an exploratory analysis of the baseline patient characteristics across the median categories of the ImAgeS score. Finally, we aimed to evaluate the association between selected patient characteristics and all-cause mortality.

Statistical Analysis
Continuous data were presented as median (interquartile range [IQR]) and analyzed using the Kruskal Wallis test, while categorical variables were expressed as numbers (percentages) and analyzed using the Chi-squared test. Descriptive statistics across the ImAgeS risk score quantiles were performed for exploratory purposes.
To determine the relationship between ImAgeS risk score and 30-day mortality, Cox proportional hazard regression analysis was performed. The results were presented as a hazard ratio (HR) and 95% confidence intervals (CI). The risk score was evaluated as a continuous variable, and the results refer to a 1-unit change of risk score on a continuous scale. The results of this analysis allow for insights into the association between ImAgeS risk score and mortality.
In addition, the performance of the ImAgeS risk score was assessed using the receiver operating characteristic (ROC) and area under the curve (AUC). A better performance (diagnostic accuracy) of the risk score is defined by a higher AUC value and a ROC curve that is closer to the top-left corner of the graph. Finally, the Hosmer-Lemeshow goodnessof-fit test, Somers' D and Harrell's C concordance index, and calibration plots were used to determine the calibration and goodness-of-fit of the models.
Finally, binomial multivariable logistic regression was conducted to determine the association between selected patient characteristics and all-cause mortality. The number of variables was restricted to prevent overfitting of the model.
A two-sided p-value of <0.05 was considered significant. P-values were not adjusted for multiple tests and should be interpreted as exploratory only. Statistical data analysis was carried out using a Statistical Package for the Social Sciences (SPSS) software (IBM Corp., New York, NY, USA; version 20) and Stata software (StataCorp, College Station, TX, USA; version 17).

Exploratory Analysis
The study analysis included data on 65 patients. The baseline characteristics of the study sample are presented in Table 1. An exploratory analysis evaluating descriptive patient characteristics across the ImAgeS score quantiles showed that there were not statistically significant between-group differences, except for younger age (50 vs. 62 years, p = 0.001) and higher estimated glomerular filtration rate (66.7 vs. 36.0 mL/min/1.73 m 2 , p = 0.042) in patients with lower ImAgeS score (Table 1). There was no statistically significant difference in the comorbidities across the groups, including arterial hypertension, diabetes mellitus, chronic heart failure, active smoking, atrial fibrillation, prior acute myocardial infarction, prior cerebrovascular accident, peripheral artery disease, and chronic obstructive pulmonary disease/asthma. Importantly, there was no difference in vaccination status, oxygen supplementation, utilization of remdesivir, and corticosteroid therapy between the study groups (p > 0.05) ( Table 1). Finally, there was no difference in the duration of follow-up (Table 1). Data are expressed as number (percent) or median (interquartile range). * Comparison of groups based on the median. Abbreviations: AMI-acute myocardial infarction; COPD-chronic obstructive pulmonary disease; CRP-C-reactive peptide; CVI-cerebrovascular incident; eGFR-estimated glomerular filtration rate; Hgb-hemoglobin; PAD-peripheral arterial disease; WBC-white blood cells.

Discussion
To the best of our knowledge, this is the first external validation study of the ImAgeS risk score that was developed on a Brazilian cohort of kidney transplant recipients with COVID-19. The high-risk profile of kidney transplant recipients and the lack of specific risk stratification tools mandate further validation of the ImAgeS score. There are several important findings of this study. First, the high mortality rates of kidney transplant recipients due to COVID-19 are consistent with previous literature. Second, the ImAgeS risk score showed a statistically significant association with 30-day all-cause mortality, including modest but significant discrimination. Finally, the calibration of the model was acceptable. These findings confirm the initial reports from the internal validation cohort, suggesting that the ImAgeS risk score could aid in the clinical decision-making process. Risk stratification is a critical component of the care of kidney transplant recipients with COVID-19. By identifying patients at the highest risk of severe disease and mortality, clinicians could tailor their care to improve outcomes.
The impaired prognosis of kidney transplant recipients with COVID-19 has been at- When evaluating the association of patient characteristics and all-cause mortality, there was no statistically significant association among the selected variables, such as age, sex, arterial hypertension, diabetes mellitus, chronic heart failure and transplant duration (Supplementary Table S3).

Calibration
The calibration of the ImAgeS risk score model was acceptable with a Hosmer-Lemeshow value of 3.74, Harrell's C concordance index of 0.699 and Somers' D of 0.397 (Table 2). This was confirmed by a calibration plot (Supplementary Figure S2).

Discussion
To the best of our knowledge, this is the first external validation study of the ImAgeS risk score that was developed on a Brazilian cohort of kidney transplant recipients with COVID-19. The high-risk profile of kidney transplant recipients and the lack of specific risk stratification tools mandate further validation of the ImAgeS score. There are several important findings of this study. First, the high mortality rates of kidney transplant recipients due to COVID-19 are consistent with previous literature. Second, the ImAgeS risk score showed a statistically significant association with 30-day all-cause mortality, including modest but significant discrimination. Finally, the calibration of the model was acceptable. These findings confirm the initial reports from the internal validation cohort, suggesting that the ImAgeS risk score could aid in the clinical decision-making process. Risk stratification is a critical component of the care of kidney transplant recipients with COVID-19. By identifying patients at the highest risk of severe disease and mortality, clinicians could tailor their care to improve outcomes.
The impaired prognosis of kidney transplant recipients with COVID-19 has been attributed to various factors. Chronic immunosuppression carries a consequently higher risk of co-infections [5,[10][11][12]. Recent transplantation has been previously associated with higher infection rates and the severe disease form, suggesting that a stronger level of immunosuppression mediates worse outcomes [4,13]. Another well-known complication of immunosuppression is lymphopenia which proved as an independent predictor of COVID-19-associated mortality in the general population [4,[14][15][16][17]. Other important features likely contribute to the advanced risk of this population, such as kidney dysfunction, dependency on routine healthcare, polypharmacy, and frailty. The heterogeneity of this patient population further aggravates the clinical decision-making process and warrants better risk stratification.
Up to this moment, many risk scores have been investigated in the general population of COVID-19 patients [18][19][20][21][22][23][24], but only a few have been studied in the setting of kidney transplant recipients with COVID-19. A close author group recently evaluated different laboratory-derived biomarkers in assessing 30-day mortality risk in a small population of kidney transplant recipients [7]. Additionally, the CROW-65 risk score demonstrated acceptable calibration and discrimination in these patients, but further evaluation in a larger cohort was warranted [8]. The predictive ability of several other clinical risk scores was recently evaluated in a cohort of 57 kidney transplant recipients treated in the intensive care unit for severe COVID-19. The analysis encompassed SOFA, SAPS 3, and APACHE IV scores, while only the APACHE IV showed good performance [25].
Andrade et al. has recently developed the ImAgeS risk score to predict mortality in kidney transplant recipients with COVID-19, which can be used to identify high-risk patients [4]. External validation of this risk score was crucial to ensure its accuracy and reliability in predicting risk. Having in mind that the ImAgeS score includes clinical parameters available at hospital admission, this risk score could aid physicians in the decision-making process at the time of the first clinical assessment [4].
The ImAgeS risk score includes relevant parameters for kidney transplant recipients, such as chronic immunosuppressive therapy, specifically the use of steroids, antimetabolite (mycophenolate/azathioprine) and mTOR inhibitors [4]. Available literature suggests that patients treated with mTOR inhibitors may sustain a milder form of COVID-19 due to the potential antiviral effects of this drug family [4,13,26]. Additionally, according to de Andrade et al., patients with mTOR inhibitors in chronic immunosuppressive therapy have lower ImAgeS scores and are, therefore, at a lower risk of mortality [4]. There is evidence in the literature suggesting that the PI3K/Akt/mTOR pathway could be one of the potential pathophysiologic pathways of COVID-19. Therefore, affecting that pathway could downregulate inflammatory response and affect the course of COVID-19 [27,28]. On the other hand, patients receiving antimetabolite had a higher risk of worse outcomes which could be attributed to drug-induced lymphopenia, which is a recognized contributor to COVID-19-associated mortality [4,14,15,29]. In this study, there was no statistically significant difference in chronic immunosuppressive therapy between patients with lower and higher ImAgeS risk scores.
Kidney dysfunction has been associated with mortality in the overall COVID-19 population, with a doubled risk of death in patients with eGFR < 30 mL/min/1.73 m 2 compared to patients with eGFR > 60 mL/min/1.73 m 2 [2]. Furthermore, Bajpai et al. reported a statistically significant association between eGFR levels and survival amongst kidney transplant recipients with COVID-19 [30]. Worse baseline graft function in kidney transplant recipients is a well-known predictor of worse outcomes. Importantly, this clinical factor is included in the ImAgeS risk score [4]. Similar results were reported in our cohort with higher eGFR in patients with higher ImAgeS scores than in those with lower ImAgeS scores.
Another important characteristic that showed an association with adverse outcomes in previous studies is age. There are a lot of reports about the more severe clinical picture in elderly patients, including higher mortality rates [31][32][33][34][35]. When comparing patients with lower and higher ImAgeS scores, similar results were reported in our study, with elderly patients having higher ImAgeS scores. Increasing age is associated with frailty, higher comorbidity burden, and increased susceptibility to co-infections, all of which mediate adverse outcomes. Therefore, age represents an important component of the ImAgeS risk score, although this study did not show an independent association of age with mortality.
Furthermore, there was no significant difference in comorbidities between patients with lower and higher ImAgeS risk scores in this study. This is in contrast with available literature where patients with severe disease form more often had comorbidities, such as diabetes, arterial hypertension and cardiovascular disease [32,33], and often higher mortality rates [34]. This may be a consequence of the specificity of kidney transplant recipients and indicates that other patient characteristics are more important for their prognosis.
This study was conducted across a relatively wide time frame during which therapeutic strategies for COVID-19 have evolved, with different vaccination types becoming available worldwide. Global vaccination has reduced the infection rate and disease severity, but its effects inevitably vary across different patient populations [36]. Kidney transplant recipients exhibit a low response to the messenger ribonucleic acid (mRNA) vaccine, warranting improved risk stratification and management strategies for this fragile population [37][38][39]. It is important to emphasize that there was no significant difference between patients with lower and higher ImAgeS risk scores regarding vaccination status in this study.
Having in mind that this risk score has previously shown a satisfactory prediction of mortality in the pivotal study, it could be expected that an association between the ImAgeS risk score and mortality was confirmed in this external validation study. However, different effect sizes should be acknowledged and interpreted regarding the differences between these studies. The value of the ImAgeS risk score should be investigated in further studies.
This study could have several clinical implications. First, it reintroduces this potentially useful risk score to the clinicians. Risk stratification can help guide decisions about hospitalization, monitoring, and treatment for kidney transplant recipients with COVID-19. It can also allow less resource utilization and improved outcomes. Second, the inclusion of different variables that are both widely available and clinically relevant to kidney transplant recipients is particularly important for everyday practice. Having in mind the burden of the COVID-19 pandemic on healthcare systems, focused interventions in high-risk individuals could be both patient-centered and resource-saving. For example, patients at higher risk may require more frequent monitoring of their oxygen levels, kidney function, and immune status, as well as closer observation for signs of progression to severe disease [40]. In addition, these patients may benefit from more aggressive treatment strategies, such as early initiation of antiviral therapy or the use of monoclonal antibodies. Nevertheless, a cautious approach is necessary as there may be important geographical and population differences that could interfere with the performance of the ImAgeS risk score, and the limited sample size of the external validation cohort could interfere with its clinical applica-tion. The predictability, calibration, and discrimination of the ImAgeS risk score should be further validated in different geographic areas using sufficiently powered cohorts.
This study has several limitations. First, a relatively small sample size from a single center can affect study strength. The observed effect size was mild, which could be mediated by a limited sample size. Second, the COVID-19 management algorithms have changed throughout the study, which could potentially affect the prognosis of the cohort. Third, the observational retrospective nature of the study has certain inherent drawbacks, including the possibility of selection bias. The fourth, single-center analysis could also support selection bias. Importantly, any causal inferences could not be determined with this study. Finally, due to short-term follow-up, it can't be used for assessment of long-term prognosis in this patient population.

Conclusions
In conclusion, the ImAgeS risk score showed a statistically significant association with 30-day all-cause mortality in kidney transplant recipients with COVID-19, including modest discrimination and satisfactory calibration, confirming the findings from the computational study. Further studies are needed to determine the utility of the ImAgeS score in this high-risk population.