Next Article in Journal
History of Technological Advancements towards MR-Linac: The Future of Image-Guided Radiotherapy
Previous Article in Journal
Mechanical Circulatory Support in Delayed Surgery of Post-Infarction Ventricular Septal Rupture in Patients in Cardiogenic Shock—A Review
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Application of Machine Learning in Hospitalized Patients with Severe COVID-19 Treated with Tocilizumab

Department of Pharmacy, General University Hospital, 46014 Valencia, Spain
Institute of Technology, University of Castilla-La Mancha, 16002 Cuenca, Spain
Department of Pharmacology, Faculty of Medicine, University of Valencia, 46010 Valencia, Spain
Centre for Biomedical Research Network on Respiratory Diseases (CIBERES), Health Institute Carlos III, 28029 Madrid, Spain
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
J. Clin. Med. 2022, 11(16), 4729;
Submission received: 4 July 2022 / Revised: 5 August 2022 / Accepted: 7 August 2022 / Published: 12 August 2022


Among the IL-6 inhibitors, tocilizumab is the most widely used therapeutic option in patients with SARS-CoV-2-associated severe respiratory failure (SRF). The aim of our study was to provide evidence on predictors of poor outcome in patients with COVID-19 treated with tocilizumab, using machine learning (ML) techniques. We conducted a retrospective study, analyzing the clinical, laboratory and sociodemographic data of patients admitted for severe COVID-19 with SRF, treated with tocilizumab. The extreme gradient boost (XGB) method had the highest balanced accuracy (93.16%). The factors associated with a worse outcome of tocilizumab use in terms of mortality were: baseline situation at the start of tocilizumab treatment requiring invasive mechanical ventilation (IMV), elevated ferritin, lactate dehydrogenase (LDH) and glutamate-pyruvate transaminase (GPT), lymphopenia, and low PaFi [ratio between arterial oxygen pressure and inspired oxygen fraction (PaO2/FiO2)] values. The factors associated with a worse outcome of tocilizumab use in terms of hospital stay were: baseline situation at the start of tocilizumab treatment requiring IMV or supplemental oxygen, elevated levels of ferritin, glutamate-oxaloacetate transaminase (GOT), GPT, C-reactive protein (CRP), LDH, lymphopenia, and low PaFi values. In our study focused on patients with severe COVID-19 treated with tocilizumab, the factors that were weighted most strongly in predicting worse clinical outcome were baseline status at the start of tocilizumab treatment requiring IMV and hyperferritinemia.

1. Introduction

COVID-19 is an acute respiratory illness caused by severe acute respiratory coronavirus-2 (SARS-CoV-2) [1]. Since December 2019, it has caused a pandemic, affecting millions of lives worldwide. It presents a clinical course in two phases. The first is an asymptomatic or silent viral replication phase of about five days after infection, with symptoms of cough, fever and dyspnea. This phase may progress to a second more severe phase that can lead to bilateral pneumonia with severe respiratory failure (SRF) with or without acute respiratory distress syndrome (ARDS) and/or fatal multiorgan failure [2,3]. About 10% of patients require care in intensive care units (ICU) [4]. The mortality rate in the first wave was <3%, although the case fatality rate of severe cases was high, according to the World Health Organization (WHO).
Risk factors associated with the progression and severity of SARS-CoV-2 have been identified as advanced age, different comorbidities, as well as laboratory parameters, such as lactate dehydrogenase (LDH), procalcitonin, high-sensitivity C-reactive protein (CRP), ferritin, and proinflammatory cytokines such as interleukin (IL)-6 and IL-1β [5,6]. Elevated IL-6 levels and hyperferritinemia are considered indicators of systemic inflammation and poor prognosis in COVID-19 [7]. The neutrophil-to-lymphocyte ratio is usually elevated in patients with severe COVID-19 [8]. Potential complications include acute kidney injury, coagulation disorders, ARDS, or shock that may result in mortality rates >30% [5].
In patients with severe COVID-19, a hyperinflammatory response to SARS-CoV-2 occurs due to a dysregulated innate host immune response, with increased levels of proinflammatory cytokines (IL-1, IL-6, TNF-α) and other acute phase reactants (CRP, D-dimer or ferritin) [9]. This cytokine release syndrome (CRS), or “cytokine storm”, plays a key role in the pathogenesis of severe COVID-19, including lung damage and microvascular thrombosis [10], potentially leading to multiorgan failure and death [4]. The IL-6 signaling pathway is one of the key mediators of the hyperinflammatory syndrome and CRS in COVID-19. IL-6 is involved in the signal transduction pathway through Janus kinases-signal transducers and activators of transcription (JAK-STAT). This pathway is essential in the processes of cell proliferation and differentiation, immunity and apoptosis [3,4]. Blocking this pathway at different levels could prevent dysregulation of the inflammatory, immune and coagulation systems and limit lung damage, reducing the risk of progression to SRF. For this reason, current research is focused on the use of different molecules capable of blocking the cytokine storm, such as IL-1 or IL-6 antagonists or JAK-STAT inhibitors.
Tocilizumab is a humanized monoclonal antibody which binds to the soluble and membrane-bound IL-6 receptor, blocking the signal transduction by which JAK-STAT is activated, perpetuating the cytokine storm [11]. Tocilizumab has demonstrated a favorable benefit–risk ratio, improving the prognosis of patients with COVID-19 [4,12,13]. The recent meta-analysis by Luo L et al. concludes that tocilizumab significantly reduces the relative risk of 28–30-day mortality and the risk of invasive mechanical ventilation (IMV), without increasing the risk of infection and/or adverse events [14].
Based on all these data, the European Society of Clinical Microbiology and Infectious Diseases suggests using tocilizumab in patients with SARS-CoV-2-related SRF [15]. Despite this evidence, the risks of mortality and IMV in patients treated with tocilizumab are not negligible [15]. Currently, little is known about the predictive factors associated with poor outcome in patients with COVID-19 treated with immunomodulators in general and tocilizumab in particular in clinical practice.
To solve this lack of knowledge, machine learning (ML) models can be used, designed to make accurate predictions on any question of interest, using data from a multitude of variables, as opposed to classical statistical models created to make inferences about the relationships between variables. ML, as part of artificial intelligence (AI), uses statistical and mathematical algorithms that enable it to opt for patterns that help to make complex decisions [16]. These algorithms can be used to develop predictive models and reduce the complexity of clinical phenotypes. AI tools have been implemented in the fight against COVID-19. They have been used primarily in: early diagnosis; the prediction of severity, mortality and complications; as well as in drug and vaccine discovery or repurposing [17,18,19].
To our knowledge, this is the first study to develop, compare and validate five ML models to predict factors associated with reduced clinical benefit of tocilizumab, in terms of mortality and hospital stay, in patients with SRF due to SARS-CoV-2.

2. Materials and Methods

2.1. Data Source

Patient data were obtained from different internal hospital sources: (1) electronic medical record (EMR) (Hosix. Net. Ink.), which includes a module for recording clinical analysis results and an electronic drug prescription module, and (2) ICU prescription software (IntelliSpace Critical Care and Anesthesia, version H.02.00. Philips Ibérica SA, Eindhoven, The Netherlands). With this information, an individualized data collection questionnaire (DCQ) was constructed for each patient.

2.2. Study Design and Population

This was a longitudinal, retrospective, observational study conducted in a tertiary hospital. Sixty-seven patients (64.2% male) admitted to the hospital with microbiologically confirmed SARS-CoV-2 by reverse transcriptase polymerase chain reaction (RT-PCR) assay of a nasopharyngeal swab between 10 March and 16 September 2020 were included.
Inclusion criteria were: patients > 18 years old admitted to the General University Hospital of Valencia (GUHV) with severe COVID-19 pneumonia, with comorbidity and requiring admission to the ICU. The patients selected had to be candidates for tocilizumab administration according to the hospital’s internal protocol at the start date of the study: onset of symptoms ≥ 7 days, interstitial pneumonia with SRF and extensive pulmonary infiltrate, rapid respiratory worsening requiring noninvasive or invasive ventilation (COVID respiratory severity scale ≥ 3), presence of other organ failure (mainly shock or SOFA scale score ≥ 3) and criteria for severe systemic inflammatory response: elevated IL-6 levels (>40 pg/mL) or alternatively elevated D-dimer levels (>1500 ng/mL) or progressively increasing D-dimer. The approved posology for tocilizumab was: (a) patients ≥ 80 kg, who were given a first dose of 600 mg in 100 mL of sterile, non-pyrogenic sodium chloride 9 mg/mL (0.9%) solution for injection and 1-h intravenous infusion and a second dose of 600 mg after 12 h of the first dose. In case of persistent fever and worsening of analytical parameters (CRP, D-dimer or IL-6) a third dose could be considered 16–24 h after the second dose; and (b) patients < 80 kg, who were given a first dose of 600 mg and a second dose of 400 mg. Likewise, a maximum of 3 infusions could be considered, only if there is no significant clinical improvement.
In our study, all patients received corticosteroids: (a) methylprednisolone 250 mg/24 h (1 day) followed by 40 mg/12 h (4 days) intravenously, or (b) dexamethasone 6 mg/24 h for 10 days (starting intravenously and switching to oral if clinical stability and tolerance). Similarly, all patients received azithromycin 500 mg/24 h (5 days) and prophylactic doses of enoxaparin (60 mg/24 h if BMI < 30 or 80 mg/24 h if BMI ≥ 30). If D-dimer > 2000 ng/mL, enoxaparin 1.5 mg/kg/24 h.
Exclusion criteria were: patients ≤ 18 years, patients with missing data for more than one clinical/analytical variable in this period, and patients who were not candidates for tocilizumab for any of the following reasons: GOT and/or GPT > 5 times normality levels, neutrophils < 500 cells/mm3, platelets < 50,000 cells/mm3, documented sepsis due to pathogens other than SARS-CoV-2, complicated diverticulitis or intestinal perforation, ongoing skin infection (e.g., pyodermitis not controlled with antibiotic treatment), immunosuppressive anti-rejection therapy, and/or use of monoclonal antibodies that could produce immunosuppression.
The study was approved by the Ethics Committee of the GUHV. Participants gave informed consent before participating in the study. Protocol code: XGB-COVID-19.

2.3. Study Data

Data on demographic, clinical and laboratory variables, as well as the date of hospital admission, were included in the DCQ. In all patients, data on laboratory parameters were collected at the moment of tocilizumab initiation. Demographic variables included age and sex, weight and height. Clinical variables included: (1) pharmacotherapy prior to and on admission: (a) use of antivirals: oral lopinavir/ritonavir (400/100 mg/12 h) or intravenous remdesivir (200 mg on day 1, followed by 100 mg/day for up to 10 days); (b) use of oral hydroxychloroquine [400 mg/12 h (day 1) and 200 mg/12 h (days 2–4)]; (c) use of subcutaneous interferon beta-1b 250 mcg/48 h; (d) use of subcutaneous anakinra, preferably in patients with poor evolution (PaO2/FiO2 < 300, SpO2 < 92% in room air, tachypnea or high ferritin). Doses of 200 mg/24 h (day 1) and 100 mg/24 h (days 2–5); (2) baseline patient status at initiation of tocilizumab therapy: (a) patient requiring supplemental O2 but not IMV and (b) patient requiring IMV; (3) clinical status at day 7 and 21 after hospital admission: (a) hospital discharge with resumption of normal activities; (b) hospital discharge with difficulty resuming normal activities; (c) hospitalized not requiring supplemental O2; (d) hospitalized requiring supplemental O2 but not IMV; (e) hospitalized requiring IMV and (f) exitus; (4) comorbidities: smoking, arterial hypertension, obesity, ischemic heart disease, dyslipidemia, diabetes mellitus (DM), chronic obstructive pulmonary disease (COPD) and chronic kidney disease (CKD); and (5) use or not of IMV and types. Finally, the following laboratory variables were collected: PaO2/FiO2, FiO2, SpO2, CRP, ferritin, LDH, glutamate-oxaloacetate transaminase (GOT) and glutamate-pyruvate transaminase (GPT), procalcitonin and lymphocytes.
To measure the efficacy of tocilizumab, in-hospital mortality data and days of hospital stay were recorded.

2.4. Method

2.4.1. Model Development

For this study, the extreme gradient boost (XGB) based algorithm has been proposed as the reference algorithm because it presents several important features such as its scalability, parallel computing and its speed in execution [20,21]. To build this classification model we start from a dataset S = xj, yj,
y j ^ = p = 1 P t p x j
where xj represents the input vector with m time variables, y j ^ shows the predicted output, yj is the output, tp represents a tree with leaf weight wp and structure up, j = 1; 2; ...; n, and P corresponds to the number of trees.
The regularized objective function is presented in Equation (2) for the proposed XGB method. A second-order Taylor expansion is applied to approximate the XGB objective function in order to optimize the prediction accuracy [20].
R = j r y j ^ , y j + p Ψ t p
Ψ t p = λ f p + 1 2 γ ω p 2
In Equation (3), fp shows the number of leaves on the tree. The function R penalizes the complexity of the method. The learning rate is shown by λ, where wp is the vector of leaf scores. By means of the parameter γ the complexity of the system is monitored. The objective is to optimize Equation (2) [21].
In this work, other ML algorithms have been implemented to test the performance of the proposed method. The four that gave the best results in the comparison were selected. The following systems stood out: decision tree (DT) [22], Bayesian linear discriminant analysis (BLDA) [23], K-nearest neighbour (KNN) [24], and support vector machine (SVM) [25]. All of them were designed using Matlab 2022 software. For the validation of the results, a 5-fold cross-validation was performed to avoid overfitting. The input data was divided into two blocks, 70% was used for training and 30% for validation. In all simulations, 100 iterations were performed to obtain the standard deviation and mean values in a uniformly random manner. In this way, the impact of noise is reduced, appropriate values are calculated and statistically valid results are obtained [26]. In Figure 1, the procedure carried out in this study can be seen.

2.4.2. Performance Evaluation

In this work, the following metrics were used to test the performance of the compared methods: degenerated Younden index (DYI), specificity, recall, precision, balanced precision, receiver operating characteristic (ROC), F1-score, Matthew’s correlation coefficient (MCC), Cohen’s Kappa index (CK), and area under the curve (AUC) [26].

3. Results

This section describes the results obtained with the patient registries used for training and validation to define the factors with the greatest influence on in-hospital mortality and days of hospital stay in patients with severe COVID-19 treated with tocilizumab. The performance of the proposed system has been compared with different ML classification methods accepted in the scientific community.
Table 1 presents the results obtained from the classification methods applied: SVM, DT, BLDA, KNN and the proposed XGB system. As can be seen, the systems based on SVM and BLDA obtain a lower accuracy value than the rest of the methods; these values are close to 82%. As for the DT and KNN methods, they improve the classification capacity obtaining accuracy values of 86%, presenting a better performance than SVM and BLDA. On the other hand, the proposed XGB system achieves accuracy values of 93%, a significant increase over previous methods, which translates into better prediction. As can be seen in Table 1, the same happens again with the F1 score parameter, where XGB obtains higher values that imply an improvement in classification.
To test the performance of the proposed XGB system, other parameters widely used in the literature were calculated, such as the AUC, the MCC, the DYI and the Kappa index. For this analysis, one of the most reliable statistical indices available, the MCC, was used. This coefficient produces a high score only if the prediction has performed well in all four categories of the confusion matrix (true positives, false negatives, true negatives and false positives). The results in the four categories of the matrix are proportional to the size of the positive elements and the size of the negative elements in the dataset. As can be seen in Table 1, the proposed method XGB achieves a value of 84.41%, increasing the values achieved by KNN and DT. The latter show values of 76.97% and 74.18%. Both SVM and BLDA show a worse performance in this parameter. As for the Kappa index, XGB obtains a value close to 84%, improving the value of KNN and DT by 6.76% and 10.02%, respectively. As for the AUC and DYI parameters, the same is true again; the XGB method achieves higher values, which means that it obtains a better classification of patients who may benefit less from tocilizumab treatment.
On the other hand, the ROC curve has been used to compare the classification ability of the proposed system with the other ML methods. The curve is the result of plotting the sensitivity and specificity for each threshold value [26]. As can be seen in Figure 2, the results obtained by the different systems for classification between patients according to the objective of the proposed study, are shown. A larger surface area is seen for the XGB method, which implies a better classification of the two classes; these values can be seen in Table 1.
For clarity, all metrics have been grouped for each data set (training and validation) and are presented as a radar plot. A perfect score on all metrics would be represented by a circle the size of the entire grid. In our study, the training sets of the model have higher scores on all metrics and, in general, have lower scores on the validation set. The shape of the graphs can also be indicative of the quality of the models. The larger the area of the circle in the validation set, the better the prediction method. The proposed XGB system (Figure 3) is a good example of a balanced model. The training and validation sets result in similar pie charts. These similarities are because the system obtains an optimal training point, with no overfitting or underfitting, making the method highly generalisable. That is, given a new input, the system does well to provide a correct output. As can be seen, the BLDA method was the worst performer on most metrics. In view of the results obtained, we can say that the proposed XGB system manages to classify patients according to the study objective with high accuracy and automatically, which makes this tool very helpful for clinical practice.
The parameters used for the implementation of the different methods are shown in Table 2.
With the proposed XGB method, the factors associated with a worse outcome of tocilizumab use in terms of mortality were: baseline situation at the start of tocilizumab treatment requiring IMV, elevated ferritin, LDH and GPT, lymphopenia, and low PAFI values. The factors associated with a worse outcome of tocilizumab use in terms of hospital stay were: baseline situation at the start of tocilizumab treatment requiring IMV or supplemental oxygen, elevated levels of ferritin, GOT, GPT, CRP, LDH, lymphopenia, and low PAFI values.
The baseline clinical data of the 67 patients included in the study are shown in Table 3.

4. Discussion

To our knowledge, this is the first study to develop, compare and evaluate five supervised ML methods to predict the factors that most negatively influence the efficacy, in terms of mortality and hospital stay, of tocilizumab in patients hospitalized with SRF due to SARS-CoV-2. Forty-one demographic, clinical and laboratory variables were collected. After analysis of the five ML algorithms, the XGB method had the highest balanced accuracy (93.16%).
Infection of airway epithelial cells by SARS-CoV-2 causes cytological damage by activating the local immune response, releasing proinflammatory mediators such as IFNγ, IL-1β, IL-6, TNF-α [27]. IL-1β acts on endothelial and vascular smooth muscle cells and induces IL-6 production [28]. IL-6, through induction of proinflammatory chemokines and cytokines, plays a critical role in the progression from mild inflammation to hyperinflammatory conditions, CRS, ARDS and corresponding lung damage causing mortality in critically ill COVID-19 patients [29,30]. Therefore, blockade of the IL-6 pathway is considered a key approach to reducing severe lung damage caused by SARS-CoV-2 [4]. ARDS is the most common cause of death in these patients, with mortality rates that can exceed 75% [3,31]. ARDS is a consequence of dysregulated immune response, hyperinflammation and activation of the coagulation cascade, with CRS playing the major role [32]. Currently available treatment focuses on modulating immune responses, and obtaining antiviral, antithrombotic or anticoagulant effects [3]. The drugs with the most accumulating evidence for CRS blockade are IL-1 (anakinra) or IL-6 (tocilizumab or sarilumab) inhibitors, janus kinase inhibitors [JAKi] (e.g., baricitinib), as well as corticosteroids [4,33,34]. Increasing evidence supports the incremental efficacy of corticosteroids alone or in combination with tocilizumab/JAKi in moderate to severe and critical COVID-19 [4,35,36]. Therapeutic plasma exchange also appears to have potential benefits in reducing the risk of mortality among critically ill COVID-19 patients [37]. Of this therapeutic arsenal, tocilizumab is the most studied drug in the setting of severe COVID-19. Most randomized clinical trials (RCTs) and observational studies show favourable evidence that tocilizumab, when added to standard therapy, significantly reduces both 30-day mortality and the need for IMV without causing serious adverse events [38,39,40,41]. The RECOVERY study, initiated in March 2020, is the RCT with the largest number of participants enrolled so far, to test potential treatments against COVID-19 [38]. More than 4000 hospitalized COVID-19 patients with hypoxia and systemic inflammation were included. Tocilizumab was found to improve survival and other clinical outcomes, such as time to hospital discharge and the number of patients eventually receiving IMV. These benefits were observed regardless of the amount of ventilatory support and were in addition to the benefits of systemic corticosteroids (82% of patients received baseline corticosteroids). Despite this favourable evidence, a non-negligible percentage of patients with SRF due to SARS-CoV-2 treated with tocilizumab ultimately dies or undergoes IMV. Indeed, a recent meta-analysis of RCTs and cohort studies including more than 3000 patients with COVID-19 on tocilizumab treatment showed that 30-day mortality was more than 20%, mainly in the most severe patients, similar to the incidence of IMV [14]. Therefore, studies to identify factors influencing response to tocilizumab are warranted.
In our study, in-hospital mortality was 35.8% and 19.4% of all patients required IMV. The median hospital stay was 14 days (interquartile range [IQR = 22]). Of all the ML classifiers applied, the XGB method was the pattern recognition method that most accurately discriminated between patients at increased risk of poor outcome with tocilizumab use from those who were not. This model was analyzed and compared with different supervised ML methods described in the literature, such as BLDA, DT, KNN or SVM. In our study, BLDA and SVM were the worst performing methods, with KNN being the method that comes closest to the accuracy values of XGB. This is consistent with the results of studies describing these supervised ML algorithms in predicting COVID-19 mortality [42,43]. Our study provides a balanced radar graph between the training and test phases. The results show that the XGB model can handle large data dimensions while avoiding overtraining and significantly improves the performance of other classification methods. It obtains higher precision, recall and accuracy values than those achieved by the other methods. This guarantees its reliability for automatic classification of the desired result. XGB is a predictive model that has excellent scalability and high execution speed, used in various fields of biomedicine, such as in predicting the stage of cancer patients [44].
In our research, the factors associated with a worse outcome of tocilizumab in terms of mortality were: baseline status at initiation of tocilizumab treatment requiring IMV, elevated ferritin, LDH and GPT, lymphopenia, and low PaFi [ratio of arterial oxygen pressure to inspired oxygen fraction (PaO2/FIO2)] values. The factors associated with worse outcome of tocilizumab use in terms of hospital stay were: baseline status at the start of tocilizumab treatment requiring IMV or supplemental oxygen, elevated levels of ferritin, GOT, GPT, CRP, LDH, lymphopenia, and low PaFi values.
Some studies have investigated factors influencing response to tocilizumab in patients with severe COVID-19. In our case, in contrast to other studies, advanced age and comorbidities were not found to play a crucial role in the lack of efficacy of tocilizumab on mortality [45,46]. In our study, there were no differences in mortality between older and younger patients. In other studies, in contrast to ours, high procalcitonin values seem to predict a worse outcome in tocilizumab patients [45,47]. Procalcitonin is increased in patients with severe COVID-19 infection [48]. In the meta-analysis by Lansbury L et al., a low proportion of patients with COVID-19 were found to have bacterial co-infection [49]. Our findings, as with other studies, suggest that decreased PAFI is associated with worse mortality outcomes, as well as low lymphocyte counts [45,46]. In the study by Sarabia De Ardanaz L et al., it was observed that deceased patients who received tocilizumab, compared to discharged patients, had higher LDH (p = 0.013), CRP (p = 0.013) and lower lymphocyte (p = 0.013) values, as well as a lower mean PAFI index, as in our study [46]. In this study, high neutrophil levels (p = 0.024) and low platelet levels (p = 0.013) were negative predictors of tocilizumab efficacy. In our study, platelet counts <50,000 cells/mm3 were considered exclusion criteria for tocilizumab use. It appears that patients who receive tocilizumab earlier have a lower risk of death [46]. In our study, patients received tocilizumab between 7 and 10 days after the onset of symptoms, that is, during the phase of hyperinflammation and CRS.
Other studies highlight that poor outcomes of tocilizumab use are due to elevated creatinine and CRP values, as well as computed tomography (CT) lung surface area involvement > 50% [45,46]. In our study, all patients fulfilled the latter criterion and CRP was a negative predictor of efficacy in relation to hospital stay.
In our investigation, no pharmacological treatment had sufficient weight in predicting poor outcome with tocilizumab. In other studies, the combination of tocilizumab together with remdesivir did not provide any outcome advantage [45,50]. In our study, all patients took corticosteroids, based on the favourable evidence for the use of corticosteroids in combination with tocilizumab in patients with severe COVID-19 [36,38,45,51]. Thus, the recent meta-analysis by Moosazadeh M confirms that the risk of death in COVID-19 patients treated with corticosteroids and tocilizumab was lower than in the tocilizumab alone and control groups (26% and 52%, respectively) [52].
Our findings confirm that both baseline status at the start of tocilizumab treatment requiring IMV and hyperferritinemia are associated with poor outcome and add insight into predictors of tocilizumab failure in patients with SARS-CoV2 and SRF. This is in line with the Bayesian reanalysis of a previous meta-analysis of 15 studies of hospitalized patients with COVID-19 treated with tocilizumab and corticosteroids [53]. This study showed that simple oxygen therapy and non-invasive ventilation (NIV) was associated with a clinically significant mortality benefit of tocilizumab but not in patients with IMV. Elevated IL-6 levels and hyperferritinemia are considered indicators of systemic inflammation and poor prognosis in COVID-19 [7]. In the study by Lohse A et al., which included 34 patients, those with severity criteria at the start of tocilizumab treatment (high levels of CRP, ferritin, fibrinogen, D-dimer and liver enzymes, as well as lymphopenia) were the ones who would eventually die in the highest proportion [54]. In fact, in this study, as in our case, patients with higher lymphopenia and O2 needs and elevated GOT levels had the worst outcome.
On the other hand, the efficacy of tocilizumab treatment seems to be related to the improvement and recovery of the values of analytical parameters associated with systemic inflammation, immune and coagulation disorders, as confirmed by the study by Chamorro-de-Vega E et al., on a Spanish cohort of patients with severe COVID-19 [55]. In that study, the overall mortality was 33.3%, similar to ours, being higher in patients who received IMV than in those who did not (46.2% vs. 26.7%, p < 0.001). In our study, the exitus rate among IMV patients was 53.8%. A significant improvement in lymphocyte count, CRP, LDH and D-dimer was also observed. These factors are relevant in our ML study as influencing the final objective. Therefore, monitoring these parameters during hospitalisation of the patient on tocilizumab treatment would be the key to see if there is a beneficial response or, on the contrary, an increased risk of mortality. The scientific evidence concludes that a poor outcome is associated with persistently high biomarker values over time after tocilizumab administration, such as LDH, IL-6 and low absolute lymphocyte counts [56]. To circumvent this problem, and due to the large subject-to-subject variability in response to anti-inflammatory treatment for COVID-19, personalized strategies are being developed, guided by rapid cytokine assays, such as the digital immunoassay, which allows rapid “real-time” cytokine monitoring [57].
Nevertheless, a significant number of tocilizumab patients still die regardless of the improvement in some inflammatory biomarkers [55]. Therefore, it is important to consider the timing of IL-6 inhibitor use in relation to disease severity, as it seems to influence treatment efficacy [58]. In our case, tocilizumab was administered within 7–10 days of the first symptoms, that is, during the hyperinflammation phase. It has been suggested that administration of treatment in critical illness may not reverse cytokine-mediated injury that has already occurred [59]. Deaths in tocilizumab-treated patients may be due to patients with a poor initial prognosis and a potentially irreversible hyperinflammatory syndrome [60].
There is evidence for the existence of different phenotypes of COVID-19 patients with different inflammatory and immune response, mortality risk and response to pharmacological treatment and respiratory support [32,61,62]. In the study by Chen H et al., using an ML approach, two phenotypes of COVID-19 were identified: hypoinflammatory and hyperinflammatory, the latter characteried by elevated levels of proinflammatory cytokines and higher rates of complications [62]. Steroid treatment was associated with reduced 28-day mortality (HR, 0.45; 95% CI, 0.25–0.80; p = 0.0062) in the hyperinflammatory phenotype. Treatment failure with tocilizumab may support this hypothesis, raising the need to identify factors to tailor treatments.
Our study has limitations and strengths. The main limitations are secondary to the retrospective methodology and the single-centre design with a limited sample, but these are compensated for by the use of powerful tools such as ML. In addition, one of the main advantages of the ML methods is the used of small datasets. Thus, the complexity and classification time of the method proposed can be very low. This is the main reason of using just one small dataset. Moreover, data augmentation technique has been employed [26]. Another limitation is that during the period in which the study was conducted, patients were not yet optimally treated, and this could somewhat influence the results of the study and add confounding factors to our results.
Among the strengths is the existence of a diverse patient population with a relatively high prevalence of comorbidities, which tend to be under-represented in most RCTs. As such, our results are more generalizable. This methodology has the ability to identify patients in whom tocilizumab treatment is associated with longer survival time. Lam C et al., used the ML system to identify hospitalized patients with COVID-19 in whom treatment with a corticosteroid or remdesivir is associated with increased survival time [63]. Similarly, ML algorithms have been used to predict mortality in COVID-19 patients treated with corticosteroids and remdesivir [64]. Comparative studies have revealed that ML methods can be more accurate and efficient than traditional logistic regression analysis, especially when sample size is limited [65].

5. Conclusions

Tocilizumab has been shown to be beneficial in patients with SRF associated with SARS-CoV-2 pneumonia. However, a non-negligible number of patients die or require longer hospital stays until recovery or exitus, despite treatment with tocilizumab. Therefore, we use ML techniques, which are the most sophisticated and accurate tools for predicting events of interest. With them, we selected predictors of tocilizumab treatment failure in patients with severe COVID-19. XGB was the method with the highest predictive accuracy. Among these factors, patients with baseline IMV at the start of tocilizumab treatment and/or hyperferritinemia were candidates for higher tocilizumab treatment failure. This should be taken into account in clinical practice to identify these patients as they will require more aggressive management and closer follow-up to avoid a poor outcome. Further research is needed to advance the understanding of pathophysiological mechanisms and characterize immunoinflammatory alterations across the full range of phenotypes to better classify patients for optimal clinical management. These ML methods will help improve patient outcomes and resource allocation during the COVID-19 pandemic.

Author Contributions

Conceptualization, A.R., M.Z., J.M. (Javier Milara), A.M.T., J.C., P.B. and J.M. (Jorge Mateo); Data curation, A.R., M.Z., J.M. (Javier Milara), A.M.T., J.C., P.B. and J.M. (Jorge Mateo); Formal analysis, A.R., M.Z., J.M. (Javier Milara), A.M.T., J.C., P.B. and J.M. (Jorge Mateo); Funding acquisition, J.M. (Javier Milara) and J.M. (Jorge Mateo); Investigation, A.R., M.Z., J.M. (Javier Milara), A.M.T., J.C., P.B. and J.M. (Jorge Mateo); Methodology, A.R., M.Z., J.M. (Javier Milara), A.M.T., J.C., P.B. and J.M. (Jorge Mateo); Project administration, A.R., M.Z., J.M. (Javier Milara), A.M.T., J.C., P.B. and J.M. (Jorge Mateo); Resources, A.R., M.Z., J.M. (Javier Milara), A.M.T., J.C., P.B. and J.M. (Jorge Mateo); Software, A.R., M.Z., J.M. (Javier Milara), A.M.T., J.C., P.B. and J.M. (Jorge Mateo); Supervision, A.R., M.Z., J.M. (Javier Milara), A.M.T., J.C., P.B. and J.M. (Jorge Mateo); Validation, A.R., M.Z., J.M. (Javier Milara), A.M.T., J.C., P.B. and J.M. (Jorge Mateo); Visualization, A.R., M.Z., J.M. (Javier Milara), A.M.T., J.C., P.B. and J.M. (Jorge Mateo); Writing—original draft, A.R., M.Z., J.M. (Javier Milara), A.M.T., J.C., P.B. and J.M. (Jorge Mateo); Writing—review & editing, A.R., M.Z., J.M. (Javier Milara), A.M.T., J.C., P.B. and J.M. (Jorge Mateo). All authors have read and agreed to the published version of the manuscript.


Research funded by Instituto de Salud Carlos III (PI20/01363), J.M. (Javier Milara), European Regional Development Fund, Centro de Investigaciones Biomédicas en Red de Enfermedades Respiratorias, CIBERES (CB06/06/0027), J.M. (Javier Milara) and Institute of Technology (University of Castilla-La Mancha), J.M. (Jorge Mateo).

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and approved by the Institutional Review Board (or Ethics Committee) of General University Hospital of Valencia (GUHV) (protocol code: XGB-COVID-19; 6 August 2021).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The datasets used and/or analyzed during the present study are available from the corresponding author on reasonable request.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.


  1. Chen, N.; Zhou, M.; Dong, X.; Qu, J.; Gong, F.; Han, Y.; Qiu, Y.; Wang, J.; Liu, Y.; Wei, Y.; et al. Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: A descriptive study. Lancet 2020, 395, 507–513. [Google Scholar] [CrossRef]
  2. Thakur, V.; Ratho, R.K.; Kumar, P.; Bhatia, S.K.; Bora, I.; Mohi, G.K.; Saxena, S.K.; Devi, M.; Yadav, D.; Mehariya, S. Multi-Organ Involvement in COVID-19: Beyond Pulmonary Manifestations. J. Clin. Med. 2021, 10, 446. [Google Scholar] [CrossRef] [PubMed]
  3. Thapa, K.; Verma, N.; Singh, T.G.; Grewal, A.K.; Kanojia, N.; Rani, L. COVID-19-Associated acute respiratory distress syndrome (CARDS): Mechanistic insights on therapeutic intervention and emerging trends. Int. Immunopharmacol. 2021, 101, 108328. [Google Scholar] [CrossRef] [PubMed]
  4. Elahi, R.; Karami, P.; Heidary, A.H.; Esmaeilzadeh, A. An updated overview of recent advances, challenges, and clinical considerations of IL-6 signaling blockade in severe coronavirus disease 2019 (COVID-19). Int. Immunopharmacol. 2022, 105, 108536. [Google Scholar] [CrossRef]
  5. Wang, Z.; Deng, H.; Ou, C.; Liang, J.; Wang, Y.; Jiang, M.; Li, S. Clinical symptoms, comorbidities and complications in severe and non-severe patients with COVID-19: A systematic review and meta-analysis without cases duplication. Medicine 2020, 99, e23327. [Google Scholar] [CrossRef]
  6. Gao, Y.D.; Ding, M.; Dong, X.; Zhang, J.J.; Azkur, A.K.; Azkur, D.; Gan, H.; Sun, Y.L.; Fu, W.; Li, W.; et al. Risk factors for severe and critically ill COVID-19 patients: A review. Allergy 2021, 76, 428–455. [Google Scholar] [CrossRef]
  7. Melo, A.; Milby, K.M.; Caparroz, A.; Pinto, A.; Santos, R.; Rocha, A.P.; Ferreira, G.A.; Souza, V.A.; Valadares, L.; Vieira, R.; et al. Biomarkers of cytokine storm as red flags for severe and fatal COVID-19 cases: A living systematic review and meta-analysis. PLoS ONE 2021, 16, e0253894. [Google Scholar] [CrossRef]
  8. Li, X.; Liu, C.; Mao, Z.; Xiao, M.; Wang, L.; Qi, S.; Zhou, F. Predictive values of neutrophil-to-lymphocyte ratio on disease severity and mortality in COVID-19 patients: A systematic review and meta-analysis. Crit. Care 2020, 24, 647. [Google Scholar] [CrossRef]
  9. Gustine, J.N.; Jones, D. Immunopathology of Hyperinflammation in COVID-19. Am. J. Pathol. 2021, 191, 4–17. [Google Scholar] [CrossRef]
  10. Dorward, D.A.; Russell, C.D.; Um, I.H.; Elshani, M.; Armstrong, S.D.; Penrice-Randal, R.; Millar, T.; Lerpiniere, C.; Tagliavini, G.; Hartley, C.S.; et al. Tissue-Specific Immunopathology in Fatal COVID-19. Am. J. Respir. Crit. Care Med. 2021, 203, 192–201. [Google Scholar] [CrossRef]
  11. Saha, A.; Sharma, A.R.; Bhattacharya, M.; Sharma, G.; Lee, S.S.; Chakraborty, C. Tocilizumab: A Therapeutic Option for the Treatment of Cytokine Storm Syndrome in COVID-19. Arch. Med. Res. 2020, 51, 595–597. [Google Scholar] [CrossRef]
  12. Masotti, L.; Landini, G.; Panigada, G.; Grifoni, E.; Tarquini, R.; Cei, F.; Cimolato, B.; Vannucchi, V.; Di Pietro, M.; Piani, F.; et al. Predictors of poor outcome in tocilizumab treated patients with Sars-CoV-2 related severe respiratory failure: A multicentre real world study. Int. Immunopharmacol. 2022, 107, 108709. [Google Scholar] [CrossRef]
  13. WHO Rapid Evidence Appraisal for COVID-19 Therapies (REACT) Working Group; Shankar-Hari, M.; Vale, C.L.; Godolphin, P.J.; Fisher, D.; Higgins, J.; Spiga, F.; Savovic, J.; Tierney, J.; Baron, G.; et al. Association Between Administration of IL-6 Antagonists and Mortality Among Patients Hospitalized for COVID-19: A Meta-analysis. JAMA 2021, 326, 499–518. [Google Scholar] [CrossRef]
  14. Luo, L.; Luo, T.; Du, M.; Mei, H.; Hu, Y. Efficacy and safety of tocilizumab in hospitalized COVID-19 patients: A systematic review and meta-analysis. J. Infect. 2022, 84, 418–467. [Google Scholar] [CrossRef]
  15. Bartoletti, M.; Azap, O.; Barac, A.; Bussini, L.; Ergonul, O.; Krause, R.; Paño-Pardo, J.R.; Power, N.R.; Sibani, M.; Szabo, B.G.; et al. ESCMID COVID-19 living guidelines: Drug treatment and clinical management. Clin. Microbiol. Infect. 2022, 28, 222–238. [Google Scholar] [CrossRef]
  16. Vollmer, S.; Mateen, B.A.; Bohner, G.; Király, F.J.; Ghani, R.; Jonsson, P.; Cumbers, S.; Jonas, A.; McAllister, K.; Myles, P.; et al. Machine learning and artificial intelligence research for patient benefit: 20 critical questions on transparency, replicability, ethics, and effectiveness. BMJ 2020, 368, l6927. [Google Scholar] [CrossRef]
  17. Abd-Alrazaq, A.; Alajlani, M.; Alhuwail, D.; Schneider, J.; Al-Kuwari, S.; Shah, Z.; Hamdi, M.; Househ, M. Artificial Intelligence in the Fight Against COVID-19: Scoping Review. J. Med. Internet Res. 2020, 22, e20756. [Google Scholar] [CrossRef]
  18. Rasheed, J.; Jamil, A.; Hameed, A.A.; Al-Turjman, F.; Rasheed, A. COVID-19 in the Age of Artificial Intelligence: A Comprehensive Review. Interdiscip. Sci. 2021, 13, 153–175. [Google Scholar] [CrossRef]
  19. Syeda, H.B.; Syed, M.; Sexton, K.W.; Syed, S.; Begum, S.; Syed, F.; Prior, F.; Yu, F., Jr. Role of Machine Learning Techniques to Tackle the COVID-19 Crisis: Systematic Review. JMIR Med. Inform. 2021, 9, e23811. [Google Scholar] [CrossRef]
  20. Chen, C.; Dong, D.; Qi, B.; Petersen, I.R.; Rabitz, H. Quantum Ensemble Classification: A Sampling-Based Learning Control Approach. IEEE Trans. Neural Netw. Learn. Syst. 2017, 28, 1345–1359. [Google Scholar] [CrossRef]
  21. Chang, W.; Liu, Y.; Wu, X.; Xiao, Y.; Zhou, S.; Cao, W. A New Hybrid XGBSVM Model: Application for Hypertensive Heart Disease. IEEE Access 2019, 7, 175248–175258. [Google Scholar] [CrossRef]
  22. Rivera-Lopez, R.; Canul-Reich, J. Construction of near-optimal axis-parallel decision trees using a differential-evolution-based approach. IEEE Access 2018, 6, 5548–5563. [Google Scholar] [CrossRef]
  23. Ma, D.; Yuan, S.; Shang, J.; Liu, J.; Dai, L.; Kong, X.; Xu, F. The Automatic Detection of Seizure Based on Tensor Distance and Bayesian Linear Discriminant Analysis. Int. J. Neural Syst. 2021, 31, 2150006. [Google Scholar] [CrossRef]
  24. Xing, W.; Bei, Y. Medical Health Big Data Classification Based on KNN Classification Algorithm. IEEE Access 2020, 8, 28808–28819. [Google Scholar] [CrossRef]
  25. Yu, S.; Li, X.; Zhang, X.; Wang, H. The OCS-SVM: An objective-cost-sensitive SVM with sample-based misclassification cost invariance. IEEE Access 2019, 7, 118931–118942. [Google Scholar] [CrossRef]
  26. Han, J.; Kamber, M.; Pei, J. Data Mining: Concepts and Techniques, 3rd ed.; Morgan Kaufmann Publishers: Burlington, MA, USA, 2012. [Google Scholar]
  27. Hariharan, A.; Hakeem, A.R.; Radhakrishnan, S.; Reddy, M.S.; Rela, M. The Role and Therapeutic Potential of NF-kappa-B Pathway in Severe COVID-19 Patients. Inflammopharmacology 2021, 29, 91–100. [Google Scholar] [CrossRef]
  28. Nile, S.H.; Nile, A.; Qiu, J.; Li, L.; Jia, X.; Kai, G. COVID-19: Pathogenesis, cytokine storm and therapeutic potential of interferons. Cytokine Growth Factor Rev. 2020, 53, 66–70. [Google Scholar] [CrossRef]
  29. Blanco-Melo, D.; Nilsson-Payant, B.E.; Liu, W.C.; Uhl, S.; Hoagland, D.; Møller, R.; Jordan, T.X.; Oishi, K.; Panis, M.; Sachs, D.; et al. Imbalanced Host Response to SARS-CoV-2 Drives Development of COVID-19. Cell 2020, 181, 1036–1045.e9. [Google Scholar] [CrossRef]
  30. McElvaney, O.J.; McEvoy, N.L.; McElvaney, O.F.; Carroll, T.P.; Murphy, M.P.; Dunlea, D.M.; Ní Choileáin, O.; Clarke, J.; O’Connor, E.; Hogan, G.; et al. Characterization of the Inflammatory Response to Severe COVID-19 Illness. Am. J. Respir. Crit. Care Med. 2020, 202, 812–821. [Google Scholar] [CrossRef]
  31. Grasselli, G.; Zangrillo, A.; Zanella, A.; Antonelli, M.; Cabrini, L.; Castelli, A.; Cereda, D.; Coluccello, A.; Foti, G.; Fumagalli, R.; et al. Baseline Characteristics and Outcomes of 1591 Patients Infected with SARS-CoV-2 Admitted to ICUs of the Lombardy Region, Italy. JAMA 2020, 323, 1574–1581. [Google Scholar] [CrossRef]
  32. Osuchowski, M.F.; Winkler, M.S.; Skirecki, T.; Cajander, S.; Shankar-Hari, M.; Lachmann, G.; Monneret, G.; Venet, F.; Bauer, M.; Brunkhorst, F.M.; et al. The COVID-19 puzzle: Deciphering pathophysiology and phenotypes of a new disease entity. Lancet Respir. Med. 2021, 9, 622–642. [Google Scholar] [CrossRef]
  33. Ghanei, M.; Solaymani-Dodaran, M.; Qazvini, A.; Ghazale, A.H.; Setarehdan, S.A.; Saadat, S.H.; Ghobadi, H.; Hoseininia, S.; Elahikhah, M.; Samadi, A.H.; et al. The efficacy of corticosteroids therapy in patients with moderate to severe SARS-CoV-2 infection: A multicenter, randomized, open-label trial. Respir Res. 2021, 22, 245. [Google Scholar] [CrossRef] [PubMed]
  34. Zhang, X.; Shang, L.; Fan, G.; Gu, X.; Xu, J.; Wang, Y.; Huang, L.; Cao, B. The Efficacy and Safety of Janus Kinase Inhibitors for Patients with COVID-19: A Living Systematic Review and Meta-Analysis. Front. Med. 2022, 8, 800492. [Google Scholar] [CrossRef] [PubMed]
  35. Alunno, A.; Najm, A.; Machado, P.M.; Bertheussen, H.; Burmester, G.R.; Carubbi, F.; De Marco, G.; Giacomelli, R.; Hermine, O.; Isaacs, J.D.; et al. 2021 update of the EULAR points to consider on the use of immunomodulatory therapies in COVID-19. Ann. Rheum. Dis. 2022, 81, 34–40. [Google Scholar] [CrossRef]
  36. Duarte-Millán, M.A.; Mesa-Plaza, N.; Guerrero-Santillán, M.; Morales-Ortega, A.; Bernal-Bello, D.; Farfán-Sedano, A.I.; García de Viedma-García, V.; Velázquez-Ríos, L.; Frutos-Pérez, B.; De Ancos-Aracil, C.L.; et al. Prognostic factors and combined use of tocilizumab and corticosteroids in a Spanish cohort of elderly COVID-19 patients. J. Med. Virol. 2022, 94, 1540–1549. [Google Scholar] [CrossRef]
  37. Wardhani, S.O.; Fajar, J.K.; Soegiarto, G.; Wulandari, L.; Maliga, H.A.; Ilmawan, M.; Merysa, R.; Simamora, A.B.; Aini, Q.; Noviantari, K.; et al. The association between therapeutic plasma exchange and the risk of mortality among patients critically ill with COVID-19: A meta-analysis. F1000Research 2021, 10, 1280. [Google Scholar] [CrossRef]
  38. RECOVERY Collaborative Group. Tocilizumab in patients admitted to hospital with COVID-19 (RECOVERY): A randomised, controlled, open-label, platform trial. Lancet 2021, 397, 1637–1645. [Google Scholar] [CrossRef]
  39. Narain, S.; Stefanov, D.G.; Chau, A.S.; Weber, A.G.; Marder, G.; Kaplan, B.; Malhotra, P.; Bloom, O.; Liu, A.; Lesser, M.L.; et al. Comparative Survival Analysis of Immunomodulatory Therapy for Coronavirus Disease 2019 Cytokine Storm. Chest 2021, 159, 933–948. [Google Scholar] [CrossRef]
  40. Chober, D.; Aksak-Wąs, B.; Bobrek-Lesiakowska, K.; Budny-Finster, A.; Hołda, E.; Mieżyńska-Kurtycz, J.; Jamro, G.; Parczewski, M. Effectiveness of Tocilizumab in Patients with Severe or Critical Lung Involvement in COVID-19: A Retrospective Study. J. Clin. Med. 2022, 11, 2286. [Google Scholar] [CrossRef]
  41. Campbell, C.; Andersson, M.I.; Ansari, M.A.; Moswela, O.; Misbah, S.A.; Klenerman, P.; Matthews, P.C. Risk of Reactivation of Hepatitis B Virus (HBV) and Tuberculosis (TB) and Complications of Hepatitis C Virus (HCV) Following Tocilizumab Therapy: A Systematic Review to Inform Risk Assessment in the COVID-19 Era. Front. Med. 2021, 8, 706482. [Google Scholar] [CrossRef]
  42. Adamidi, E.S.; Mitsis, K.; Nikita, K.S. Artificial intelligence in clinical care amidst COVID-19 pandemic: A systematic review. Comput. Struct. Biotechnol. J. 2021, 19, 2833–2850. [Google Scholar] [CrossRef]
  43. Bottino, F.; Tagliente, E.; Pasquini, L.; Napoli, A.D.; Lucignani, M.; Figà-Talamanca, L.; Napolitano, A. COVID Mortality Prediction with Machine Learning Methods: A Systematic Review and Critical Appraisal. J. Pers. Med. 2021, 11, 893. [Google Scholar] [CrossRef]
  44. Ma, B.; Meng, F.; Yan, G.; Yan, H.; Chai, B.; Song, F. Diagnostic classification of cancers using extreme gradient boosting algorithm and multi-omics data. Comput. Biol. Med. 2020, 121, 103761. [Google Scholar] [CrossRef]
  45. REMAP-CAP Investigators; Gordon, A.C.; Mouncey, P.R.; Al-Beidh, F.; Rowan, K.M.; Nichol, A.D.; Arabi, Y.M.; Annane, D.; Beane, A.; van Bentum-Puijk, W.; et al. Interleukin-6 Receptor Antagonists in Critically Ill Patients with COVID-19. N. Engl. J. Med. 2021, 384, 1491–1502. [Google Scholar] [CrossRef]
  46. De Ardanaz, L.S.; Andreu-Ubero, J.M.; Navidad-Fuentes, M.; Ferrer-González, M.Á.; Del Valle, V.R.; Salcedo-Bellido, I.; Barrios-Rodríguez, R.; Cáliz-Cáliz, R.; Requena, P. Tocilizumab in COVID-19: Factors Associated with Mortality before and after Treatment. Front. Pharmacol. 2021, 12, 620187. [Google Scholar] [CrossRef]
  47. Eşkazan, A.E.; Balkan, İ.İ.; Demirbaş, K.C.; Ar, M.C.; Karaali, R.; Sekibağ, Y.; Mulamahmutoğlu, S.; Dumanlı, G.Y.; Çakmak, F.; Yurttaş, N.Ö.; et al. Tocilizumab in COVID-19: The Cerrahpaşa-PREDICT score. J. Infect. Chemother. 2021, 27, 1329–1335. [Google Scholar] [CrossRef]
  48. Heidari-Beni, F.; Vahedian-Azimi, A.; Shojaei, S.; Rahimi-Bashar, F.; Shahriary, A.; Johnston, T.P.; Sahebkar, A. The Level of Procalcitonin in Severe COVID-19 Patients: A Systematic Review and Meta-Analysis. Adv. Exp. Med. Biol. 2021, 1321, 277–286. [Google Scholar] [CrossRef]
  49. Lansbury, L.; Lim, B.; Baskaran, V.; Lim, W.S. Co-infections in people with COVID-19: A systematic review and meta-analysis. J. Infect. 2020, 81, 266–275. [Google Scholar] [CrossRef]
  50. Rosas, I.O.; Diaz, G.; Gottlieb, R.L.; Lobo, S.M.; Robinson, P.; Hunter, B.D.; Cavalcante, A.W.; Overcash, J.S.; Hanania, N.A.; Skarbnik, A.; et al. Tocilizumab and remdesivir in hospitalized patients with severe COVID-19 pneumonia: A randomized clinical trial. Intensive Care Med. 2021, 47, 1258–1270. [Google Scholar] [CrossRef]
  51. Mutua, V.; Henry, B.M.; Csefalvay, C.V.; Cheruiyot, I.; Vikse, J.; Lippi, G.; Bundi, B.; Mong’are, N. Tocilizumab in addition to standard of care in the management of COVID-19: A meta-analysis of RCTs. Acta Biomed. 2022, 93, e2022014. [Google Scholar] [CrossRef]
  52. Moosazadeh, M.; Mousavi, T. Combination therapy of tocilizumab and steroid for COVID-19 patients: A meta-analysis. J. Med. Virol. 2022, 94, 1350–1356. [Google Scholar] [CrossRef]
  53. Albuquerque, A.M.; Tramujas, L.; Sewanan, L.R.; Williams, D.R.; Brophy, J.M. Mortality Rates Among Hospitalized Patients with COVID-19 Infection Treated with Tocilizumab and Corticosteroids: A Bayesian Reanalysis of a Previous Meta-analysis. JAMA Netw. Open 2022, 5, e220548. [Google Scholar] [CrossRef]
  54. Lohse, A.; Klopfenstein, T.; Balblanc, J.C.; Royer, P.Y.; Bossert, M.; Gendrin, V.; Charpentier, A.; Bozgan, A.M.; Badie, J.; Bourgoin, C.; et al. Predictive factors of mortality in patients treated with tocilizumab for acute respiratory distress syndrome related to coronavirus disease 2019 (COVID-19). Microbes Infect. 2020, 22, 500–503. [Google Scholar] [CrossRef]
  55. Chamorro-de-Vega, E.; Rodriguez-Gonzalez, C.G.; Manrique-Rodríguez, S.; Lobato-Matilla, E.; García-Moreno, F.; Olmedo, M.; Correa-Rocha, R.; Valerio, M.; Aldámiz-Echevarria, T.; Machado, M.; et al. Clinical course of severe patients with COVID-19 treated with tocilizumab: Report from a cohort study in Spain. Expert Rev. Clin. Pharmacol. 2021, 14, 249–260. [Google Scholar] [CrossRef]
  56. Lakatos, B.; Szabo, B.G.; Bobek, I.; Gopcsa, L.; Beko, G.; Kiss-Dala, N.; Petrik, B.; Gaspar, Z.; Farkas, B.F.; Sinko, J.; et al. Laboratory parameters predicting mortality of adult in-patients with COVID-19 associated cytokine release syndrome treated with high-dose tocilizumab. Acta Microbiol. Immunol. Hung. 2021, 68, 145–152. [Google Scholar] [CrossRef]
  57. Song, Y.; Ye, Y.; Su, S.H.; Stephens, A.; Cai, T.; Chung, M.T.; Han, M.K.; Newstead, M.W.; Yessayan, L.; Frame, D.; et al. A digital protein microarray for COVID-19 cytokine storm monitoring. Lab Chip 2021, 21, 331–343. [Google Scholar] [CrossRef]
  58. Sinha, P.; Jafarzadeh, S.R.; Assoumou, S.A.; Bielick, C.G.; Carpenter, B.; Garg, S.; Harleen, S.; Neogi, T.; Nishio, M.J.; Sagar, M.; et al. The Effect of IL-6 Inhibitors on Mortality Among Hospitalized COVID-19 Patients: A Multicenter Study. J. Infect. Dis. 2021, 223, 581–588. [Google Scholar] [CrossRef]
  59. Sinha, P.; Mostaghim, A.; Bielick, C.G.; McLaughlin, A.; Hamer, D.H.; Wetzler, L.M.; Bhadelia, N.; Fagan, M.A.; Linas, B.P.; Assoumou, S.A.; et al. Early administration of interleukin-6 inhibitors for patients with severe COVID-19 disease is associated with decreased intubation, reduced mortality, and increased discharge. Int. J. Infect. Dis. 2020, 99, 28–33. [Google Scholar] [CrossRef]
  60. Eimer, J.; Vesterbacka, J.; Svensson, A.K.; Stojanovic, B.; Wagrell, C.; Sönnerborg, A.; Nowak, P. Tocilizumab shortens time on mechanical ventilation and length of hospital stay in patients with severe COVID-19: A retrospective cohort study. J. Intern. Med. 2021, 289, 434–436. [Google Scholar] [CrossRef]
  61. Gutiérrez-Gutiérrez, B.; Del Toro, M.D.; Borobia, A.M.; Carcas, A.; Jarrín, I.; Yllescas, M.; Ryan, P.; Pachón, J.; Carratalà, J.; Berenguer, J.; et al. Identification and validation of clinical phenotypes with prognostic implications in patients admitted to hospital with COVID-19: A multicentre cohort study. Lancet Infect. Dis. 2021, 21, 783–792. [Google Scholar] [CrossRef]
  62. Chen, H.; Xie, J.; Su, N.; Wang, J.; Sun, Q.; Li, S.; Jin, J.; Zhou, J.; Mo, M.; Wei, Y.; et al. Corticosteroid Therapy Is Associated with Improved Outcome in Critically Ill Patients with COVID-19 with Hyperinflammatory Phenotype. Chest 2021, 159, 1793–1802. [Google Scholar] [CrossRef] [PubMed]
  63. Lam, C.; Siefkas, A.; Zelin, N.S.; Barnes, G.; Dellinger, R.P.; Vincent, J.L.; Braden, G.; Burdick, H.; Hoffman, J.; Calvert, J.; et al. Machine Learning as a Precision-Medicine Approach to Prescribing COVID-19 Pharmacotherapy with Remdesivir or Corticosteroids. Clin. Ther. 2021, 43, 871–885. [Google Scholar] [CrossRef] [PubMed]
  64. Kuno, T.; Sahashi, Y.; Kawahito, S.; Takahashi, M.; Iwagami, M.; Egorova, N.N. Prediction of in-hospital mortality with machine learning for COVID-19 patients treated with steroid and remdesivir. J. Med. Virol. 2022, 94, 958–964. [Google Scholar] [CrossRef] [PubMed]
  65. Fernández-Delgado, M.; Cernadas, E.; Barro, S.; Amorim, D. Do we Need Hundreds of Classifiers to Solve Real World Classification Problems? J. Mach. Learn. Res. 2014, 15, 3133–3181. [Google Scholar]
Figure 1. The figure shows the scheme followed in the learning and testing process of this work.
Figure 1. The figure shows the scheme followed in the learning and testing process of this work.
Jcm 11 04729 g001
Figure 2. ROC curves for the five assessed machine learning predictors. Abbreviations: ROC: receiver operating characteristic, XGB: extreme gradient boost, KNN: K-nearest neighbour, DT: decision tree, SVM: support vector machine, BLDA: Bayesian linear discriminant analysis.
Figure 2. ROC curves for the five assessed machine learning predictors. Abbreviations: ROC: receiver operating characteristic, XGB: extreme gradient boost, KNN: K-nearest neighbour, DT: decision tree, SVM: support vector machine, BLDA: Bayesian linear discriminant analysis.
Jcm 11 04729 g002
Figure 3. The figure shows the radar plot of the training phase (left) and validation (right) for the prediction of mortality in COVID-19 patients treat to tocilizumab. Abbreviations: AUC: area under curve, MCC: Matthew’s correlation coefficient, XGB: extreme gradient boost, KNN: K-nearest neighbour, DT: decision tree, SVM: support vector machine, BLDA: Bayesian linear discriminant analysis.
Figure 3. The figure shows the radar plot of the training phase (left) and validation (right) for the prediction of mortality in COVID-19 patients treat to tocilizumab. Abbreviations: AUC: area under curve, MCC: Matthew’s correlation coefficient, XGB: extreme gradient boost, KNN: K-nearest neighbour, DT: decision tree, SVM: support vector machine, BLDA: Bayesian linear discriminant analysis.
Jcm 11 04729 g003
Table 1. Summary of the mean values and standard deviation of balanced accuracy, recall, precision, F1 score, AUC, MCC, DYI and Kappa index of the machine learning models and the proposed method implemented in this article.
Table 1. Summary of the mean values and standard deviation of balanced accuracy, recall, precision, F1 score, AUC, MCC, DYI and Kappa index of the machine learning models and the proposed method implemented in this article.
MethodsBalanced AccuracyRecallPrecisionF1 Score
SVM82.77 ± 0.4782.87 ± 0.5382.18 ± 0.5782.52 ± 0.62
BLDA80.91 ± 0.8181.02 ± 0.7880.31 ± 0.7580.66 ± 0.73
DT83.54 ± 0.6383.63 ± 0.6883.01 ± 0.6483.32 ± 0.62
KNN86.86 ± 0.5486.98 ± 0.5186.59 ± 0.4686.79 ± 0.47
XGB93.16 ± 0.2593.25 ± 0.3192.49 ± 0.2792.87 ± 0.29
SVM0.82 ± 0.0273.45 ± 0.5882.78 ± 0.6172.91 ± 0.59
BLDA0.80 ± 0.0271.79 ± 0.7380.92 ± 0.7271.89 ± 0.71
DT0.83 ± 0.0274.18 ± 0.6783.55 ± 0.6573.71 ± 0.68
KNN0.86 ± 0.0276.97 ± 0.4886.87 ± 0.4577.15 ± 0.46
XGB0.93 ± 0.0284.41 ± 0.2593.17 ± 0.2683.91 ± 0.28
Abbreviations: SVM: support vector machine, BLDA: Bayesian linear discriminant analysis, DT: decision tree, KNN: K-nearest neighbour, XGB: extreme gradient boost, AUC: area under curve, MCC: Matthew’s correlation coefficient, DYI: degenerated Younden index.
Table 2. Parameters used for the implementation of different machine learning methods.
Table 2. Parameters used for the implementation of different machine learning methods.
SVMC = 1.0
sigma = 0.5
Numerical tolerance = 0.001
Iteration limit = 100
Kernel function: Linear kernel, Gaussian, Quadratic and Cubic
BLDAKernel: Bayesian
DTMinimum number of instances in leaves = 4
Minimum number of instances in internal nodes = 6
Maximum depth = 100
KNNNumber of neighbours = 20
Distance metric: Euclidean
Weight: Uniform
XGBBase estimator: tree
Maximum number of splits = 20
Learning rate = 0.1
Number of learners = 50
Abbreviations: SVM: support vector machine, BLDA: Bayesian linear discriminant analysis, DT: decision tree, KNN: K-nearest neighbour, XGB: extreme gradient boost.
Table 3. Basal clinical data of patients. Data are n (%) or median (IQR), unless otherwise stated.
Table 3. Basal clinical data of patients. Data are n (%) or median (IQR), unless otherwise stated.
Number of patients67
Age (years) (IQR)65 (57–74.5)
Male (%)43 (64.2)
Exitus, n (yes %)24 (35.8)
Hospital admission (days) after tocilizumab administration (IQR)14 (10–29.5)
IMV, n (yes %)13 (19.4)
7-day mortality, n (yes %)7 (10.4)
21-day mortality, n (yes %)10 (14.9)
Antivirals drugs, n (yes %)30 (44.8)
Lopinavir/ritonavir, n (yes %)27 (40.3)
Remdesivir, n (yes %)3 (4.5)
Hydroxychloroquine, n (yes %)34 (50.7)
Interferon-beta, n (yes %)7 (10.4)
Anakinra, n (yes %)33 (49.2)
Baseline situation at the start of tocilizumab treatment requiring supplemental oxygen, n (yes %)60 (89.5)
Baseline situation at the start of tocilizumab treatment requiring IMV, n (yes %) 7 (10.4)
Smoker/ex-smoker, n (yes %)14 (20.9)
Diabetes, n (yes %)18 (26.9)
COPD, n (yes %)4 (5.9)
Arterial hypertension, n (yes %)34 (50.7)
Dyslipemia, n (yes %)21 (31.3)
Obesity [BMI ≥ 30 kg/m2], n (yes %)4 (5.9)
Ischemic heart disease, n (yes %)5 (7.5)
Chronic kidney disease, n (yes %)2 (2.9)
Lymphocytes (10 × 9/L) (IQR)0.95 (0.5–1.3)
CRP (mg/L) (IQR)12.5 (5.8–21.1)
LDH (U/L) (IQR)759 (538–934)
Procalcitonin (ng/mL) (IQR)3.49 (0.3–10.9)
Ferritin (µg/L) (IQR)828 (412–1388)
FiO2 (%) (IQR)37.5 (28–50)
PaFi (IQR)198.5 (135–251.5)
GPT (U/L) (IQR)39 (26–66.5)
GOT(U/L) (IQR)45 (31–67)
Abbreviations: IQR: interquartile range, IMV: invasive mechanical ventilation, COPD: chronic obstructive pulmonary disease, BMI: body mass index, CRP: C-reactive protein, LDH: lactate dehydrogenase, FiO2: inspired oxygen fraction, PaFi: ratio between arterial oxygen pressure and inspired oxygen fraction (PaO2/FiO2), GPT: glutamate-pyruvate transaminase, GOT: glutamate-oxaloacetate transaminase.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Ramón, A.; Zaragozá, M.; Torres, A.M.; Cascón, J.; Blasco, P.; Milara, J.; Mateo, J. Application of Machine Learning in Hospitalized Patients with Severe COVID-19 Treated with Tocilizumab. J. Clin. Med. 2022, 11, 4729.

AMA Style

Ramón A, Zaragozá M, Torres AM, Cascón J, Blasco P, Milara J, Mateo J. Application of Machine Learning in Hospitalized Patients with Severe COVID-19 Treated with Tocilizumab. Journal of Clinical Medicine. 2022; 11(16):4729.

Chicago/Turabian Style

Ramón, Antonio, Marta Zaragozá, Ana María Torres, Joaquín Cascón, Pilar Blasco, Javier Milara, and Jorge Mateo. 2022. "Application of Machine Learning in Hospitalized Patients with Severe COVID-19 Treated with Tocilizumab" Journal of Clinical Medicine 11, no. 16: 4729.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop