Next Article in Journal
Evidence-Based Aerobic Exercise Training in Metabolic-Associated Fatty Liver Disease: Systematic Review with Meta-Analysis
Previous Article in Journal
Pelvic Floor Morbidity Following Vaginal Delivery versus Cesarean Delivery: Systematic Review and Meta-Analysis
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
JCM
Volume 10
Issue 8
10.3390/jcm10081656
jcm-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Admission High-Sensitive Cardiac Troponin T Level Increase Is Independently Associated with Higher Mortality in Critically Ill Patients with COVID-19: A Multicenter Study

by
Romaric Larcher
1,2,*,
Noemie Besnard
3,
Aziz Akouz
4,
Emmanuelle Rabier
5,
Lauranne Teule
4,
Thomas Vandercamere
5,
Samuel Zozor
1,
Matthieu Amalric
3,
Racim Benomar
3,
Vincent Brunot
3,
Philippe Corne
3,
Olivier Barbot
4,
Anne-Marie Dupuy
1,
Jean-Paul Cristol
1,2,† and
Kada Klouche
2,3,†
1
Biochemistry and Hormonology Department, Lapeyronie Hospital, University Hospital of Montpellier, 34090 Montpellier, France
2
PhyMedExp, University of Montpellier, INSERM, CNRS, Arnaud de Villeneuve Hospital, University Hospital of Montpellier, 34090 Montpellier, France
3
Intensive Care Medicine Department, Lapeyronie Hospital, University Hospital of Montpellier, 34090 Montpellier, France
4
Intensive Care Unit, Hospital of Perpignan, 66000 Perpignan, France
5
Intensive Care Unit, Hospital of Narbonne, 11100 Narbonne, France
*
Author to whom correspondence should be addressed.
†
Contributed equally to this work as last authors.
J. Clin. Med. 2021, 10(8), 1656; https://doi.org/10.3390/jcm10081656
Submission received: 26 February 2021 / Revised: 8 April 2021 / Accepted: 10 April 2021 / Published: 13 April 2021
(This article belongs to the Section Intensive Care)
Browse Figures
Review Reports Versions Notes

Abstract

:
Background: In coronavirus disease 2019 (COVID-19) patients, increases in high-sensitive cardiac troponin T (hs-cTnT) have been reported to be associated with worse outcomes. In the critically ill, the prognostic value of hs-cTnT, however, remains to be assessed given that most previous studies have involved a case mix of non- and severely ill COVID-19 patients. Methods: We conducted, from March to May 2020, in three French intensive care units (ICUs), a multicenter retrospective cohort study to assess in-hospital mortality predictability of hs-cTnT levels in COVID-19 patients. Results: 111 laboratory-confirmed COVID-19 patients (68% of male, median age 67 (58–75) years old) were included. At ICU admission, the median Charlson Index, Simplified Acute Physiology Score II, and PaO2/FiO2 were at 3 (2–5), 37 (27–48), and 140 (98–154), respectively, and the median hs-cTnT serum levels were at 16.0 (10.1–31.9) ng/L. Seventy-five patients (68%) were mechanically ventilated, 41 (37%) were treated with norepinephrine, and 17 (15%) underwent renal replacement therapy. In-hospital mortality was 29% (32/111) and was independently associated with lower PaO2/FiO2 and higher hs-cTnT serum levels. Conclusions: At ICU admission, besides PaO2/FiO2, hs-cTnT levels may allow early risk stratification and triage in critically ill COVID-19 patients.

1. Introduction

Coronavirus disease 2019 (COVID-19), caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection, has spread worldwide, resulting in a dramatic pandemic [1]. The number of COVID-19 cases is still dramatically increasing all around the world, and health systems confronting the pandemic have to face intensive care unit (ICU) resource scarcity [2]. Early risk stratification and identification of predictive factors of mortality in critically ill patients are therefore an important matter of concern in order to determine suitability for transfer to the ICU and to perform optimal follow-up and management.
In ICU settings, COVID-19 has initially been mainly associated with acute respiratory distress syndrome (ARDS), but evidence of multiple organ dysfunction, including myocardial injury, has been reported [3,4,5,6]. Cardiac disease in patients with COVID-19 has therefore been recognized as a common condition associated with a more severe clinical course and increased mortality [7,8]. In the same line, increases in cardiac troponin (cTn), also known as myocardial injury, have been reported to be associated with worse outcomes in patients with COVID-19 [8,9,10,11,12,13,14,15]. However, the prognostic value of cardiac biomarkers in COVID-19 critically ill patients have not yet been clearly defined. Obviously, data in the ICU population remains scarce, since almost all studies have involved a case mix of non-severe and severe patients, or few ICU patients [16,17,18,19].
The aim of this multicentric study was to assess the prognostic value of high-sensitive cardiac troponin T (hs-cTnT) levels at ICU admission in patients with critical SARS-CoV-2 infection.

2. Materials and Methods

2.1. Study Design and Settings

This observational retrospective cohort study was carried out from 9 March 2020 to 3 May 2020 in three French ICUs. Investigators on each site collected prospectively clinical and biological data for all critically ill patients diagnosed with COVID-19. Patients were followed up until hospital discharge or death and three months later by phone call.

2.2. Participants

All consecutive patients with a SARS-CoV-2 infection confirmed by RT-PCR who were at least 18 years old and admitted to the ICU during the study period were included. Patients with suspected COVID-19 but not confirmed by SARS-CoV-2 RT-PCR and those who did not have an hs-cTnT measurement within the first 24 h of ICU admission were excluded.

2.3. Data Collection

Demographical and clinical data, including morbidities, were collected. Patients’ prior health status was evaluated by the Charlson Comorbidity Index [20]. Chronic heart disease was defined as a medical history of myocardial infarction, coronary ischemia treated by angioplasty and/or stents, cardiac surgery, or chronic heart failure (i.e., exertional or paroxysmal nocturnal dyspnea that has responded to digitalis, diuretics, or afterload-reducing agents). The severity of disease was assessed 24 h after ICU admission using the Simplified Acute Physiology Score (SAPS) II [21]. At ICU admission, the oxygen arterial partial pressure to the fraction of inspired oxygen (PaO2/FiO2) ratio [22] was recorded. Biological data were also collected at admission, particularly hs-cTnT, C reactive protein (CRP), and D-dimer. The D-dimer level was determined using D-Dimer HS 500 assay on ACL TOP 700 analyzer (Werfen, Barcelona, Spain). Procalcitonin (PCT) was determined using the BRAHMS PCT assay based on TRACE (Time-Resolved Amplified Cryptate Emission) on the Kryptor Gold© instrument (ThermoScientific, BRAHMS AG, Hennigsdorf, Germany). All other biological parameters, such as hs-cTnT, serum creatinine, CRP, and lactate, were performed on a Cobas 8000 instrument using e802 and c702 modules (Roche Diagnostic, Meylan, France).
Within the first five days of ICU admission, a transthoracic echocardiography was performed using a Vivid S70 ultrasound machine (GE Healthcare, Horton, Norway), equipped with a M5S (1.4–4.6MHz) cardiac probe allowing M-mode and two-dimensional measurements. Ventricular systolic function was assessed by measuring the ejection fraction (EF) from the mean of three measurements in different cardiac cycles using the biplane disc summation method and by calculation of fractional shortening, from the reduction of the right ventricular (LV) internal diameter during the cardiac cycle (modified Simpson method). Tricuspid annular plane systolic excursion (TAPSE) measurements were also performed using M-mode. Left ventricular dysfunction was defined as a left ventricular ejection fraction (LVEF) < 50%, whereas right ventricular dysfunction was defined with a TAPSE measurement < 15 mm. A cardiac dysfunction was considered to be present in case of left or/and right ventricular dysfunction.
Therapies instituted during ICU stay, including vasoactive drugs, invasive mechanical ventilation, renal replacement therapy, and extracorporeal membrane oxygenation (ECMO), were recorded.
The ICU length of stay and the outcome, including ICU, in-hospital, and 90-day mortalities, were recorded. Predictive factors of in-hospital mortality were then identified.

2.4. Statistical Analysis

Data are described as median and interquartile range (IQR) or number and percentage. The population was divided into two groups according to vital status at hospital discharge. Categorical variables were compared using the Chi-square test and continuous variables using the nonparametric Wilcoxon test. Mortality is displayed as Kaplan–Meier plots and compared using log-rank tests. Risk factors associated with in-hospital mortality and need for invasive mechanical ventilation were assessed using univariable and multivariable cox regression. For the multivariable analysis, variables were selected according to their statistical significance in univariable analysis (with a p-value < 0.2). Because of skewed distributions, biomarker values were log-transformed before modeling. Proportional hazard assumption was assessed by inspecting the scaled Shoenfeld residuals and regression splines. Hazard ratios (HRs) were given with a 95% confidence interval. The sensitivity and specificity of the factors associated with in-hospital mortality were assessed and receiver operating characteristic (ROC) curves were plotted. The optimal cut-off values were estimated by maximizing the Youden index. The risk factors associated with the occurrence of cardiac dysfunction were assessed using a logistic regression model. All tests were two-sided and a p-value less than 0.05 was considered statistically significant. R software version 4.0.2 (Free Software Foundation, Boston, MA, USA) was used for analyses.

3. Results

3.1. Study Population

Among the 184 patients admitted to ICUs with suspected COVID-19, 73 were excluded: 14 because of non-confirmed diagnosis and 59 because of missing data (Figure 1). One hundred and eleven patients (75 males, 68%) with a median age of 67 (58–75) years were then included in the study. As shown in Table 1, the median Charlson Index was 3 (2–5) and most of the patients (48%) had at least two comorbid conditions such as hypertension (47%), diabetes (31%), chronic heart diseases (20%), chronic pulmonary disease (20%), or chronic kidney disease (13%).
At ICU admission, the median SAPS II was 37 (27–48). The severity of patients’ illnesses was mainly related to respiratory failure, given that the median PaO2/FiO2 ratio at admission was 140 (98–154) and severe acute respiratory distress syndrome (ARDS) according to the Berlin definition occurred in almost one third of the patients (31%). Almost all patients (83%) included in the study were admitted to the ICU from the emergency department or after a hospital stay in the ward of ≤ 24 h.
The biological data at admission are summarized in Table 1. A huge elevation of CRP and D-dimer was observed at 153 (112–222) mg/L and 890 (572–1950) ng/mL, respectively. Levels of cTnT were increased over the 99th percentile (>14 ng/L) in 61 patients (55%) and above 52 ng/L in 18 patients (16%). The PCT values were low and mostly under the cutoff value of 0.5 ng/mL (median PCT at 0.4 (0.2–0.9) ng/mL).

3.2. Outcomes

During ICU stay, more than two third of patients (75 patients, 68%) required invasive mechanical ventilation, and among them, 38 (34% of the whole population and 51% of mechanically ventilated patients) needed—in addition—a prone positioning, and three required a veno-venous ECMO (3%) for refractory hypoxemia. Acute kidney injury at a KDIGO stage ≥ 1 occurred in 66 patients (60%) and 17 (15%) underwent renal replacement therapy. Norepinephrine (≥1 µg/kg/min) was administered in 41 (37%) patients, mainly in the case of vasoplegic shock, except in five patients who had mixed cardiogenic and vasoplegic shock. Dobutamine was administered in one patient. A bacterial infection (six pneumonia, two bacteremia, and one urosepsis) was documented concomitantly with the SARS-CoV-2 infection in nine patients treated with norepinephrine.
In 66 patients, echocardiography was performed, and the median LEVF was 60% (50–60%). A cardiac dysfunction was diagnosed in 15 (23%) of these 66 patients (Table S1). Two patients had pulmonary embolism without right ventricular dysfunction, shock, or hs-cTnT elevation.
Thirty-two patients died before hospital discharge, showing a mortality rate of 29%. Median ICU and hospital lengths of stay were five (2–13) and 27 (14–49) days; three months later, all discharged patients from the hospital were still alive (Table 1). The ninety-day survival probability was 72% (64–81%).

3.3. Factors Associated with In-Hospital Mortality

By univariable analysis, age, sex, and a high Charlson Index were significantly associated with in-hospital mortality. At admission, higher SAPS II, a lower PaO2/FiO2 ratio, and increased serum creatinine and hs-cTnT levels were also significantly associated with mortality (Table 1). During ICU stay, the requirement for vasoactive drugs and/or renal replacement therapy significantly worsen the outcome.
By multivariable analysis, only a low PaO2/FiO2 ratio and an elevated hs-cTnT serum level among the variables studied were independently associated with in-hospital mortality (Table 2).
Of note, the analysis of the 66 patients in which echocardiography was performed failed to demonstrate any association between hs-cTnT serum levels and cardiac function (p = 0.12) (Table S1).

3.4. ROC Curves of the Main Prognostic Factors of Mortality at ICU Admission

At ICU admission, age, hs-cTnT, and the PaO2/FiO2 ratio area under the curves (AUCs) for predicting in-hospital mortality were 0.794 (95% CI, 0.705–0.883), 0.792 (95% CI, 0.702–0.882), and 0.696 (95% CI, 0.589–0.802), respectively. The D-dimer and CRP levels showed an ROC-AUC of 0.609, (95% CI, 0.485–0.734) and 0.539 (95% CI, 0.410–0.669), respectively. The ROC curves are displayed in Figure 2.
Hs-cTnT has a sensitivity and a specificity at 69% and 79%, respectively, to predict in-hospital mortality using a cutoff value at 22 ng/L (Table 3). Age and the PaO2/FiO2 ratio had a higher specificity (81% and 82%, respectively) but lower sensitivity (63% and 50%, respectively) with a cutoff value at 74 years old and 100, respectively. D-dimer, at a cutoff value of 500 ng/L, showed the highest sensitivity (96%) but the lowest specificity (26%), and D-dimer ≥ 500 ng/L also had the highest negative predictive value (NPV) at 94%, whereas hs-cTnT ≥ 22 ng/L, age ≥ 74 years old, and PaO2/FiO2 ratio ≤ 100 had an NPV of 86%, 84%, and 80%, respectively. All factors had low positive predictive values (PPVs).
Ninety-day survival was significantly altered according to the level of hs-cTnT and the PaO2/FiO2 ratio. The corresponding Kaplan–Meier curves are displayed in Figure 3.

3.5. Factors Associated with Invasive Mechanical Ventilation Needs

By univariable analysis, age, high Charlson Index and SAPS II, a low PaO2/FiO2 ratio, need for vasoactive drugs, and increased serum hs-cTnT levels were significantly associated with invasive mechanical ventilation (Table S2).
By multivariable analysis, only higher SAPS II was independently associated with invasive mechanical ventilation (Table S2). None of studied biological variables were independently associated with the need for invasive mechanical ventilation.

4. Discussion

In this multicentric study, including 111 critically ill laboratory-confirmed COVID-19 patients, we observed an in-hospital mortality rate of 29%. We found that a low PaO2/FiO2 ratio and elevated hs-cTnT serum levels at ICU admission were independently associated with in-hospital mortality. All variables had a fair negative but low positive value to predict in-hospital mortality.
Previous studies have already reported that a lower PaO2/FiO2 ratio is associated with mortality in critical COVID-19 patients [1,12,23,24]. A lower PaO2/FiO2 ratio is also recognized as the major determinant predicting morbidity and mortality in patients with non-COVID-19 ARDS [22,25,26]. In addition, we showed in this study that an increase in hs-cTnT is associated with in-hospital mortality in critically ill COVID-19 patients. There is evidence of an association between cTnI levels and mortality in mild–severe COVID-19 patients [9,16,19,27]. Moreover, recent meta-analyses have concluded that increased cTnI levels are significantly associated with the most severe forms of COVID-19 [14,28]. Our observations confirmed these previous reports and are in line with the results recently reported by Demir et al. [29]. However, conflicting results have been reported by Metkus et al. [30], highlighting that the association between cTn elevation and in-hospital mortality of mechanically ventilated COVID-19 patients was weak after adjustment for cofounders, namely, age, sex, and multi-system organ dysfunction. Though they take into account normal values accordingly, the use of both cTnT and cTnI measurements may have introduced some bias in their results [30]. Nonetheless, in our study, hs-cTnT levels, as well as PaO2/FiO2, were independently associated with mortality, given that other organ dysfunctions, age, and gender were included in the multivariable analysis.
In critically ill patients with SARS-CoV-2 infection, several mechanisms could be involved in myocardial injury and the subsequent elevation of hs-cTn levels [31]. Acute myocardial infarction (types 1 and 2) may complicate COVID-19, as reported in acute bacterial or viral infections [32]. In our cohort, though hs-cTnT levels were frequently over the 99th percentile (>14 ng/L), they rarely crossed 52 ng/L, the cut-off value for myocardial infarction diagnosis [33]. In addition, electro/echocardiographic and hs-cTnT kinetics along ICU stay ruled out myocardial necrosis. COVID-19-associated myocarditis may also induce an increase in troponin [8,31,34,35,36]. The clinical features of acute myocarditis are heterogeneous, from new-onset heart failure to arrhythmic events, or infarct-like symptoms with usually preserved LVEF [37]. In our patients, the occurrence of acute myocarditis with preserved systolic function could explain the increase in hs-cTnT without echocardiographic abnormalities. Actually, there are limited reports on SARS-CoV-2-associated myocarditis, and a few autopsy series have shown virus in the myocardium [34,38]. Nonetheless, sepsis, cardiac adrenergic hyperstimulation, and pulmonary embolism could enhance troponin liberation [30,31,35]. Moreover, comorbid conditions such as coronary artery disease, chronic ischemic cardiomyopathy, or hypertension, which are common in COVID-19 patients [7,12,13], may also contribute to troponin elevation. Regardless, COVID-19 led to inflammation, endothelial dysfunction, hypercoagulation state, hypoxia, and hemodynamic instability that may induce myocardial injury and, consequently, troponin elevation [8,10,13,16,34,38,39].
Our results are in line with those previously reported in septic critically ill patients regarding both cTnT and cTnI levels [40,41,42,43], suggesting that cTnT levels may add to risk stratification and in a better manner than inflammatory or thrombotic markers in COVID-19 patients [9,16,17,18,19]. Though a recent meta-analysis have highlighted the prognostic value of increased CRP and D-dimer plasma levels in non-severe and severe COVID-19 [44], the value of such markers in ICU settings is flawed, given that they are frequently increased, as we observed in our patients [16,45]. Our results should prompt intensivists to pay attention to hs-cTnT levels, in addition to the PaO2/FiO2 ratio, among critically ill patients admitted with SARS-CoV-2 infection for triage and risk stratification. A higher admission severity score may also help to predict the need for invasive mechanical ventilation.
We must acknowledge some limitations to this study. First, it was limited by its retrospective design, inducing bias in data collection and result interpretation. Some hs-cTnT missing values forced us to exclude 59 patients, but this work is the largest multicenter analysis on the prognostic value of cTn in ICU settings. Second, hs-cTnT levels were not systematically monitored during ICU stay. Obviously, hs-cTnT kinetics would be very informative in both diagnosis and prognosis in this population. Last, echocardiography was performed in a small subset of included patients. Moreover, endomyocardial biopsy and cardiac magnetic resonance imaging might have been useful for diagnosis of acute myocarditis, particularly for those with preserved systolic function.

5. Conclusions

In conclusion, elevated troponin plasmatic concentrations are commonly observed in critically ill COVID-19 patients. We showed that, at admission, myocardial injury and a low PaO2/FiO2 ratio were strongly associated with in-hospital mortality. These variables may help for an early and best risk stratification in such patients. Further studies are, however, mandatory to confirm our results.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/jcm10081656/s1, Table S1: Echocardiographic evaluation in 66 patients with critical coronavirus disease 2019 (COVID-19); Table S2: Risk factors associated with invasive mechanical ventilation needs in critically ill COVID-19 patients.

Author Contributions

Conceptualization, R.L., J.-P.C., and K.K.; methodology, R.L., J.-P.C., and K.K.; software, R.L.; validation, A.-M.D., J.-P.C., and K.K.; formal analysis, R.L., A.-M.D., J.-P.C., and K.K.; investigation, R.L., N.B., A.A., E.R., L.T., T.V., and S.Z.; resources, R.L.; data curation, R.L.; writing—original draft preparation, R.L.; writing—review and editing, A.-M.D., J.-P.C., and K.K.; visualization, M.A., R.B., V.B., P.C. and O.B.; supervision, J.-P.C. and K.K.; project administration, R.L. and K.K. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

This study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Institutional Review Board of Montpellier University Hospital (IRB-MTP_2020_11_202000644, accessed on 16 November 2020). This study was also registered at ClinicalTrials.gov (NCT04641988), accessed on 23 November 2020).

Informed Consent Statement

The Institutional Review Board of Montpellier University Hospital waived the need for signed consent to participate, according to the French laws.

Data Availability Statement

The authors consent to share the collected data with others. Data will be provided to qualified investigators free of charge, after careful examination of required documents (summary of the research plan, request form, and IRB approval) by the study board of investigators. Data will be available immediately after the main publication and indefinitely.

Acknowledgments

This work is dedicated to the loving memory of Nils Kuster, talented pharmacist and biochemist.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Guan, W.J.; Ni, Z.Y.; Hu, Y.; Liang, W.H.; Ou, C.Q.; He, J.X.; Liu, L.; Shan, H.; Lei, C.L.; Hui, D.S.C.; et al. Clinical Characteristics of Coronavirus Disease 2019 in China. N. Engl. J. Med. 2020, 382, 1708–1720. [Google Scholar] [CrossRef] [PubMed]
  2. Alhazzani, W.; Møller, M.H.; Arabi, Y.M.; Loeb, M.; Gong, M.N.; Oczkowski, S.; Levy, M.M.; Derde, L.; Dzierba, A.; Du, B.; et al. Surviving Sepsis Campaign: Guidelines on the Management of Critically Ill Adults with Coronavirus Disease 2019 (COVID-19). Intensive Care Med. 2020, 104, 854–887. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Zheng, Y.-Y.; Ma, Y.-T.; Zhang, J.-Y.; Xie, X. COVID-19 and the cardiovascular system. Nat. Rev. Cardiol. 2020, 17, 259–260. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Madjid, M.; Safavi-Naeini, P.; Solomon, S.D.; Vardeny, O. Potential Effects of Coronaviruses on the Cardiovascular System: A Review. JAMA Cardiol. 2020, 5, 831. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Fried, J.A.; Ramasubbu, K.; Bhatt, R.; Topkara, V.K.; Clerkin, K.J.; Horn, E.; Rabbani, L.; Brodie, D.; Jain, S.S.; Kirtane, A.J.; et al. The Variety of Cardiovascular Presentations of COVID-19. Circulation 2020, 141, 1930–1936. [Google Scholar] [CrossRef] [Green Version]
  6. Li, L.; Hill, J.; Spratt, J.C.; Jin, Z. Myocardial injury in severe COVID-19: Identification and management. Resuscitation 2021, 160, 16–17. [Google Scholar] [CrossRef]
  7. Inciardi, R.M.; Adamo, M.; Lupi, L.; Cani, D.S.; Di Pasquale, M.; Tomasoni, D.; Italia, L.; Zaccone, G.; Tedino, C.; Fabbricatore, D.; et al. Characteristics and outcomes of patients hospitalized for COVID-19 and cardiac disease in Northern Italy. Eur. Heart J. 2020, 41, 1821–1829. [Google Scholar] [CrossRef]
  8. Shi, S.; Qin, M.; Cai, Y.; Liu, T.; Shen, B.; Yang, F.; Cao, S.; Liu, X.; Xiang, Y.; Zhao, Q.; et al. Characteristics and clinical significance of myocardial injury in patients with severe coronavirus disease 2019. Eur. Heart J. 2020, 41, 2070–2079. [Google Scholar] [CrossRef]
  9. Huang, C.; Wang, Y.; Li, X.; Ren, L.; Zhao, J.; Hu, Y.; Zhang, L.; Fan, G.; Xu, J.; Gu, X.; et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. Lancet 2020, 395, 497–506. [Google Scholar] [CrossRef] [Green Version]
  10. Zhou, F.; Yu, T.; Du, R.; Fan, G.; Liu, Y.; Liu, Z.; Xiang, J.; Wang, Y.; Song, B.; Gu, X.; et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: A retrospective cohort study. Lancet 2020, 395, 1054–1062. [Google Scholar] [CrossRef]
  11. Chen, R.; Liang, W.; Jiang, M.; Guan, W.; Zhan, C.; Wang, T.; Tang, C.; Sang, L.; Liu, J.; Ni, Z.; et al. Risk Factors of Fatal Outcome in Hospitalized Subjects With Coronavirus Disease 2019 From a Nationwide Analysis in China. Chest 2020, 158, 97–105. [Google Scholar] [CrossRef]
  12. Wang, Y.; Lu, X.; Li, Y.; Chen, H.; Chen, T.; Su, N.; Huang, F.; Zhou, J.; Zhang, B.; Yan, F.; et al. Clinical Course and Outcomes of 344 Intensive Care Patients with COVID-19. Am. J. Respir. Crit. Care Med. 2020, 201, 1430–1434. [Google Scholar] [CrossRef]
  13. Guo, T.; Fan, Y.; Chen, M.; Wu, X.; Zhang, L.; He, T.; Wang, H.; Wan, J.; Wang, X.; Lu, Z. Cardiovascular Implications of Fatal Outcomes of Patients With Coronavirus Disease 2019 (COVID-19). JAMA Cardiol. 2020, 5, 811–818. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Lippi, G.; Lavie, C.J.; Sanchis-Gomar, F. Cardiac troponin I in patients with coronavirus disease 2019 (COVID-19): Evidence from a meta-analysis. Prog. Cardiovasc. Dis. 2020, 63, 390–391. [Google Scholar] [CrossRef]
  15. Du, R.-H.; Liang, L.-R.; Yang, C.-Q.; Wang, W.; Cao, T.-Z.; Li, M.; Guo, G.-Y.; Du, J.; Zheng, C.-L.; Zhu, Q.; et al. Predictors of mortality for patients with COVID-19 pneumonia caused by SARS-CoV-2: A prospective cohort study. Eur. Respir. J. 2020, 55, 2000524. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  16. Wang, D.; Hu, B.; Hu, C.; Zhu, F.; Liu, X.; Zhang, J.; Wang, B.; Xiang, H.; Cheng, Z.; Xiong, Y.; et al. Clinical Characteristics of 138 Hospitalized Patients With 2019 Novel Coronavirus–Infected Pneumonia in Wuhan, China. JAMA 2020, 323, 1061–1069. [Google Scholar] [CrossRef] [PubMed]
  17. Arentz, M.; Yim, E.; Klaff, L.; Lokhandwala, S.; Riedo, F.X.; Chong, M.; Lee, M. Characteristics and Outcomes of 21 Critically Ill Patients With COVID-19 in Washington State. JAMA 2020, 323, 1612. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Bhatraju, P.K.; Ghassemieh, B.J.; Nichols, M.; Kim, R.; Jerome, K.R.; Nalla, A.K.; Greninger, A.L.; Pipavath, S.; Wurfel, M.M.; Evans, L.; et al. Covid-19 in Critically Ill Patients in the Seattle Region—Case Series. N. Engl. J. Med. 2020, 382, 2012–2022. [Google Scholar] [CrossRef] [PubMed]
  19. Ruan, Q.; Yang, K.; Wang, W.; Jiang, L.; Song, J. Clinical predictors of mortality due to COVID-19 based on an analysis of data of 150 patients from Wuhan, China. Intensiv. Care Med. 2020, 46, 846–848. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  20. Charlson, M.; Szatrowski, T.P.; Peterson, J.; Gold, J. Validation of a combined comorbidity index. J. Clin. Epidemiol. 1994, 47, 1245–1251. [Google Scholar] [CrossRef]
  21. Le Gall, J.-R.; Lemeshow, S.; Saulnier, F. A New Simplified Acute Physiology Score (SAPS II) Based on a European/North American Multicenter Study. JAMA 1993, 270, 2957–2963. [Google Scholar] [CrossRef]
  22. Force, A.D.T.; Ranieri, V.M.; Rubenfeld, G.D.; Thompson, B.T.; Ferguson, N.D.; Caldwell, E.; Fan, E.; Camporota, L.; Slutsky, A.S. Acute Respiratory Distress Syndrome: The Berlin Definition. JAMA 2012, 307, 2526–2533. [Google Scholar] [CrossRef]
  23. Karagiannidis, C.; Mostert, C.; Hentschker, C.; Voshaar, T.; Malzahn, J.; Schillinger, G.; Klauber, J.; Janssens, U.; Marx, G.; Weber-Carstens, S.; et al. Case characteristics, resource use, and outcomes of 10 021 patients with COVID-19 admitted to 920 German hospitals: An observational study. Lancet Respir. Med. 2020, 8, 853–862. [Google Scholar] [CrossRef]
  24. Grasselli, G.; Zangrillo, A.; Zanella, A.; Antonelli, M.; Cabrini, L.; Castelli, A.; Cereda, D.; Coluccello, A.; Foti, G.; Fumagalli, R.; et al. Baseline Characteristics and Outcomes of 1591 Patients Infected with SARS-CoV-2 Admitted to ICUs of the Lombardy Region, Italy. JAMA 2020, 323, 1574–1581. [Google Scholar] [CrossRef] [Green Version]
  25. Herridge, M.S.; Chu, L.M.; Matte, A.; Tomlinson, G.; Chan, L.; Thomas, C.; Friedrich, J.O.; Mehta, S.; Lamontagne, F.; Levasseur, M.; et al. The RECOVER Program: Disability Risk Groups and 1-Year Outcome after 7 or More Days of Mechanical Ventilation. Am. J. Respir. Crit. Care Med. 2016, 194, 831–844. [Google Scholar] [CrossRef]
  26. Villar, J.; Ambrós, A.; Soler, J.A.; Martínez, D.; Ferrando, C.; Solano, R.; Mosteiro, F.; Blanco, J.; Martín-Rodríguez, C.; Fernández, M.D.M.; et al. Age, PaO2/FIO2, and Plateau Pressure Score: A Proposal for a Simple Outcome Score in Patients with the Acute Respiratory Distress Syndrome. Crit. Care Med. 2016, 44, 1361–1369. [Google Scholar] [CrossRef] [PubMed]
  27. Gunduz, R.; Yildiz, B.S.; Ozdemir, I.H.; Cetin, N.; Ozen, M.B.; Bakir, E.O.; Ozgur, S.; Bayturan, O. CHA2DS2-VASc score and modified CHA2DS2-VASc score can predict mortality and intensive care unit hospitalization in COVID-19 patients. J. Thromb. Thrombolysis 2021. [Google Scholar] [CrossRef] [PubMed]
  28. Danwang, C.; Endomba, F.T.; Nkeck, J.R.; Wouna, D.L.A.; Robert, A.; Noubiap, J.J. A meta-analysis of potential biomarkers associated with severity of coronavirus disease 2019 (COVID-19). Biomark. Res. 2020, 8, 1–13. [Google Scholar] [CrossRef] [PubMed]
  29. Demir, O.M.; Ryan, M.; Cirillo, C.; Desai, N.; Pericao, A.; Sinclair, H.; Stylianidis, V.; Victor, K.; Alaour, B.; Jones, A.; et al. Impact and Determinants of High-Sensitivity Cardiac Troponin-T Concentration in Patients With COVID-19 Admitted to Critical Care. Am. J. Cardiol. 2021. [Google Scholar] [CrossRef] [PubMed]
  30. Metkus, T.S.; Sokoll, L.J.; Barth, A.S.; Czarny, M.J.; Hays, A.G.; Lowenstein, C.J.; Michos, E.D.; Nolley, E.P.; Post, W.S.; Resar, J.R.; et al. Myocardial Injury in Severe COVID-19 Compared with Non–COVID-19 Acute Respiratory Distress Syndrome. Circulation 2021, 143, 553–565. [Google Scholar] [CrossRef]
  31. Sandoval, Y.; Januzzi, J.L.; Jaffe, A.S. Cardiac Troponin for Assessment of Myocardial Injury in COVID-19. J. Am. Coll. Cardiol. 2020, 76, 1244–1258. [Google Scholar] [CrossRef]
  32. Musher, D.M.; Abers, M.S.; Corrales-Medina, V.F. Acute Infection and Myocardial Infarction. N. Engl. J. Med. 2019, 380, 171–176. [Google Scholar] [CrossRef]
  33. Collet, J.-P.; Thiele, H.; Barbato, E.; Barthélémy, O.; Bauersachs, J.; Bhatt, D.L.; Dendale, P.; Dorobantu, M.; Edvardsen, T.; Folliguet, T.; et al. 2020 ESC Guidelines for the management of acute coronary syndromes in patients presenting without persistent ST-segment elevation. Eur. Heart J. 2020, 79. [Google Scholar] [CrossRef]
  34. Doyen, D.; Moceri, P.; Ducreux, D.; Dellamonica, J. Myocarditis in a patient with COVID-19: A cause of raised troponin and ECG changes. Lancet 2020, 395, 1516. [Google Scholar] [CrossRef]
  35. Imazio, M.; Klingel, K.; Kindermann, I.; Brucato, A.; De Rosa, F.G.; Adler, Y.; De Ferrari, G.M. COVID-19 pandemic and troponin: Indirect myocardial injury, myocardial inflammation or myocarditis? Heart 2020, 106, 1127–1131. [Google Scholar] [CrossRef]
  36. Lala, A.; Johnson, K.W.; Januzzi, J.L.; Russak, A.J.; Paranjpe, I.; Richter, F.; Zhao, S.; Somani, S.; Van Vleck, T.; Vaid, A.; et al. Prevalence and Impact of Myocardial Injury in Patients Hospitalized With COVID-19 Infection. J. Am. Coll. Cardiol. 2020, 76, 533–546. [Google Scholar] [CrossRef]
  37. Caforio, A.L.P.; Pankuweit, S.; Arbustini, E.; Basso, C.; Gimeno-Blanes, J.; Felix, S.B.; Fu, M.; Heliö, T.; Heymans, S.; Jahns, R.; et al. Current state of knowledge on aetiology, diagnosis, management, and therapy of myocarditis: A position statement of the European Society of Cardiology Working Group on Myocardial and Pericardial Diseases. Eur. Heart J. 2013, 34, 2636–2648. [Google Scholar] [CrossRef]
  38. Jonigk, D.; Märkl, B.; Helms, J. COVID-19: What the clinician should know about post-mortem findings. Intensiv. Care Med. 2021, 47, 86–89. [Google Scholar] [CrossRef] [PubMed]
  39. Klok, F.A.; Kruip, M.J.H.A.; van der Meer, N.J.M.; Arbous, M.S.; Gommers, D.A.M.P.J.; Kant, K.M.; Kaptein, F.H.J.; van Paassen, J.; Stals, M.A.M.; Huisman, M.V.; et al. Incidence of thrombotic complications in critically ill ICU patients with COVID-19. Thromb. Res. 2020, 191, 145–147. [Google Scholar] [CrossRef]
  40. Klouche, K.; Pommet, S.; Amigues, L.; Bargnoux, A.S.; Dupuy, A.M.; Machado, S.; Serveaux-Delous, M.; Morena, M.; Jonquet, O.; Cristol, J.P. Plasma Brain Natriuretic Peptide and Troponin Levels in Severe Sepsis and Septic Shock: Relationships with Systolic Myocardial Dysfunction and Intensive Care Unit Mortality. J. Intensiv. Care Med. 2012, 29, 229–237. [Google Scholar] [CrossRef] [PubMed]
  41. Kim, J.-S.; Kim, M.; Kim, Y.-J.; Ryoo, S.M.; Sohn, C.H.; Ahn, S.; Kim, W.Y. Troponin Testing for Assessing Sepsis-Induced Myocardial Dysfunction in Patients with Septic Shock. J. Clin. Med. 2019, 8, 239. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  42. Gayat, E.; Cariou, A.; Deye, N.; Vieillard-Baron, A.; Jaber, S.; Damoisel, C.; Lu, Q.; Monnet, X.; Rennuit, I.; Azoulay, E.; et al. Determinants of long-term outcome in ICU survivors: Results from the FROG-ICU study. Crit. Care 2018, 22, 1–10. [Google Scholar] [CrossRef] [Green Version]
  43. Landesberg, G.; Jaffe, A.S.; Gilon, D.; Levin, P.D.; Goodman, S.; Abu-Baih, A.; Beeri, R.; Weissman, C.; Sprung, C.L.; Landesberg, A. Troponin Elevation in Severe Sepsis and Septic Shock: The Role of Left Ventricular Diastolic Dysfunction and Right Ventricular Dilatation. Crit. Care Med. 2014, 42, 790–800. [Google Scholar] [CrossRef] [PubMed]
  44. Moutchia, J.; Pokharel, P.; Kerri, A.; McGaw, K.; Uchai, S.; Nji, M.; Goodman, M. Clinical laboratory parameters associated with severe or critical novel coronavirus disease 2019 (COVID-19): A systematic review and meta-analysis. PLoS ONE 2020, 15, e0239802. [Google Scholar] [CrossRef] [PubMed]
  45. Auld, S.C.; Caridi-Scheible, M.; Blum, J.M.; Robichaux, C.; Kraft, C.; Jacob, J.T.; Jabaley, C.S.; Carpenter, D.; Kaplow, R.; Hernandez-Romieu, A.C.; et al. ICU and Ventilator Mortality Among Critically Ill Adults with Coronavirus Disease 2019. Crit. Care Med. 2020. [Google Scholar] [CrossRef]
Jcm 10 01656 g001 550
Figure 1. Flow chart of the study population. ICU, intensive care unit; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.
Figure 1. Flow chart of the study population. ICU, intensive care unit; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.
Jcm 10 01656 g001
Jcm 10 01656 g002 550
Figure 2. Receiver operating characteristic (ROC) curves of age (purple semi-dashed line), initial PaO2/FiO2 ratio (blue dashed line), initial high-sensitive cardiac troponin T (hs-cTnT, red line), initial D-Dimer (green dotted line), and initial C reactive protein (CRP, orange dashed and dotted line) for predicting in-hospital mortality in critically ill patients with COVID-19.
Figure 2. Receiver operating characteristic (ROC) curves of age (purple semi-dashed line), initial PaO2/FiO2 ratio (blue dashed line), initial high-sensitive cardiac troponin T (hs-cTnT, red line), initial D-Dimer (green dotted line), and initial C reactive protein (CRP, orange dashed and dotted line) for predicting in-hospital mortality in critically ill patients with COVID-19.
Jcm 10 01656 g002
Jcm 10 01656 g003a 550Jcm 10 01656 g003b 550
Figure 3. Kaplan–Meier curve of 90-day survival after ICU admission of critically ill patients with COVID-19: (A) hs-cTnT < 22 ng/L versus hs-cTnT > 22 ng/L; (B) no acute respiratory distress syndrome (ARDS) (PaO2/FiO2 >300) versus mild (PaO2/FiO2 = 201–300), moderate (PaO2/FiO2 = 101–200), and severe ARDS (PaO2/FiO2 ≤ 100).
Figure 3. Kaplan–Meier curve of 90-day survival after ICU admission of critically ill patients with COVID-19: (A) hs-cTnT < 22 ng/L versus hs-cTnT > 22 ng/L; (B) no acute respiratory distress syndrome (ARDS) (PaO2/FiO2 >300) versus mild (PaO2/FiO2 = 201–300), moderate (PaO2/FiO2 = 101–200), and severe ARDS (PaO2/FiO2 ≤ 100).
Jcm 10 01656 g003aJcm 10 01656 g003b
Table
Table 1. Baseline characteristics of the study population and mortality.
Table 1. Baseline characteristics of the study population and mortality.
Total (n = 111)Non-Survivor (n = 32)Survivor (n = 79)p-Value
Age (years), median (IQR)67 (58–75)75 (67–81)63.0 (56–72)<0.001
Male, n (%)75 (68%)16 (50%)59 (75%)0.01
BMI 1 (kg/m²), median (IQR)28 (25–32)26 (23–31)28 (25–32)0.57
Hypertension, n (%)52 (47%)18 (56%)34 (43%)0.21
Diabetes, n (%)33 (31%)12 (38%)22 (28%)0.32
Chronic heart disease, n (%)22 (20%)10 (31%)12 (15%)0.06
Chronic kidney disease, n (%)14 (13%)8 (25%)6 (8%)0.02
Chronic pulmonary disease, n (%)22 (20%)11 (34%)11 (14%)0.02
Charlson Index, median (IQR)3 (2–5)5 (4–8)3 (2–4)<0.001
SAPS II 2, median (IQR)37 (27–48)43 (34–54)33 (25–46)0.01
PaO2/FiO2, median (IQR)140 (98–154)100 (85–139)160 (105–206)0.003
PaO2/FiO2 ≤100, n (%)34 (31%)17 (53%)17 (22%)0.002
Invasive mechanical ventilation, n (%)75 (68%)25 (78%)50 (63%)0.14
  Duration (days), median (IQR)10 (5–19)9 (6–16)12 (5–18)-
Prone positioning, n (%)38 (34%)14 (44%)24 (30%)0.02
ECMO 3, n (%)3 (3%)1 (3%)2 (3%)0.86
  Duration (days), median (IQR)17 (14–20)---
Norepinephrine (>1 µg/kg/min), n (%)41 (37%)17 (53%)24 (30%)0.03
  Duration (days), median (IQR)7 (4–12)9 (5–15)5 (3–10)-
RRT 4, n (%)17 (15%)10 (31%)7 (9%)0.005
  Duration (days), median (IQR)14 (4–31)10 (3–19)22 (10–38)-
Creatininemia (µmol/L), median (IQR)84 (62–111)92 (72–168)82 (57–101)0.01
hs-cTnT 5 (ng/L), median (IQR)16.0 (10.2–31.9)35.0 (16.8–106.0)12.5 (7.5–20.0)<0.001
CRP 6 (mg/L), median (IQR)153 (112–222)134 (93–215)154 (115–223)0.45
PCT 7 (ng/mL), median (IQR)0.4 (0.2–0.9)0.6 (0.3–1.3)0.3 (0.2–0.9)0.11
Lactatemia (mmol/L), median (IQR)1.3 (0.9–1.7)1.3 (0.8–1.8)1.3 (1.0–1.6)0.62
D-dimer (ng/mL), median (IQR)890 (572–1950)1340 (658–1958)859 (497–1861)0.09
LVEF 8 (%), median (IQR)60 (50–60)55 (50–60)60 (50–60)0.28
TAPSE 9 (mm), median (IQR)18 (16–22)18 (14–23)18 (17–22)0.49
ICU LOS 10 (days), median (IQR)10 (6–18)8 (5–17)10 (6–18)-
Hospital LOS (days), median (IQR)19 (11–31)10 (7–22)21 (13–34)-
Death, n (%)32 (29%)---
1 BMI, body mass index; 2 SAPS II, Simplified Acute Physiology Score II; 3 ECMO, extracorporeal membrane oxygenation; 4 RRT, renal replacement therapy; 5 hs-cTnT, high-sensitive cardiac troponin T; 6 CRP, C reactive protein; 7 PCT, procalcitonin; 8 LVEF, left ventricular ejection fraction (evaluated in a subset of 66 patients); 9 TAPSE, tricuspid annular plane systolic excursion (evaluated in a subset of 66 patients); 10 ICU LOS, intensive care unit length of stay.
Table
Table 2. Factors associated with in-hospital mortality in critically ill COVID-19 patients.
Table 2. Factors associated with in-hospital mortality in critically ill COVID-19 patients.
Univariable Analysis
Hazard Ratio (IC 95%)		p-ValueMultivariable Analysis
Hazard Ratio (IC 95%)		p-Value
Age1.1 (1.1–1.2)0.011.06 (0.99–1.14)0.11
Male sex0.4 (0.2–0.81)<0.0010.50 (0.21–1.05)0.64
BMI 10.99 (0.93–1.1)0.68    
Smokers0.5 (0.068–3.6)0.49    
Charlson Index1.3 (1.2–1.4)<0.0014.46 (0.26–75.55)0.30
SAPS II 210 (1.7–58)0.011.06 (0.08–13.36)0.96
PaO2/FiO2 ≤1003.4 (1.7–6.8)<0.0014.65 (1.81–11.97)0.001
Invasive mechanical ventilation 1.8 (0.76–4.1)0.192.88 (0.76–10.97)0.12
Norepinephrine (>1 µg/kg/min)2.1 (1–4.1)0.041.08 (0.30–2.84)0.90
Renal replacement therapy2.9 (1.4–6.2)0.0051.58 (0.58–4.31)0.37
Creatininemia (log10)8.5 (2.3–32)0.0021.45 (0.20–10.73)0.71
Hs-cTnT 3 (log10)4.3 (2.5–7.2)<0.0014.96 (1.92–12.86)<0.001
CRP 4 (log10)0.83 (0.37–1.9)0.65    
PCT 5 (log10)1.7 (1–2.9)0.051.60 (0.60–4.25)0.3
Lactatemia (log10)1.8 (0.21–16)0.58    
D-dimer (log10)2.4 (0.81–6.9)0.121.21 (0.17–3.95)0.8
1 BMI, body mass index; 2 SAPS II, Simplified Acute Physiology Score II; 3 hs-cTnT, high-sensitive cardiac troponin T; 4 CRP, C reactive protein; 5 PCT, procalcitonin.
Table
Table 3. Sensibility, specificity, positive, and negative predictive values for mortality prediction in critically ill patients with COVID-19.
Table 3. Sensibility, specificity, positive, and negative predictive values for mortality prediction in critically ill patients with COVID-19.
Sensibility (%)Specificity (%)PPV 1 (%)NPV 2 (%)
age ≥ 74 years old63815784
hs-cTnT 3 ≥ 22 ng/L69795686
PaO2/FiO2 ≤ 11566735084
PaO2/FiO2 ≤ 10050825380
D-dimer ≥ 500 ng/L96263494
CRP 4 ≥ 135 mg/L52683978
1 PPV, positive predictive value; 2 NPV, negative predictive value; 3 hs-cTnT, high-sensitive cardiac troponin T; 4 CRP, C reactive protein. The optimal cutoff value of each variable estimated by maximizing the Youden index was selected, except for PaO2/FiO2 ≤ 100, which is the threshold of the Berlin definition for severe acute respiratory distress syndrome.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Larcher, R.; Besnard, N.; Akouz, A.; Rabier, E.; Teule, L.; Vandercamere, T.; Zozor, S.; Amalric, M.; Benomar, R.; Brunot, V.; et al. Admission High-Sensitive Cardiac Troponin T Level Increase Is Independently Associated with Higher Mortality in Critically Ill Patients with COVID-19: A Multicenter Study. J. Clin. Med. 2021, 10, 1656. https://doi.org/10.3390/jcm10081656

AMA Style

Larcher R, Besnard N, Akouz A, Rabier E, Teule L, Vandercamere T, Zozor S, Amalric M, Benomar R, Brunot V, et al. Admission High-Sensitive Cardiac Troponin T Level Increase Is Independently Associated with Higher Mortality in Critically Ill Patients with COVID-19: A Multicenter Study. Journal of Clinical Medicine. 2021; 10(8):1656. https://doi.org/10.3390/jcm10081656

Chicago/Turabian Style

Larcher, Romaric, Noemie Besnard, Aziz Akouz, Emmanuelle Rabier, Lauranne Teule, Thomas Vandercamere, Samuel Zozor, Matthieu Amalric, Racim Benomar, Vincent Brunot, and et al. 2021. "Admission High-Sensitive Cardiac Troponin T Level Increase Is Independently Associated with Higher Mortality in Critically Ill Patients with COVID-19: A Multicenter Study" Journal of Clinical Medicine 10, no. 8: 1656. https://doi.org/10.3390/jcm10081656

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop