The COMPASS-COVID-19-ICU Study: Identification of Factors to Predict the Risk of Intubation and Mortality in Patients with Severe COVID-19

Round 1

Reviewer 1 Report

This is an interesting article with an excellent summary of the coagulopathy of Covid.  I think it would be helpful if there were a diagram included which illustrates the mechanisms of the activation of Covid, so well described in the text.  There are numerous diagrams in the literature, which can be taken and placed in the article.

This is a single hospital experience and a nonrandomized observational trial where patients who are in the ICU are compared to healthy patients who do not have Covid. It should include comment that the protein S and factor VII are disproportionately decreased compared to protein C, and the other clotting factors. Why do the authors feel that is?  Anti-thrombin, the hereditary thrombophilia usually most correlated with thrombosis, is not diminished greatly.  A large number of predispositions to thrombosis are sampled and six factors emerge, which can be placed together in a risk assessment model to distinguish ICU patients who at highest risk of intubation and death

How do the authors envision doctors incorporating the information from this article in ICU patients? Tissue factor and protein S seem to have the2 greatest difference sverses the controls. Why do the authors think that it?

 Would they intubate them sooner or would they give him a higher dose of steroids or would they anti initiate IL-6 treatment sooner.  Complement is not emphasized enough in the article as complement activates directly through C5a the coagulation cascade, chemokines, cytokines NETs etc. In figure one, are the different percentages an indication of how much each factor influences the overall c statistic or area under the curve. This point should be clarified into the legend.

In table 3 the confidence Intervals are shown regarding the top six features, it would be good to include the odds or risk ratio for the factors as well. That would apply to the mortality risk in table four as well

Please explain why less than ten variables should be used why ten is the number. Why do the authors feel the six factors which emerged did so?

Author Response

Reviewer 1

1. This is an interesting article with an excellent summary of the coagulopathy of Covid.  I think it would be helpful if there were a diagram included which illustrates the mechanisms of the activation of Covid, so well described in the text.  There are numerous diagrams in the literature, which can be taken and placed in the article.

We asked for the Editors’s permission for the Figure 1 of the article Gerotziafas GT, Catalano M, Colgan MP, et al. Guidance for the Management of Patients with Vascular Disease or Cardiovascular Risk Factors and COVID-19: Position Paper from VAS-European Independent Foundation in Angiology/Vascular Medicine. Thromb Haemost. 2020;120(12):1597-1628. Nevertheless the procedure to get it seems to be long and we prefere to publish rapidly this paper.

2. This is a single hospital experience and a nonrandomized observational trial where patients who are in the ICU are compared to healthy patients who do not have Covid. It should include comment that the protein S and factor VII are disproportionately decreased compared to protein C, and the other clotting factors. Why do the authors feel that is?  Anti-thrombin, the hereditary thrombophilia usually most correlated with thrombosis, is not diminished greatly.  A large number of predispositions to thrombosis are sampled and six factors emerge, which can be placed together in a risk assessment model to distinguish ICU patients who at highest risk of intubation and death

We added a paragraph in the Discussion Section where we discuss the alteration of the clotting factors and natural coagulation inhibitors. “Critically ill patients with COVID-19 at the first day of ICU admission presented significant alterations of clotting factors and natural coagulation inhibitors. Patients, as compared to healthy controls, showed significant decrease of factor XII which is the origin of aPTT prolongation. Patients also showed a significant decrease of factor VIIa which was related with the prolongation of PT. The decrease of FVIIa could result from factor VII  consumption following exaggerated TF pathway activation and/or increased inhibition of factor VIIa by AT. The consumption of AT observed in the patients’ cohort favors the second hypothesis which needs to be explored by measuring the factor VIIa-AT complexes in patients’ plasma. The data presented herein underline the importance of exacerbated activation of contact phase and intrinsic clotting pathway and TF pathway in the disease worsening process.(39-42)  In addition, patients with critical COVID-19 showed an important consumption of the natural coagulation inhibitors AT, protein C and protein S which shifts the equilibrium of coagulation towards hypercoagulability. Nevertheless, this shift was not reflected on thrombin generation measured with calibrated automated thrombogram most probably because of the increase of the TFPI levels as a consequence of endothelial cell activation. “

3. How do the authors envision doctors incorporating the information from this article in ICU patients? Tissue factor and P- Selectin seem to have the2 greatest difference sverses the controls. Why do the authors think that it?

Thank you for this question. Accurate prediction requires the full set of reduced variables. It is possible that a single multiplex immunoassay could be constructed to measure the identified biomarkers and predict outcome shortly after admission. Such prediction would aid resource mobilization, discussions with patients’ and their families, and help stratify patients for clinical trials. So, we modified accordingly the conclusion of the article. “In conclusion, this study provides compelling evidence showing that critical COVID-19 is related with severe endothelial cell activation and hypercoagulability or-chestrated in the context of inflammation. This study led to comprehensive and accurate predictive model for the evaluation of the risk of mechanical ventilation and death in patients with critical COVID-19. Accurate prediction requires the full set of reduced variables. The assessment of patients with critical COVID-19 with COMPASS- COVID-19-ICU score It is possible that a single multiplex immunoassay could be con-structed to measure the identified biomarkers and predict outcome shortly after admis-sion. Such prediction with COMPASS- COVID-19-ICU would aid resource mobilization, discussions with patients’ and their families, and help stratify patients for clinical trials .and to apply is feasible in the context of tertiary hospitals and could be a useful strategy for early identification of patients at risk of intubation or death. In this context it could be placed in the diagnostic procedure of personalized medical management and prompt therapeutic intervention.”

4. Would they intubate them sooner or would they give him a higher dose of steroids or would they anti initiate IL-6 treatment sooner.  cytokines NETs etc. In figure one, are the different percentages an indication of how much each factor influences the overall c statistic or area under the curve. This point should be clarified into the legend.

Our goal in this study was to identify accurate biomarkers that predict a given outcome (intubation or mortality). A future prospective clinical trial could be initiated to validate the identified biomarkers.  With regards to the percentages, they represent how much each feature contributes to the overall model and its calculated metrics (e.g., classification accuracy, area under the curve).

5. In Table 3 the confidence Intervals are shown regarding the top six features, it would be good to include the odds or risk ratio for the factors as well. That would apply to the mortality risk in table four as well

We thank the reviewer for this comment, but the odds or risk ratios cannot be included for the following reasons. First, calculation of the odds ratios for the data (continuous variables and binary outcomes) require regression-based methodologies, which would lead to confusion for readers due to the difference in classifiers. Second, odds ratios in the tables can be misinterpreted as representing the odds ratio for the random forest classifier which would be incorrect. Third, the regression-based odds ratios that combine all variables will be different from population odds ratios that independently assess each variable. Lastly, it is important to note that as the prevalence of intubation and mortality in our sample is greater than 10%, meaning the odds ratio will not accurately represent the data.

6. Please explain why less than ten variables should be used why ten is the number. Why do the authors feel the six factors which emerged did so?

Our goal was to find the least number of variables that provide the highest accuracy, thereby creating a model that can be translated into practice. With added variables, the accuracy does not improve and the model reduces the “ease of use”. Functionally, when conducting feature reduction using the Boruta algorithm, we do not manually configure the algorithm to produce a certain number of features. The algorithm produces a reduced set of features that provides the best model based on a complex methodology, which includes shadow (dummy/random) features to test each feature’s importance and interactions with other features. In our case, for intubation, the algorithm produced 6 features and for mortality, it produced 4 features. Thus, the number of features produced was determined solely by the algorithm.

Reviewer 2 Report

The article "The COMPASS-COVID19-ICU study: identification of factors to predict the risk of intubation and mortality in patients with severe COVID-19" by Gerotziafas described an analysis of the biomarker data obtained from patients with severe COVID-19. This paper focussed on establishing a correlation between the biomarkers (on hypercoagulation, endothelial cell activation and inflammation) with patient outcome (measured by intubation and mortality). This work has generated the COMPASS-COVID19-ICU score based on P-selectin, D-dimer, free TFPI, TF activity, IL-6 and FXII, age and duration of hospitalization. This score provides a sensitive and specific prediction on intubation and death (0.9 and 0.92, respectively) in COVID-19 patients. It is an exciting study that will collectively contribute to better COVID patient cares in the future.

Listed below requires authors' attention:

1. Would you mind providing keys to abbreviations used throughout, for example, Choc, MV and EER?
2. Would you mind stating or providing information on if these patients were fully vaccinated?
3. Please include cutoff values for Figure 1A and 2A in the figure legends.
4. Please also state in the discussion if these patients responded to the intensified antithrombotic treatment, ventilation, etc., as suggested by the author ln 339-340, if they were the treatments provided.
5. The authors had included the age and duration of hospitalization plus the top 6 biomarkers in calculating the COMPASS-COVID19-ICU score. However, Figure 1 presents data on all six biomarkers; whereas, Figure 2 shows two biomarkers together with the age and MV duration. It is not very consistent. Can the authors please include all parameters listed above as additional panels in Figures 1 and 2 to make it more transparent for the readers?
6. Typos check throughout also.

Author Response

Reviewer 2

Listed below requires authors' attention:

1. Would you mind providing keys to abbreviations used throughout, for example, Choc, MV and EER?

We added the explanations of the acronyms

2. Would you mind stating or providing information on if these patients were fully vaccinated?

None of the patients was vaccinated because they were enrolled before the vaccination program

3. Please include cutoff values for Figure 1A and 2A in the figure legends.

Of note, the cutoff values are based on decision trees using the variables together, not the same as using them separately. For intubation: sP-selectin (ng/ml) ≥ 21.5; D-Dimer (ng/ml) ≥ 1347; TFPI (ng/mL)  ≥ 21.3; TF Ag (pg/ml)  ≥ 72.75; IL-6  (pg/mL) ≥ 13.95;  and FXII (%) ≤ 71.50. For mortality: Age ≥ 69.5; TF Ag (pg/ml) ≥ 80.44; TFPI free (ng/mL) ≥ 35.05; and MV_Duration ≥ 0.5. As you recommended we added the corresponding cut-off values at the legends of Figures 1 and 2.

4. Please also state in the discussion if these patients responded to the intensified antithrombotic treatment, ventilation, etc., as suggested by the author ln 339-340, if they were the treatments provided.

The response of the patients to the antithrombotic treatment is a subject of an ongoing analysis of our database. Since we applied an adapted antithrombotic strategy based on daily biological follow-up and dose adjustment we do not wish to present here the data on the efficacy of our strategy in order to avoid misleading conclusions.

5. The authors had included the age and duration of hospitalization plus the top 6 biomarkers in calculating the COMPASS-COVID19-ICU score. However, Figure 1 presents data on all six biomarkers; whereas, Figure 2 shows two biomarkers together with the age and MV duration. It is not very consistent. Can the authors please include all parameters listed above as additional panels in Figures 1 and 2 to make it more transparent for the readers?

Figures 1 and 2 represent 2 distinct models; the models were generated separately, but using the same methodologies.  The data was identical in both models (28 clinical, 41 biomarkers) with the exception that MV duration was excluded for the intubation outcome. For intubation, the best model resulted in 6 features determined by the algorithm. For mortality, the best model resulted in 4 features determined by the algorithm. As such figures 1 and 2 only represent the features of these best models.

6. Typos check throughout also.

We did our best to correct typing and syntax errors.