A Predictive Model of Early Readmission for Patients with Heart Failure

Round 1

Reviewer 1 Report

Hu et al have developed a predictive model for 30-day hospital readmissions for heart failure patients. 

My main concerns are the following:

1. There are several models that have been developed to predict 30-day readmissions. Of late, machine learning has been used to develop algorithms, with better AUC. As machine learning was not used in this study, rather manual EMR review was performed, there should have been other clinical variables used which have been shown to predict HF readmissions - such as orthodema score at time of discharge, BP on admission, change in NTpro BNP from admission to discharge, absolute NT pro BNP at time of discharge. Most importantly HF goal directed medical therapy has not been included, which has been shown across all studies to reduce readmissions. 

2. Precision recall curves should be used to better demonstrate predictive success.

3. The readmission rate in this cohort is much lower than other studies and registry data which report a 30-day readmission rate of 20-30%. 

4. The overall EF was 45-60%, implying only HFmrEF and HFpEF patients have been included in this study. 

5. The introduction and discussion need to be more focused 

Minor points:

1. There are grammatical errors in lines 35, 43,145, 148,149,157

Author Response

Reviewer #1:

My main concerns are the following:

1. There are several models that have been developed to predict 30-day readmissions. Of late, machine learning has been used to develop algorithms, with better AUC. As machine learning was not used in this study, rather manual EMR review was performed, there should have been other clinical variables used which have been shown to predict HF readmissions - such as orthodema score at time of discharge, BP on admission, change in NTpro BNP from admission to discharge, absolute NT pro BNP at time of discharge. Most importantly HF goal directed medical therapy has not been included, which has been shown across all studies to reduce readmissions. 

Re: Thanks for your comments. A machine learning model has been used to predict 30-day readmission in HF patients from seven major hospitals in the Boston Metro area and eastern Massachusetts. However, it may be unsuitable for patients in other countries due to differences in disease management patterns and readmission rates. In addition, machine learning methods may be difficult for most clinicians as there are no equation used for calculating risk scores. In order to facilitate the operation of clinicians to predict the disease outcomes in advance, we used the LASSO method to develop the prediction model which was easy for clinicians to evaluate the risk of 30-day HF readmission. Besides, the AUCs of the machine learning model are slightly higher than this prediction model. Therefore, novel biomarkers should be developed and used to precisely predict the prognosis of HF in future studies. For these cases, we had supplemented clarifications in the Introduction and Discussion.

The model we developed was convenient for clinicians to make an early judgment on the 30-day readmission risk, thus we only collected the main indicators on the first day of admission. In addition, we had supplemented the medical therapy in the Methods.

2. Precision recall curves should be used to better demonstrate predictive success.

Re: Thanks for your suggestion. The reason why we choose the ROC curves rather than precision recall curves is that most of the current prediction models use ROC curves to evaluate the prediction value by comparing the AUCs.

3. The readmission rate in this cohort is much lower than other studies and registry data which report a 30-day readmission rate of 20-30%.

Re: Emerging evidences suggests that the 30-day readmission rate for HF differs greatly in different hospitals and periods, potentially related to disease management. Our data were collected from a large-scale teaching hospital, in which the 30-day readmission rate of HF patients seemed lower than the average level reported in most literatures. In order to verify whether the model is applicable to HF patients in the general hospital of grade three, we collected clinical data of 302 patients hospitalized in the same period from Xinqiao Hospital of the Army Medical University, showing that the 30-day readmission rate was only 8.9%. However, we finally gave up this external verification due to the small number of cases.

4. The overall EF was 45-60%, implying only HFmrEF and HFpEF patients have been included in this study.

Re: The IQR of LVEF was 45-60%. Actually, the values of LVEF ranged from 11% to 83%, exhibiting a non-normal distribution.

5. The introduction and discussion need to be more focused.

Re: Thanks for your suggestion. We had supplemented the Introduction and Discussion.

Minor points:

1. There are grammatical errors in lines 35, 43,145, 148,149,157

Re: Thank you. We had corrected these grammatical errors.

Reviewer 2 Report

A predictive model of early readmission for patients with heart failure

The manuscript entitled “A predictive model of early readmission for patients with heart failure” by Hu et al., is an interesting article. Here, the authors developed a predictive model of early readmission for patients with heart failure (HF). They have screened out independent risk factors that can effectively judge the early readmission of HF.

Overall, the information presented in this manuscript is useful and I approve its publication after some minor updates. 

Minor comments: I suggest that these comments to be updated before publication.

1. Introduction part is good, but there is not much information about predictive model.  It will be useful to include more details about multi-factor predictive model
2. In results section- For figure 2 & 3 the results were not explained properly so elaborate the results for figure 2 & 3.
3. It will be good to have uniform font size for all figures. For Figure- 1, the X- axis/Y-axis labels and scale are not similar size like other figures (figure 3,4). Even for figure 2, font size is bigger than figure 3,4.

Author Response

Reviewer #2:

The manuscript entitled “A predictive model of early readmission for patients with heart failure” by Hu et al., is an interesting article. Here, the authors developed a predictive model of early readmission for patients with heart failure (HF). They have screened out independent risk factors that can effectively judge the early readmission of HF.

Overall, the information presented in this manuscript is useful and I approve its publication after some minor updates.

Minor comments: I suggest that these comments to be updated before publication.

Re: Thanks for your comments and suggestions.

1. Introduction part is good, but there is not much information about predictive model.  It will be useful to include more details about multi-factor predictive model.

Re: We had revised relevant information in the Introduction.

2. In results section- For figure 2 & 3 the results were not explained properly so elaborate the results for figure 2 & 3.

Re: We had revised the description of Figure 2 and 3 in the Results.

3. It will be good to have uniform font size for all figures. For Figure- 1, the X- axis/Y-axis labels and scale are not similar size like other figures (figure 3,4). Even for figure 2, font size is bigger than figure 3,4.

Re: Thanks for your good suggestion. We had re-drawn the Figures.

Reviewer 3 Report

In this manuscript, Hu et al. aim to develop a predictive risk model for readmission of HF at 30-days after the index admission. Although this is an interesting topic and previous risk models lack general approval by Guideline recommendations there are several issues that need to be clarified.

1.     Compare the diagnostic value of this predictive model with previously published scores and comment on the strengths and weaknesses of this model.

2.     Please provide data regarding previously diagnosed cardiovascular disease such as myocardial infarction, revascularization procedures, ischemic stroke, and peripheral artery disease. Other comorbidities such as atrial fibrillation, valvular diseases, myocardiopathies and chronic kidney disease should be reported and included as potential prognostic risk factors.

3.     Why do all patients have mildly-reduced or preserved LVEF (>45)? Is this stated in the methods of the study? The Authors should declare throughout that patients with preserved / mildly-reduced LVEF were included and limit their conclusions to this population.

4.     Please provide data regarding heart failure treatment regimens used in this cohort (loop diuretics, b-blockers, ACEi/ARBs, ARNI, aldosterone receptor antagonists, SGLT2).

The Authors should acknowledge study limitations including:

1.     Lack of a validation cohort

2.     The derivation cohort is of the Asian race and therefore generalizability to other populations is limited.

3.     Lack of reclassification and discrimination value in comparison to previous predictive models.  

Author Response

Reviewer #3:

In this manuscript, Hu et al. aim to develop a predictive risk model for readmission of HF at 30-days after the index admission. Although this is an interesting topic and previous risk models lack general approval by Guideline recommendations there are several issues that need to be clarified.

Re: Thanks for your comments and suggestions.

1. Compare the diagnostic value of this predictive model with previously published scores and comment on the strengths and weaknesses of this model.

Re: We had supplemented it in the Discussion.

2. Please provide data regarding previously diagnosed cardiovascular disease such as myocardial infarction, revascularization procedures, ischemic stroke, and peripheral artery disease. Other comorbidities such as atrial fibrillation, valvular diseases, myocardiopathies and chronic kidney disease should be reported and included as potential prognostic risk factors.

Re: Most of the HF patients in this study were comorbided with coronary artery diseases, which were much more than other primary comorbidities. Moreover, we found that the primary comorbidities were not relevant to the risk of 30-day readmission.

3. Why do all patients have mildly-reduced or preserved LVEF (>45)? Is this stated in the methods of the study? The Authors should declare throughout that patients with preserved / mildly-reduced LVEF were included and limit their conclusions to this population.

Re: The IQR of LVEF was 45-60%. Actually, the values of LVEF ranged from 11% to 83%, exhibiting a non-normal distribution..

4. Please provide data regarding heart failure treatment regimens used in this cohort (loop diuretics, b-blockers, ACEi/ARBs, ARNI, aldosterone receptor antagonists, SGLT2).

Re: We had supplemented the medical therapy in the Methods.

The Authors should acknowledge study limitations including:

1. Lack of a validation cohort

Re: The prediction model was internally validated by a 10-fold cross-validation analysis. In order to verify whether the model is applicable to HF patients in the general hospital of grade three, we collected clinical data of 302 patients hospitalized in the same period from Xinqiao Hospital of the Army Medical University. However, we finally gave up this external verification due to the small number of cases.

2. The derivation cohort is of the Asian race and therefore generalizability to other populations is limited.

Re: While many indicators that are significantly associated with HF prognosis have been identified, few studies have used them to establish a prognosis-prediction model. Indeed, in the Chinese population, the prediction of 30-day HF readmission only applies to elderly patients aged ≥ 65 years. Emerging evidences suggests that the 30-day readmission rate for HF differs greatly in different hospitals and periods, potentially related to disease management. Therefore, prediction models based on data from different countries and hospitals of different grades are not necessarily universally applicable. Consequently, there is an urgent need for novel predictive models for 30-day HF readmission in the Chinese hospitalized population.

3. Lack of reclassification and discrimination value in comparison to previous predictive models.  

Yours sincerely,

Re: Thanks for your comments. We had supplemented these in the Introduction and Discussion.

Round 2

Reviewer 3 Report

1.     Compare the diagnostic value of this predictive model with previously published scores and comment on the strengths and weaknesses of this model.

      Re: We had supplemented it in the Discussion.

Please comment on specific previously published models and risk scores^1-4. The Authors did not make any changes and basically ignored this comment.

2.     Please provide data regarding previously diagnosed cardiovascular disease such as myocardial infarction, revascularization procedures, ischemic stroke, and peripheral artery disease. Other comorbidities such as atrial fibrillation, valvular diseases, myocardiopathies and chronic kidney disease should be reported and included as potential prognostic risk factors.

Re: Most of the HF patients in this study were comorbided with coronary artery diseases, which were much more than other primary comorbidities. Moreover, we found that the primary comorbidities were not relevant to the risk of 30-day readmission.

Please provide the data regarding coronary artery disease prevalence in Table 1. Provide details regarding coronary artery disease (i.e. acute coronary syndrome, stable coronary heart disease, history of revascularization).

You mention that nearly all of the participants had CAD but only 25% had dyslipidaemia. How was dyslipidaemia defined in this cohort?

What is the prevalence of chronic kidney disease and atrial fibrillation? These are two important comorbidities. Provide the 

3.     Why do all patients have mildly-reduced or preserved LVEF (>45)? Is this stated in the methods of the study? The Authors should declare throughout that patients with preserved / mildly-reduced LVEF were included and limit their conclusions to this population.

Re: The IQR of LVEF was 45-60%. Actually, the values of LVEF ranged from 11% to 83%, exhibiting a non-normal distribution.

Have you tested the predictive value of your model separately in HFpEF and HFrEF? These two entities are generally of different etiology, with different prevalence of comorbidities and prevalence of risk factors. Please provide sensitivity analysis in each HF phenotype. Also indicate in Table 1 if variables re indicated as mean+sd and median+IQR.

4.     Please provide data regarding heart failure treatment regimens used in this cohort (loop diuretics, b-blockers, ACEi/ARBs, ARNI, aldosterone receptor antagonists, SGLT2).

Re: We had supplemented the medical therapy in the Methods.

The regarding HF treatment should be included in Table 1 and separated as pre-hospital and post-discharge treatment if these data are available.

Study limitations

3. Lack of reclassification and discrimination value in comparison to previous predictive models.  

Yours sincerely,

Re: Thanks for your comments. We had supplemented these in the Introduction and Discussion.

The Authors have basically ignored this comment and provided no feedback. Please provide suitable answer.

Line 177: change “slao”

References

1.         Whellan DJ, Ousdigian KT, Al-Khatib SM, Pu W, Sarkar S, Porter CB, Pavri BB and O'Connor CM. Combined heart failure device diagnostics identify patients at higher risk of subsequent heart failure hospitalizations: results from PARTNERS HF (Program to Access and Review Trending Information and Evaluate Correlation to Symptoms in Patients With Heart Failure) study. J Am Coll Cardiol. 2010;55:1803-10.

2.         Fleming LM, Gavin M, Piatkowski G, Chang JD and Mukamal KJ. Derivation and validation of a 30-day heart failure readmission model. Am J Cardiol. 2014;114:1379-82.

3.         Ibrahim AM, Koester C, Al-Akchar M, Tandan N, Regmi M, Bhattarai M, Al-Bast B, Kulkarni A and Robinson R. HOSPITAL Score, LACE Index and LACE+ Index as predictors of 30-day readmission in patients with heart failure. BMJ Evid Based Med. 2020;25:166-167.

4.         Frizzell JD, Liang L, Schulte PJ, Yancy CW, Heidenreich PA, Hernandez AF, Bhatt DL, Fonarow GC and Laskey WK. Prediction of 30-Day All-Cause Readmissions in Patients Hospitalized for Heart Failure: Comparison of Machine Learning and Other Statistical Approaches. JAMA Cardiology. 2017;2:204-209.

Author Response

1. Compare the diagnostic value of this predictive model with previously published scores and comment on the strengths and weaknesses of this model.

Please comment on specific previously published models and risk scores1-4. The Authors did not make any changes and basically ignored this comment.

Re: Thanks for your comments. We had supplemented it in the Discussion.

2. Please provide data regarding previously diagnosed cardiovascular disease such as myocardial infarction, revascularization procedures, ischemic stroke, and peripheral artery disease. Other comorbidities such as atrial fibrillation, valvular diseases, myocardiopathies and chronic kidney disease should be reported and included as potential prognostic risk factors.

Please provide the data regarding coronary artery disease prevalence in Table 1. Provide details regarding coronary artery disease (i.e. acute coronary syndrome, stable coronary heart disease, history of revascularization).

You mention that nearly all of the participants had CAD but only 25% had dyslipidaemia. How was dyslipidaemia defined in this cohort?

What is the prevalence of chronic kidney disease and atrial fibrillation? These are two important comorbidities. Provide the

Re: Thanks for your suggestions. We had supplemented the comorbidities including stable coronary heart disease, acute coronary syndrome, atrial fibrillation, and chronic kidney disease. In addition, the data had been analyzed again.

Dyslipidaemia was defined according to criterion published by the 2016 Chinese guidelines for the management of dyslipidaemia in adults. Only 25% had dyslipidaemia probably due to statin treatment.

3. Why do all patients have mildly-reduced or preserved LVEF (>45)? Is this stated in the methods of the study? The Authors should declare throughout that patients with preserved / mildly-reduced LVEF were included and limit their conclusions to this population.

Have you tested the predictive value of your model separately in HFpEF and HFrEF? These two entities are generally of different etiology, with different prevalence of comorbidities and prevalence of risk factors. Please provide sensitivity analysis in each HF phenotype. Also indicate in Table 1 if variables re indicated as mean+sd and median+IQR.

Re: About 11.4% of the patients had HFrEF, thus the sample size was not enough for validation. LVEF has been identified as an independent risk factor for predicting HF 30-day readmission. According to the nomogram, we can evaluate the 30-day readmission risk in HF patients with different LVEF values.

4. Please provide data regarding heart failure treatment regimens used in this cohort (loop diuretics, b-blockers, ACEi/ARBs, ARNI, aldosterone receptor antagonists, SGLT2).

The regarding HF treatment should be included in Table 1 and separated as pre-hospital and post-discharge treatment if these data are available.

Re: We had supplemented the medical therapy in the Methods and Table 1.

Lack of reclassification and discrimination value in comparison to previous predictive models.  

The Authors have basically ignored this comment and provided no feedback. Please provide suitable answer.

Re: Thanks for your good comments. We had supplemented these in the manuscript.

Line 177: change “slao”

Re: Thanks. We had corrected it