Predictive Classifier for Cardiovascular Disease Based on Stacking Model Fusion
Round 1
Reviewer 1 Report
This paper applied different machine learning methods with factors contributing to heart disease on a single dataset. The idea is to compare the best learner of the five learners applied and
My concerns are:
1) I do not see the paper area fits into the journal's scope.
2) The paper lacks related work discussions and a clear highlight of the contribution of this work.
3) Also, there is no clear explanation of how this methodology was developed and compared to similar other works.
4) The algorithm on page 5 needs to explain how the five testing folds are selected. Unfortunately, the way this is shown in the algorithm is not correct.
5) The algorithm on page 6 needs to be fixed and show how the meta-model is selected.
6) The results indicate slight improvement of the stacking model over the other models, which may not be considered a promising method.
6) The study needs to apply this in several other datasets to see how this methodology contributes to the area.
Author Response
Please see the attachment.
Author Response File: 
Author Response.docx
Reviewer 2 Report
The paper addresses the burning problem of cardiovascular disease classification in the supervised learning setting. The problem under consideration is socially significant, reflected, and well-justified by the authors in the introduction chapter.
The authors compared several data observation mechanisms and
supervised-learning algorithms. The considered algorithms include both state-of-the-art and classic techniques, such as SVM, KNN, RF, GBDT, XGBoost, LightGBM, CatBoost, and others.
The methodology, data-processing pipeline, and the dataset itself are generally well-defined. The experiments are conducted consistently, and the results are easy to follow. The overall quality of the manuscript is good, and the paper is technically sound.
However, there are a few issues to be addressed to further improve the submission's quality.
1) The authors have applied SHAP value for the feature selection. However, this very interesting and novel methodology is not covered by the methodology chapter.
2) The quality (resolution) of figure 4 is poor.
3) The paper compares several algorithms against the standard benchmark. The performance of Stacking is indeed slightly superior compared to the MLP provided by sklearn. However, it would be fair to compare Stacking against the artificial neural network built and finetuned using the specialized framework like Keras, TF or PyTorch.
4) It is also worth discussing the algorithmic complexity and accuracy-complexity trade-offs.
5) Proofreading is recommended.
Author Response
Please see the attachment.
Author Response File: Author Response.docx
Round 2
Reviewer 1 Report
The authors have made significant changes compared to the last version. The authors managed to improve the quality of the presentation in general. However, some minor issues need to be revisited. For example, numbers are inconsistent in table 2. We found only F results are on a 0-1 scale, and other measurements are presented in a percentage format. The algorithm of the methodology is still not clear. How to pick the testing set? Why did you select the mean of the testing, and what do you mean by the mean anyway? The algorithm needs to be rewritten and explained in the text how it is implemented. Moreover, I am not sure why the authors have two datasets, why are they not a single dataset? Aren't they the same? If not, what are the features and the response variable for each one
Author Response
Reviewer: 1
Comments to the Author
The authors have made significant changes compared to the last version. The authors managed to improve the quality of the presentation in general. However, some minor issues need to be revisited.
Response:
Thank you so much for helpful suggestion. We have revised the manuscript carefully and improved it according to your essential comments and helpful suggestions.
For example, numbers are inconsistent in table 2. We found only F results are on a 0-1 scale, and other measurements are presented in a percentage format.
Response:
Thanks a lot. I have modified the F1 Score in table 3 to be in the form of a percentage. And checked the rest of the details for errors.
The algorithm of the methodology is still not clear. How to pick the testing set? Why did you select the mean of the testing, and what do you mean by the mean anyway? The algorithm needs to be rewritten and explained in the text how it is implemented.
Response:
Thank you so much for helpful suggestion. We reworked Figure 2 in Section 2.3 to show the five-fold stacking process in the bottom half of Figure 2, which makes the structure of the algorithm appear clearer and more understandable. We also followed up with a detailed description of the algorithm immediately after Figure 2, and finally we reworked it in Algorithm 1 and added detailed comments. The algorithm is noteworthy because there are two prediction results here, one is the prediction result obtained by five-fold CV on the original training set, which is stacked vertically as the new training set. The other is the prediction result from the original testing set, averaged horizontally as the new testing set (as shown in the lower part of Figure 2).
Moreover, I am not sure why the authors have two datasets, why are they not a single dataset? Aren't they the same? If not, what are the features and the response variable for each one.
Response:
Thanks a lot for your good suggestions. We must say sorry to the reviewers. According to your first comment "The study needs to apply this in several other datasets to see how this methodology contributes to the area.", so I have added another similar dataset (Heart Attack Dataset) in the area, but our main study is still Heart Dataset, our preliminary data cleaning, feature engineering and correlation analysis are only for Heart Dataset, and Heart Attack Dataset is only used to validate our model's applicability. They are not the same dataset, so we have added the description of Heart Attack Dataset in Section 2.1 as shown in Table 2.
Meanwhile, we will note these problems in our following manuscripts and will improve our English writing. The manuscript has been resubmitted to your journal. We are looking forward to your positive response.
Best wishes
Yours sincerely,
Huiqi Zhao
2022.4.7
Reviewer 2 Report
The authors considered all the critical feedback. As a result, the overall quality of the manuscript has been improved drastically.
Author Response
Please see the attachment.
Author Response File: Author Response.docx
Round 3
Reviewer 1 Report
The authors addressed all questions and concerns in the second review round.
