Machine Learning First Response to COVID-19: A Systematic Literature Review of Clinical Decision Assistance Approaches during Pandemic Years from 2020 to 2022

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This is a literature meta-analysis paper wherein search was done Web of Science and IEEE databases using a database manager tool using the term covid-19 OR SARS-CoV-2 OR coronavirus and machine learning OR deep learning OR artificial intelligence. They studied Geographical distribution of the authors, discriminant features, and type of ML algorithms used. Their results throw light on patients triage at hospital admission, admission to ICU, length of hospital stay, and death for future prediction. Mortality risk factors were identified.

However, I am not sure how can it be aligned with the “Electronics” journal theme?

Following questions needs to be addressed:

1.     Have the authors noticed any correlation in the COVID cases and gender? If yes, which gender is at a higher risk?

2.     The final message of the work isn’t clear..please write the significance of this work alongwith the statistical analysis.

3.     Please briefly discuss the pros and cons of each ML algorithm justifying the most applicable one for these type of literature based studies in future.

4.     How is this study different from text mining..please discuss.

5.     The developed prediction model needs multiple validation on a larger different dataset and/or for other disease terminologies. The present study lacks validation.

Author Response

 

Responses to reviewer #1 electronics-2850505

 

This is a literature meta-analysis paper wherein search was done Web of Science and IEEE databases using a database manager tool using the term covid-19 OR SARS-CoV-2 OR coronavirus and machine learning OR deep learning OR artificial intelligence. They studied Geographical distribution of the authors, discriminant features, and type of ML algorithms used. Their results throw light on patients triage at hospital admission, admission to ICU, length of hospital stay, and death for future prediction. Mortality risk factors were identified.

 

Response: thanks for your appreciation of the manuscript. We think that the paper can not be characterized as a proper meta-analysis, because we do not use typical statistical tools of meta-analysis, such as forest plots. We prefer to characterize it as a systematic review because we have followed the corresponding protocol for search and analysis.

 

However, I am not sure how can it be aligned with the “Electronics” journal theme?

Response: we think that the paper fits into the general area of electronics based automation, in this case computer aided decision, supported by electronics devices, in this case computers and machine learning algorithms.

 

Following questions needs to be addressed:

1. Have the authors noticed any correlation in the COVID cases and gender? If yes, which gender is at a higher risk?

Response: The gender issue has not been considered as a parameter in the literature search which has been focused on machine learning support for clinical assessment of the patients. Many studies included the gender variable, but not all, and those including it did not make epidemiological claims, only specific risk assessments that must be considered in conjunction with other risk factors such as age. In fact, gender was the second most frequently found variable in the studies. Given its social importance, we think that assessing the gender impact of COVID should be addressed by dedicated epidemiological studies in specific reviews, which will be glad to include in our future work. We added this consideration into the conclussions section.

2.The final message of the work isn’t clear..please write the significance of this work alongwith the statistical analysis.

Response: we have summarized the intended message in the conclusions section as a final paragraph. It is that the stadardization of medical data gathering and statistical methodology is a dire need in order to achieve the translation of machine learning models into the clinical practice. We included the last paragraph in the conclusions section:

“As the concluding message from this review, the COVID-19 pandemic has shown that there is a strong need to standardize medical data gathering and statistical methodology including both clinicians and policy makers in the loop, because data has been found in most of the studies to be sketchy and noisy, to the extent that such low quality of data may be considered one of the main limitations to the translation of existing ML and AI models to clinical practice. Moreover, methodological inconsistencies also impede the integration of ML approaches needed for this translation.”

3.Please briefly discuss the pros and cons of each ML algorithm justifying the most applicable one for these type of literature based studies in future.

Reponse: we have included a discussion paragraph highlighting the main characteristics of the most popular ML algorithms found in the review under the subsection “4.3. RQ 2: Machine learning algorithms ” The paragraph is too long to be reproduced here.

4.How is this study different from text mining..please discuss.

Response: according to wikipedia, text mining involves "the discovery by computer of new, previously unknown information, by automatically extracting information from different written resources." It usually proceeds by tranformation of the text into some form of numerical representation that allows the application of computational algorithms, including natural language processing (NLP) tools, which increasingly use AI tools. We have not used any AI tool for the examination of the papers selected for the survey. Neither have we used any NLP tool. We have set the research questions and then look into the papers in order to find how the provide responses to these research questions. We have no direct experience of any AI tool that can do this task at the human level over scientific papers. Our little knowledge of ChatGPT does not provide much trust into it for this task.

5.The developed prediction model needs multiple validation on a larger different dataset and/or for other disease terminologies. The present study lacks validation.

Response: It appears to us that the question comes from some kind of misunderstanding. The paper is a systematic review, it is not a proposal for a novel predictive model. Nowhere in the paper have we asserted a new model that should be validated in the scope of the paper. Neither have we claimed to have collected data for purposes of validation. This statement by the reviewer is shocking, moreover when the reviewer started its review with the phrase “This is a literature meta-analysis paper…” (see above). Sincerely, we do not understand this comment and we do no know its motivation.

Is it possible that the reviewer was reviewing several papers simultaneously and somehow exchanged comments between papers?

 

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

The paper is well written, the research questions are properly designed, evaluated topics offered a complex view on investigated research task.

- The paper contains 4 main research questions defined by the authors. The main goal of this paper is to provide a comprehensive overview of existing work or case studies related to the application of ML or AI methods to solve various tasks emerged during COVID-19 pandemic situation.

- The provided literature review summarizes and provides important information about different approaches how to support relevant decission processed during pandemic situation within data-oriented analytical methods or algoritms. Selected research works are described in more details.

- The mentioned approaches and case studies are not limited to only some parts of ML or AI domain like image processing.

- The literature review is performed according the the PRISMA guidelines.

- The conclusion summarizes the findings appropriately with a list of identified facts like a big variability in the methodological specifics for validation or the actual code of the models is often not shared.

- The references are appropriate.

- The figures look readable.

 

Author Response

 

Responses to reviewer #2 electronics-2850505

 

The paper is well written, the research questions are properly designed, evaluated topics offered a complex view on investigated research task.

- The paper contains 4 main research questions defined by the authors. The main goal of this paper is to provide a comprehensive overview of existing work or case studies related to the application of ML or AI methods to solve various tasks emerged during COVID-19 pandemic situation.

- The provided literature review summarizes and provides important information about different approaches how to support relevant decission processed during pandemic situation within data-oriented analytical methods or algorithms. Selected research works are described in more details.

- The mentioned approaches and case studies are not limited to only some parts of ML or AI domain like image processing.

- The literature review is performed according the the PRISMA guidelines.

- The conclusion summarizes the findings appropriately with a list of identified facts like a big variability in the methodological specifics for validation or the actual code of the models is often not shared.

- The references are appropriate.

- The figures look readable.

 

Response: thanks for the positive assessment. We have revised the paper trying to correct remaining typos and to improve it further. We have highligthed changes in red.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

This paper carries out research on the application of artificial intelligence in the new crown epidemic and discusses the role of artificial intelligence, which has important research implications. However, some issuses should be confirmed:

1. Some citations are not fromatted correctly, for example in line 360-362

2. Why you exclued the Open Access papers in the paper selection process?

3. Why you stated "Evidence of both the need and opportunity to improve clinical care via AI would keep increasing in the coming months, and this fast evolving pandemic can be demonstrated by changes in hospital practices, as well as in the vaccination campaign"? Whether you mean the COVID-19 still increasing? I am confused about this.  

Comments on the Quality of English Language

Appropriate expression, easily understood.

Author Response

 

Responses to reviewer #3 electronics-2850505

 

This paper carries out research on the application of artificial intelligence in the new crown epidemic and discusses the role of artificial intelligence, which has important research implications.

 

Response: thanks for the kind assessment

 

However, some issuses should be confirmed:

1. Some citations are not fromatted correctly, for example in line 360-362

response: thanks for the observation. This problem was due to the change of computers in the process of submission. We have corrected all unsolved references

2. Why you exclued the Open Access papers in the paper selection process?

Response: we used Web of Science for search which included open access editorials such as MDPI and Frontiers. In fact a large number of the references are Open Access, some belonging to these editorials and some to other editorials, such as Elsevier, where authors did publish in open access. In fact, one reason of exclussion was that the paper was not available to us for reading, hence not open access was partially a reason for exclusion. Not  open access  papers   published in editorials for which our university provides access, such as Elsevier, Springer and Wiley, were included because we were able to examine them in detail.

3. Why you stated "Evidence of both the need and opportunity to improve clinical care via AI would keep increasing in the coming months, and this fast evolving pandemic can be demonstrated by changes in hospital practices, as well as in the vaccination campaign"? Whether you mean the COVID-19 still increasing? I am confused about this.  

Response: thanks for the observation. It appears that COVID-19 has become endemic, meaning that cases may be exploding but govenrments are not worried enough to carry out systematic testing and accounting, but still recommending vaccination. Since information about number of cases is not longer published by governments, it is impossible to give any assessment of COVID-19 growing or not. We have rewritten this phrase to avoid confussion as follows

“According to some authors, the COVID-19 pandemic has provided an excellent opportunity to integrate AI tools in clinical care, already introducing 546 changes in hospital practices [] in some advanced countries.”

 

 

Author Response File: Author Response.pdf

Reviewer 4 Report

Comments and Suggestions for Authors

The paper presents a systematic review of machine learning (ML) and artificial intelligence (AI) approaches applied to clinical decision support in response to the COVID-19 pandemic, covering research from 2020 to 2022. It focuses on diagnostic assistance, patient triage, ICU admission prediction, mortality and risk factor identification, and treatment discovery. The review highlights the widespread application of ML and AI in healthcare during the pandemic, demonstrating their potential in predicting patient outcomes, identifying risk factors, and recommending treatments. However, challenges related to data heterogeneity and the need for robust model validation are noted as barriers to clinical implementation.

 

The paper is well-structured and organized.

 

To improve the paper, a more in-depth analysis of the literature is suggested, providing deeper insight into ML/AI advancements, research directions, and persistent challenges. The current approach might seem superficial, potentially inferred from abstracts or automated summaries.

Comments on the Quality of English Language

good

Author Response

 

Responses to reviewer #4 electronics-2850505

 

 

The paper presents a systematic review of machine learning (ML) and artificial intelligence (AI) approaches applied to clinical decision support in response to the COVID-19 pandemic, covering research from 2020 to 2022. It focuses on diagnostic assistance, patient triage, ICU admission prediction, mortality and risk factor identification, and treatment discovery. The review highlights the widespread application of ML and AI in healthcare during the pandemic, demonstrating their potential in predicting patient outcomes, identifying risk factors, and recommending treatments. However, challenges related to data heterogeneity and the need for robust model validation are noted as barriers to clinical implementation.

 

The paper is well-structured and organized.

Response: thanks for the positive appraisal

 

To improve the paper, a more in-depth analysis of the literature is suggested, providing deeper insight into ML/AI advancements, research directions, and persistent challenges. The current approach might seem superficial, potentially inferred from abstracts or automated summaries

 

Response: The selected 220 papers have been examined and read to extract the information relevant to the posed research questions. We have not used any automated system for text processing and extraction of summaries from papers. One practical reason for not doing that is that we do not trust this kind of systems to do the kind of analytical reading that we did. These systems are more appropriate for general recommendation systems, such as the one used by google services.

We have added  section   “ 7 Insights into ML/AI advances, reserch directions and challenges” based on our experience and insights. 

 

 

Author Response File: Author Response.pdf

Reviewer 5 Report

Comments and Suggestions for Authors

This paper focuses on reviewing ML and AI methods used for early pandemic response for clinical decisions. The literature is vast. The inclusion and exclusion criteria for paper selection are well explained and Figure 1 is very clear. The dataset, features, and methods used have been shared well along with plots. The authors dive deep into details of work done in the areas of Triage methods, ICU prediction, Mortality, treatment, and Drugs. The paper finished with an insightful discussion and conclusion.

 

 

Below are my comments:

 

·      Lines 58, 360, 362, and other places have a question mark.

·      Figure 3, some of the abbreviations are not commonly known of used and not included in Table 1, for example, J48. Can you please cite the paper there in Figure 2?

·      There are many typos in the paper please proofread the paper.

Comments on the Quality of English Language

Please check above

Author Response

 

Responses to reviewer #5 electronics-2850505

 

This paper focuses on reviewing ML and AI methods used for early pandemic response for clinical decisions. The literature is vast. The inclusion and exclusion criteria for paper selection are well explained and Figure 1 is very clear. The dataset, features, and methods used have been shared well along with plots. The authors dive deep into details of work done in the areas of Triage methods, ICU prediction, Mortality, treatment, and Drugs. The paper finished with an insightful discussion and conclusion.

Response: thanks for the positive appraisal

 

Below are my comments:

 

• Lines 58, 360, 362, and other places have a question mark.

Response: thanks for the observation. These question marks correspond to references that failed to compile when changing computers for the final version. We have corrected them and revised all the references.

• Figure 3, some of the abbreviations are not commonly known of used and not included in Table 1, for example, J48.

Response: thanks for the observation. We have included the explanation of the acronyms in the caption of the figure, for better readability.

Can you please cite the paper there in Figure 2?

Response: in the caption we refer to the 220 papers selected for the review, not an specific paper. We have changed the caption for better readability

• There are many typos in the paper please proofread the paper.

Response: thanks for the observation. We have revised the paper thoroughfully.

 

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

NIL

Comments on the Quality of English Language

NIL