Applying Regressive Machine Learning Techniques in Determination of COVID-19 Vaccinated Patients’ Influence on the Number of Confirmed and Deceased Patients
, Ivan Lorencin 1,†, Nikola Anđelić 1, Jelena Musulin 1, Daniel Štifanić 1, Matko Glučina 2, Saša Vlahinić 1 and Zlatan Car 1
Round 1
Reviewer 1 Report
The manuscript deals with the prediction of the number of COVID-19 cases and deceased patients using various machine learning techniques.
The manuscript is well written and easy to follow, with the methodology clearly presented. The english seems to be relatively well writtten, although there are some minor errors that should be corrected.
While the manuscript is very interesting, there are some details that I would like addressed before I fully reccomend the publication of the paper:
1. While the references are generally fine, the state-of-the-art review in the introduction seems a bit outdated? Authors should add two or three newer articles that cover similar topics, preferrably published in the last few months.
2. Authors should expand and comment on the figure 1, which shows the rates of different measures in each country.
3. The section 2.3.1 doesn't seem to be as detailed as previous sections of the methodology. Authors should expand this, and possibly include an illustration of the procedure.
4. I dislike having a subsection directly follow the section title, without any text like in 3.1. Please include some text here.
5. Inthe same manner for 3.2. and 3.3. maybe expand the texts immediatelly after the section title, with a sentence or two? The way authors did for India is perfect.
6. Figures 3, 5, 7, 9 - consider adding "Lower is better" or similar in the figure caption, to make it easier to understand at a glance.
Because of the above comments, I reccomend minor revisions before publication.
Author Response
Respected Reviewer,
The authors would like to thank you for their detailed review of our manuscript. We have addressed all of the reviewers' comments and made changes to our manuscript accordingly. Please find the answers to the reviewer comments below. We have marked the changes made to the manuscript with a highlight, to assist in re-review. Regarding your comment about the minor errors in English grammar and spelling we have went through the manuscript once again, and made corrections to the text of the manuscript where appropriate.
- While the references are generally fine, the state-of-the-art review in the introduction seems a bit outdated? Authors should add two or three newer articles that cover similar topics, preferrably published in the last few months.
We have added a couple of papers to our introduction:
“Bagabir et. al. [18] discuss many applications of AI regarding COVID-19, including genome sequencing and drug/vaccination development, noting it to be an indispensable tool. Despite ML being a commonly referenced tool in the scope of COVID-19 as can be seen from Mariappan et al. [22] where the shipment times of vaccines are modeled using it, or Tong et al. [23] who use AI-assisted techniques to determine the antibody flow and quantitative detection; the direct modeling of the influence of vaccination rates on the number of patients is currently an unexplored area”
- Authors should expand and comment on the figure 1, which shows the rates of different measures in each country.
The following explanation was added
“The trends for vaccinated patients (single vaccinations, full vaccinations and boosters) differ noticeably across all the analyzed countries. This indicates a difference in vaccination strategies and acceptance of vaccinations among the populace for each of the analyzed countries. Taking the difference in vaccination rates in comparison to the number of infected and deceased patients it is clear that differences exist between analyzed countries in that area as well. This illustrates the differences between analyzed countries, in both the vaccination rates and infection rates, showing why they have been selected for modeling and analysis.”
- The section 2.3.1 doesn't seem to be as detailed as previous sections of the methodology. Authors should expand this, and possibly include an illustration of the procedure.
We have added a figure illustrating a process and expanded the explanation of
it as follows:
“This process is illustrated in Figure 2. For simplicity, only 10 datapoints have been used. As it can be seen, the full dataset is split into five different folds randomly across its length. Each data point can belong to only one data fold, and it has the same chance of becoming part of each fold. Then, a single data point is selected to be the training set, while the remaining four are to be used as a testing set. Note that, while the figure shows data points to be held together as folds in this process - this is done only for a simpler understanding of the illustration, and the data points are actually shuffled. This process is then repeated, with a different fold being taken as a training set each time, until all of the folds have been used.”
- I dislike having a subsection directly follow the section title, without any text like in 3.1. Please include some text here.
The following text was added to the start of the section
“In this subsection, the results of the research performed on the data for the United States of America are presented. The following subsections present the values of correlation analysis and regression results. The obtained results are presented graphically and tabularly with the discussion of the results given.”
- Inthe same manner for 3.2. and 3.3. maybe expand the texts immediatelly after the section title, with a sentence or two? The way authors did for India is perfect.
The following text was added for 3.2.
“United Kingdom showed a rapid acceptance and distribution of vaccinations. This is paired with a relatively stable increase in infection rates, which differs it from other analyzed countries.”,
and the following for 3.3.
“Germany, as most EU countries, shows high vaccination rates with a relatively high, but stable, infection rates which continue to slightly grow even after the introduction of vaccines. As with previous countries, following subsections will present the cross correlation analysis both numerically and graphically, with the same done for the regression analysis.”
- Figures 3, 5, 7, 9 - consider adding "Lower is better" or similar in the figure caption, to make it easier to understand at a glance.
The text has been added to figure captions.
Kindest regards,
the authors.
Reviewer 2 Report
The proposed paper is addressing very important issue using modern techniques and therefor fully in line with journal topic. Aldo the overall manuscript is well written, there are some issues throughout:
- Figure 1 - use two scales or a non-linear scale, because you can't see some measurements.
- Figure 1 - what is what on the figure? Please add a legend.
- Equation 7 - what does this equation equal to? Error? This should be included.
- All tables in section 3 - please use a consistent precision for decimal score values.
- Even though they are defined at the begining of section, you should add a description of what abbreviations are in each of the graphs used to make them more readable.
- Table 10, please check method column, was there an error
- abbreviations list is missing some abbreviations (BC, BD)
I suggest a minor revision of the manuscript
Author Response
Respected Reviewer,
We thank you for your review of the manuscript. Please find answers to the comments posed below. We have marked the changes in the manuscript using a highlight.
- Figure 1 - use two scales or a non-linear scale, because you can't see some measurements.
Figures have been updated to use two scales instead of a single one.
- Figure 1 - what is what on the figure? Please add a legend.
The legend was added to each of the figures.
- Equation 7 - what does this equation equal to? Error? This should be included.
The symbol for error was included in the equation.
- All tables in section 3 - please use a consistent precision for decimal score values.
Consistent precision of 9 decimal places has been used in the corrected version of the manuscript.
- Even though they are defined at the begining of section, you should add a description of what abbreviations are in each of the graphs used to make them more readable.
Abbreviations have been added to figure captions.
- Table 10, please check method column, was there an error
The typo in the first row, second column was corrected.
- abbreviations list is missing some abbreviations (BC, BD)
The missing abbreviations have been added.
Kindest regards,
the authors
Reviewer 3 Report
Abstract:
-Please write few sentences clearly the findings of the work, and might positively influence.
-Please mention about cross-correlation lag, i.e. smaller delays be used between the data series for predictive modeling.
Introduction:
- Please rewrite lines 23-26.
- Misinformation played a major role in accepting/rejecting the vaccination process. AI have been extensively used to fight these. Please use and cite the following articles, and add few sentences discussing this.
1. Web Search Engine Misinformation Notifier Extension (SEMiNExt): A Machine Learning Based Approach during COVID-19 Pandemic. Link: https://www.mdpi.com/2227-9032/9/2/156
2. Chatbots in the fight against the COVID-19 pandemic. Link: https://www.nature.com/articles/s41746-020-0280-0
- Please add few more sentences and strengthen the motivation of the work further.
- Please briefly explain what cross-correlation determined lags mean? It might be unknown to some people.
- In the related works section, please mention their accuracies and drawback, and compare them in a table.
Materials and methods:
- In lines 71-72: How were the countries (USA, Germany, UK, and India) been selected and based on which criteria? It is only mentioned briefly. Please expand it more.
- In figure 1, please use higher size x/y axis labelling.
- In figure 1, please mention what each colored lines represent.
Results:
- Please use same figure labeling font size for all the images.
Conclusions:
- It is mentioned that “The results show that the highest correlation for all the countries, and all data pairs used, is equal to the length of the data series”. While from lines 57-59 the authors mentioned “Then, the used techniques and correlation analysis will be presented, followed by the presentation of the best-achieved results for each of the countries observed in the research are Germany, India, the United Kingdom, and the United States of America.” Do the models work for all the countries or specific countries? Please clarify it in the manuscript.
- Please rewrite lines 345-347.
Author Response
Respected Reviewer,
We thank you for your review of the manuscript. Please find answers to the comments posed below. We have marked the changes in the manuscript using a highlight.
Abstract:
- Please write few sentences clearly the findings of the work, and might positively influence.
The following text was added to the abstract:
“The obtained results indicate that the influence of vaccination rates on the number of confirmed and deceased patients exists, and can be modelled using ML methods with a relatively high precision.”
- Please mention about cross-correlation lag, i.e. smaller delays be used between the data series for predictive modeling.
The following text was added:
“Cross-correlation analysis is performed to determine the optimal lags in data, to assist in obtaining better scores.”
Introduction:
- Please rewrite lines 23-26.
The lines have been rewritten as follows:
“It is hard to determine the influence of vaccination rates on the spread of viral diseases such as COVID-19 [9, 10]. Creation of predictive models for such a problematic is a complex issue due to many interacting factors [11, 12].”
- Misinformation played a major role in accepting/rejecting the vaccination process. AI have been extensively used to fight these. Please use and cite the following articles, and add few sentences discussing this.
The following text was added, using suggested citations.
“Another area of AI use in relation to COVID-19 was the identification of misinformation shared on social media [24]. The spread of misinformation is an important factor that can have a high influence on the success of different governmental decisions, as it can influence the likelihood of people following the recommendations - such as vaccinations [25].”
- Please add few more sentences and strengthen the motivation of the work further.
The following text was added to further explain the motivations:
“Development of such models is a goal of this paper. The main motivation is to create models which will enable the prediction of the future rates of infection and patient deaths based on the vaccination rates in the given country. This would enable further strategic planning regarding the hospital systems in the given countries, because a high number of patients represents one of the biggest healthcare challenges related to the COVID-19 pandemic.”
- Please briefly explain what cross-correlation determined lags mean? It might be unknown to some people.
We have added the following explanation after the used term “(the time-shifts of discrete data points between the input and output datasets)”
- In the related works section, please mention their accuracies and drawback, and compare them in a table.
The table with the requested information has been added.
Materials and methods:
- In lines 71-72: How were the countries (USA, Germany, UK, and India) been selected and based on which criteria? It is only mentioned briefly. Please expand it more.
The following text was added.
“The trends for vaccinated patients (single vaccinations, full vaccinations and boosters) differ noticeably across all the analyzed countries. This indicates a difference in vaccination strategies and acceptance of vaccinations among the populace for each of the analyzed countries. Taking the difference in vaccination rates in comparison to the number of infected and deceased patients it is clear that differences exist between analyzed countries in that area as well. This illustrates the differences between analyzed countries, in both the vaccination rates and infection rates, showing why have they been selected for modelling and analysis”
- In figure 1, please use higher size x/y axis labelling.
Figure was updated with higher sized labels.
- In figure 1, please mention what each colored lines represent.
A legend was added to each of the subfigures.
Results:
- Please use same figure labeling font size for all the images.
The figures have been adjusted assuring each instance of cross-correlation figures and each instance of results figures use the same font sizes.
Conclusions:
- It is mentioned that “The results show that the highest correlation for all the countries, and all data pairs used, is equal to the length of the data series”. While from lines 57-59 the authors mentioned “Then, the used techniques and correlation analysis will be presented, followed by the presentation of the best-achieved results for each of the countries observed in the research are Germany, India, the United Kingdom, and the United States of America.” Do the models work for all the countries or specific countries? Please clarify it in the manuscript.
The first sentence was updated as such to clarify:
“The results show that the highest correlation \hl{for all of the analyzed countries (USA, UK, Germany and India), and all data pairs used}, is equal to the length of the data series.”
- Please rewrite lines 345-347.
The lines in question have been rewritten as:
“The analyzed results demonstrate that the highest correlation is shown when the lag is zero, or in other words, when the data series fully overlap.”
Kindest regards,
the authors
