Machine-Learning-Based COVID-19 Detection with Enhanced cGAN Technique Using X-ray Images

Round 1

Reviewer 1 Report

Attached in PDF file. 

Comments for author File: Comments.pdf

Author Response

The authors would like to thank the reviewers for giving valuable comments. We have incorporated all the comments.

Reviewer # 1

The manuscript “Machine Learning Based COVID-19 Detection with Enhanced cGAN Technique Using X-Ray Images” discuss X-Ray image classification with an unbalanced class of dataset.

Comment #1: Check how you write VGG in the manuscript. It should be ‘VGG’ not Vgg (line no 203). Similar to COVID.

Author Response: Thank you very much for the comment. It is now corrected.

Comment #2: Check the reference format for citing the website (reference number 43). The given link in the reference is not working.

Author Response: The website is checked and cited properly in the list.

Comment #3: Are you using the following dataset?

 https://www.kaggle.com/datasets/tawsifurrahman/covid19-radiography-database

1. Also, as per the dataset, you are supposed to cite the two papers which are not cited in your work.
   132676. E.H. Chowdhury, T. Rahman, A. Khandakar, R. Mazhar, M.A. Kadir, Z.B. Mahbub, K.R. Islam, M.S. Khan, A. Iqbal, N. Al-Emadi, M.B.I. Reaz, M. T. Islam, “Can AI help in screening Viral and COVID-19 pneumonia?” IEEE Access, Vol. 8, 2020, pp. 132665 - 132676. Paper link
   132677. Rahman, T., Khandakar, A., Qiblawey, Y., Tahir, A., Kiranyaz, S., Kashem, S.B.A., Islam, M.T., Maadeed, S.A., Zughaier, S.M., Khan, M.S. and Chowdhury, M.E., 2020. Exploring the Effect of Image Enhancement Techniques on COVID-19 Detection using Chest X-ray Images. Paper Link

Author Response: Yes, we are using the dataset mentioned above. We have cited the first paper already in the paper. However, the second paper was not cited. And now we have cited it. Thanks

Comment #4: Line no 242 - “generating images, , video” - check typo at this line and similar other places too.

Author Response: It is now corrected.

Comment #5: The dataset is not divided for validation purposes. Can authors explain the same? However, in figure 7, there is a label of validation. In this figure, there is no label for testing.

Author Response: Yes. 20% of the whole training data was allotted for validation. Figure 7 is a plot that depicts the performance of the model on different stages of training ( per epoch). That was the reason the test label was not there. The test dataset was kept to check the generalization capability of the developed model on unknown samples.

Comment #6: In Eq(1), all variables are NOT defined.

Author Response: Thank you for the comment. Now all the parameters are defined.

Where the goal of the Generator is to update the parameters to minimize log(1 − D(G(z)); on the other hand the discriminator goal is to update the parameters to re-duce logD(X), G is the parameters for Generator and D is the parameters for Discriminator. x ∼ p_{data}, x is a real data. By definition of G, G(z) is a fake generated data. D(x) is the output of the Discriminator for a real input x and D(G(z)) is the output of the Discriminator for a fake generated data G(z).

Comment #7: How do authors validate generated images from cGAN network? How do the images are different from real images for COVID -19 class?

Author Response: The images generated from cGAN network are validated experimentally and checked at the validation stage.

Comment #8: Is figure 6 referred from some paper? If yes, the authors need to give a reference for the same. Also, the authors need to check with other figures.

Author Response:  No, it is not taken from another paper.

Comment #9: In Figures 10 and 11, values are not matching with dataset samples given in Section3.1. Like, before cGAN, normal class data are 3616. However, it is around 450. Can authors explain the same?

Author Response: Section 3.1 presents the total number of samples. Not the test sample count. The test sample counts were presented in Table 2. The number of samples for before using cGAN and after using cGAN were different

Comment #10: Line number 531, the figure number mentioned is not correct.

Author Response: Thank you so much. Now it is corrected.

Comment #11: There is no meaning of comparing your result with other studies (as they are different). How do authors compare results with different datasets and not use cGAN?

Author Response: Now, all the comparisons are done with the same dataset.

In [31] CovidGAN sought to produce synthetic chest X-ray images using the GAN classification technique. This network obtained 95% accuracy, 90% sensitivity, and 97% specificity using the combination of three databases (COVID-19 Radiography Database, COVID-19 Chest X-Ray Dataset Initiative, and IEEE8023/Covid Chest X-Ray Dataset) with roughly 80% training and 20% testing data. All three of the fine-tuned CNNs achieved accuracy > 99%, sensitivity from 98% (AlexNet) to 100% (GoogLeNet and SqueezeNet), and specificity > 99% using the same database.

In [32] the classification of X-ray images into three classes—normal, bacterial pneumonia, and COVID-19—was done using the Bayes-SqueezeNet [10]. With roughly 89% (3697 pictures) training, validation (462 images), and testing data, and 11% (459 images) testing data, the Bayes-SqueezeNet employed the COVID-19 Radiography Database (Kaggle) and IEEE8023/Covid Chest X-Ray Dataset. Data augmentation was used for the network training. 98%, 99%, and 0.98 were the accuracy, specificity, and F1 scores, respectively.

               Authors here [33] The IEEE8023/Covid Chest X-Ray Dataset and Chestx-ray8 Database [27] were used by the DarkCovidNet [16] to perform 3-class classification and 2-class classification (COVID-19 and no-findings) (COVID-19, no-findings, and pneumonia). For the 2-class classification, the DarkCovidNet achieved accuracy = 98%, sensitivity = 95%, specificity = 95%, precision = 98%, and F1 score = 0.97 using the 5-fold cross validation (80% training and 20% testing).

               Also, in [34] The VGG-19, MobileNet-v2, Inception, Xception, and Inception ResNet-v2 were implemented for the classification of COVID-19 chest X-ray images [18]. Those networks were trained and tested using the IEEE8023/Covid Chest X-Ray Dataset and other chest X-rays collected on the internet. The best 10-cross-validation (90% training and 10% testing) results obtained from the VGG-19 were: accuracy = 98.75% for the 2-class classification and 93.48% for the 3-class classification. Using the COVID-19 Radiography Database. Some studies done on features are [51] and [52].

Reviewer 2 Report

This paper proposes a ML-based approach to identify COVID-19 disease through Chest x-ray.

I'm not sure about the originality of the paper since, as reported by the authors themselves, other papers achieve the same goal (also with good results).

Major comments:

1) The paper is very poorly written. A lot of typos and errors are present. In its current form, it is not adequate for a journal. Several examples are reported below.

2) I have a doubt about the data augmentation procedure. In general, it should be applied only to the training set. The authors should clarify why also the test set is altered.

3) The comparison with other approaches appears not fair. The authors compare the accuracy they achieve with a given dataset with the accuracy other approaches achieve with a different dataset. The authors should repeat the experiment by using the same dataset.

4)The authors just consider the images with NO-COVID and those with COVID. It is relevant also to consider the performance when images with other pathologies are submitted to the classificator.

Minor:

Figure 2 does not appear necessary.

A lot of typos:

-Some cases also are n come critically ill (Line 47)

-has had a significant (Line 65)

-architec‐ture (Line 114)

-Us‐ing GAN, (Line 101)

-modi‐fied VGG16 archi‐ (Line 123)

-and the University of Dhaka. (Line 214)

-will be worked on in Figure (Line 224)

-that ʹs fake , By this way (Line 252)

-So by the help of the Discriminator (Line 255)

-where Figure 9. (Line 409)

-and Figure 10. Shows the accuracy and loss among 27 epochs for our modified  (Line 410)

Author Response

Reviewer #2

The authors would like to thank the reviewers for giving valuable comments. We have incorporated all the comments.

Comment #1: The paper is very poorly written. A lot of typos and errors are present. In its current form, it is not adequate for a journal. Several examples are reported below.

Author Response: The paper is thoroughly checked for English proofreading by a native English speaker.

Comment #2: I have a doubt about the data augmentation procedure. In general, it should be applied only to the training set. The authors should clarify why also the test set is altered.

Author Response: For the training set, the point of the augmentation is to reduce overfitting during training. And we evaluate the quality of the augmentation by then running the trained model against our more-or-less fixed test/validation set.

we might need to perform a similar procedure to the test set as was done on the training set. This is typically so that the input data from the test set resembles as much as possible that of the training set.

we'd need to crop the test images too, so they are the same size as the training images. However, in the case of the training images, we might use each training image multiple times, with crops in different locations/offsets. At test time we'd likely either do a single centred crop, or do random crops and take an average.

Running the augmentation procedure against test data is not to make the test data bigger/more accurate, but just to make the input data from the test set resemble that of the input data from the training set, so we can feed it into the same net (eg same dimensions). We'd never consider that the test set is 'better' in some way, by applying an augmentation procedure. 

Comment #3: The comparison with other approaches appears not fair. The authors compare the accuracy they achieve with a given dataset with the accuracy other approaches achieve with a different dataset. The authors should repeat the experiment by using the same dataset.

Author Response:  The work is now compared with similar datasets.

In [31] CovidGAN sought to produce synthetic chest X-ray images using the GAN classification technique. This network obtained 95% accuracy, 90% sensitivity, and 97% specificity using the combination of three databases (COVID-19 Radiography Database, COVID-19 Chest X-Ray Dataset Initiative, and IEEE8023/Covid Chest X-Ray Dataset) with roughly 80% training and 20% testing data. All three of the fine-tuned CNNs achieved accuracy > 99%, sensitivity from 98% (AlexNet) to 100% (GoogLeNet and SqueezeNet), and specificity > 99% using the same database.

               In [32] the classification of X-ray images into three classes—normal, bacterial pneumonia, and COVID-19—was done using the Bayes-SqueezeNet [10]. With roughly 89% (3697 pictures) training, validation (462 images), and testing data, and 11% (459 images) testing data, the Bayes-SqueezeNet employed the COVID-19 Radiography Database (Kaggle) and IEEE8023/Covid Chest X-Ray Dataset. Data augmentation was used for the network training. 98%, 99%, and 0.98 were the accuracy, specificity, and F1 scores, respectively.

Authors here [33] The IEEE8023/Covid Chest X-Ray Dataset and Chestx-ray8 Database [27] were used by the DarkCovidNet [16] to perform 3-class classification and 2-class classification (COVID-19 and no-findings) (COVID-19, no-findings, and pneumonia). For the 2-class classification, the DarkCovidNet achieved accuracy = 98%, sensitivity = 95%, specificity = 95%, precision = 98%, and F1 score = 0.97 using the 5-fold cross validation (80% training and 20% testing).

Also, in [34] The VGG-19, MobileNet-v2, Inception, Xception, and Inception ResNet-v2 were implemented for the classification of COVID-19 chest X-ray images [18]. Those networks were trained and tested using the IEEE8023/Covid Chest X-Ray Dataset and other chest X-rays collected on the internet. The best 10-cross-validation (90% training and 10% testing) results obtained from the VGG-19 were: accuracy = 98.75% for the 2-class classification and 93.48% for the 3-class classification. Using the COVID-19 Radiography Database. Some studies done on features are [51] and [52].

Minor:

A lot of typos:

-Some cases also are n come critically ill (Line 47)

               That sentences is not needed. So, it is removed.

-has had a significant (Line 65)

               Corrected.

-architec‐ture (Line 114)

               Corrected.

-Us‐ing GAN, (Line 101)

               Corrected.

-modi‐fied VGG16 archi‐ (Line 123)

               Corrected.

-and the University of Dhaka. (Line 214)

               Corrected.

-will be worked on in Figure (Line 224)

               Corrected.

-that ʹs fake , By this way (Line 252)

               Corrected.

-So by the help of the Discriminator (Line 255)

            Corrected.

-where Figure 9. (Line 409)

            Corrected.

-and Figure 10. Shows the accuracy and loss among 27 epochs for our modified  (Line 410)

            Corrected.

Comment #5: How do authors calculate the AUC score for the classification problems?

Author Response:  The AUC score is simply the area under the curve which can be calculated with Simpson’s Rule. The bigger the AUC score the better our classifier is.

Comment #6: Table 5, Feature Extraction and Classification, is it the same column or different?

Author Response: The table is updated, No feature extraction column. Only Classification column.

Round 2

Reviewer 1 Report

Corrections are done. 

Author Response

Thank you

Reviewer 2 Report

In the revised version, the authors partially addressed all the comments.

Anyway, before accepting, the following issue should be addressed:

The comparative analysis (Section 4.1) is too sketchy. Moreover, the authors should report the results concerning the other metrics different from accuracy or at least explain the reason (if any) because just the accuracy is reported in the Section.

Author Response

We have added more metrics as suggested.

Author Response File: Author Response.docx