COVID-19 Patient Detection Based on Fusion of Transfer Learning and Fuzzy Ensemble Models Using CXR Images

: The COVID-19 pandemic has claimed the lives of millions of people and put a signiﬁcant strain on healthcare facilities. To combat this disease, it is necessary to monitor affected patients in a timely and cost-effective manner. In this work, CXR images were used to identify COVID-19 patients. We compiled a CXR dataset with equal number of 2313 COVID positive, pneumonia and normal CXR images and utilized various transfer learning models as base classiﬁers, including VGG16, GoogleNet, and Xception. The proposed methodology combines fuzzy ensemble techniques, such as Majority Voting, Sugeno Integral, and Choquet Fuzzy, and adaptively combines the decision scores of the transfer learning models to identify coronavirus infection from CXR images. The proposed fuzzy ensemble methods outperformed each individual transfer learning technique and several state-of-the-art ensemble techniques in terms of accuracy and prediction. Speciﬁcally, VGG16 + Choquet Fuzzy, GoogleNet + Choquet Fuzzy, and Xception + Choquet Fuzzy achieved accuracies of 97.04%, 98.48%, and 99.57%, respectively. The results of this work are intended to help medical practitioners achieve an earlier detection of coronavirus compared to other detection strategies, which can further save millions of lives and advantageously inﬂuence society.


Introduction
Coronavirus is an extremely contagious virus that mostly affects the lungs and is characterized by symptoms of cough, fever, and exhaustion. The disease has the potential to spread to the lower respiratory system, where it can potentially cause inflammation and swelling, resulting in pneumonia [1]. COVID-19 spreads by respiratory droplet production following coughing and patient contact. As a result, preventative methods, such as wearing masks, social distancing, and isolating infected people, have been implemented to deal with viral transmission thus far [2]. COVID-19 has devastated most healthcare systems around the world, causing economic damage and claiming the lives of numerous individuals. Although vaccines are being produced and administered around the globe, it will be a very long time before each human can be vaccinated. As new strains of infection arise, such worldwide vaccination will be further delayed and will lead to subsequent lockdowns. As a result, there is an even greater need for the accurate and early identification of COVID-19 to prevent viral spread.
Currently, RT-PCR remains the most utilized detection technique by health professionals, which requires samples to be collected from the upper and lower parts of the sputum and nasal cavity from people suspected of being infected with COVID-19. The RT-PCR test faces several limitations, such as the manual collection of samples from infected patients, with an accuracy of 89.3% and 94%, respectively. When reviewing the related work, it was discovered that few researchers concentrated on fuzzy ensemble approaches paired with transfer learning models. An ensemble approach is a machine learning technique that integrates many base models to generate a single optimal prediction model. The outcomes of numerous models are integrated in order to enhance overall performance. Specifically, ensemble methods integrate numerous machine learning approaches into a single predictive model to enhance overall performance and predictions while reducing bias and variance. Furthermore, combining deep transfer learning models with fuzzy ensemble approaches will assist in increasing the accuracy and robustness of a detection system. The presented study considered the following.
1. CXR images were used to diagnose coronavirus, and a transfer learning and fuzzy ensemble fusion model was proposed for COVID-19 detection.
2. At first, we trained the VGG16, GoogleNet, and Xception base classifiers using deep transfer learning techniques. Using fully connected layers and the softmax layer, we extracted features and classified CXR images using pre-trained transfer learning models.
3. Then, using Majority Voting, Sugeno Integral, and Choquet fuzzy integral, the data prediction scores from multiple models were integrated to create the final forecasted labels that are more accurate than the separate model predictions.
4. For the application of Choquet fuzzy integral with other state-of-the-art models, such as Majority Voting and Sugeno Integral, we considered the validation accuracy of each classifier to compute the fuzzy membership values of each classifier. The advantage of this type of fusion is that it creates the final prediction of each sample using adaptive weights depending on each sample's confidence scores.
5. The performance of each pre-trained model, including VGG16, GoogleNet, and Xception combined individually with Majority Voting, Sugeno Integral, and Choquet fuzzy ensemble techniques was measured in terms of precision, recall, F1-Score, sensitivityand specificity.
The goal of this article was to use CXR images to predict coronavirus, aiding healthcare providers and radiologists in the early detection and isolation of positive coronavirus patients to halt the virus' rapid spread in vulnerable countries with limited hospital capacity and a low patient-to-doctor ratio and to prevent increasing fatality rates.
The remainder of the article is organized as follows. The preliminary models are provided in Section 2. Section 3 describes the proposed algorithm. Section 4 provides a brief explanation of the proposed model and comparison analysis. Section 5 presents the conclusion and future works.

Preliminaries
VGG16: VGG16 was proposed by Karen Simonyan et al. [36] in 2014, which was termed based on its 16 layers. VGG-16 contains different layers, such as three dense layers, two FL, five max-pooling layers, and 13 convolutional layers.
GoogleNet: With a reported performance of 93.3%, GoogleNet [37] succeeds at the leading edge of the ImageNet detection and classification problem. The greater utilization of computer resources in the model is its advantage, in which a construction component is treated as an "inception module" that reflects the model's increased width and depth. This was one of the earliest CNN designs that diverge from the traditional method of just adding convolution and pooling layers in a sequential manner. Figure 1 illustrates the GoogleNet architecture, which is composed of nine initialization modules. GoogleNet is a 22 deep CNN that is a version of Google researchers' Inception Network that is built on the Inception architecture. In this case, we utilized the third version of the GoogleNet model, Inception V3. This version is made up of a hierarchy of sophisticated inception modules that integrate channel spatial convolution, pooling, and reprojection operations at different scales in each module. The model shrinks the parameter space by breaking spatial convolutions with bigger filter sizes n × n into a series of two convolutional operations with n × 1 and 1 × n filter sizes. The resultant network model is more complicated and deeper than AlexNet, although it has fewer features and lower computing complexity than AlexNet. GoogleNet does not employ fully-connected layers. Instead, the final convolutional map is subjected to channel-wise average activation values, and global average pooling values of the 2048 channels are utilized as the input image's feature vector.
Appl. Sci. 2021, 11, x FOR PEER REVIEW 4 of 14 modules that integrate channel spatial convolution, pooling, and reprojection operations at different scales in each module. The model shrinks the parameter space by breaking spatial convolutions with bigger filter sizes n × n into a series of two convolutional operations with n × 1 and 1 × n filter sizes. The resultant network model is more complicated and deeper than AlexNet, although it has fewer features and lower computing complexity than AlexNet. GoogleNet does not employ fully-connected layers. Instead, the final convolutional map is subjected to channel-wise average activation values, and global average pooling values of the 2048 channels are utilized as the input image's feature vector. Xception: The Xception [38] architecture includes a depth-wise separable convolution layer, convolutional layer, residual connections, and inception module. The design of Xception, which stands for "extreme inception", has 36 convolution layers that form the network's extractor base. Xception is an extension of Inception architecture by using deeply separable convolutions for standard Inception modules and, thus, has improved speed and accuracy compared to Inception, ImageNet, Inception V3, ResNet-101, Res-Net-50, and VGGNet models. The Xception model contains three major sections: entry, middle, and exit flow. The data first enter the input flow, and then go through the eight-fold middle flow, and finally the exit flow. Batch normalization is accompanied by all the Convolution layers. Xception replaced the conventional starting block by expanding it and changing the various (1 × 1, 5 × 5, 3 × 3) spatial dimensions with a (3 × 3) single dimension preceded by a (1 × 1) transition to maintain computational burden. Equation (1) and Equation(2) represent the mathematical modelling of Xception: Choquet fuzzy: In this study, aggregation refers to the process of merging each classifier's performance scores into a single global score. For the said purpose, an aggregation operator was used. Normally, we consider weighted average as the aggregation operator; however, in this situation, we employed the Choquet fuzzy integral [39]. This approach has already been performed in a variety of pattern identification applications due to its benefit of capturing the unpredictability that exists in the decision values obtained as extra data via classifier fusion, which is lacking in traditional ensemble approaches. As a result, fuzzy measures are the extension of aggregation operations on a set of confidence values. Weighted values are assigned to distinct classifier combinations in Xception: The Xception [38] architecture includes a depth-wise separable convolution layer, convolutional layer, residual connections, and inception module. The design of Xception, which stands for "extreme inception", has 36 convolution layers that form the network's extractor base. Xception is an extension of Inception architecture by using deeply separable convolutions for standard Inception modules and, thus, has improved speed and accuracy compared to Inception, ImageNet, Inception V3, ResNet-101, ResNet-50, and VGGNet models. The Xception model contains three major sections: entry, middle, and exit flow. The data first enter the input flow, and then go through the eight-fold middle flow, and finally the exit flow. Batch normalization is accompanied by all the Convolution layers. Xception replaced the conventional starting block by expanding it and changing the various (1 × 1, 5 × 5, 3 × 3) spatial dimensions with a (3 × 3) single dimension preceded by a (1 × 1) transition to maintain computational burden. Equation(1) and Equation (2) represent the mathematical modelling of Xception: Choquet fuzzy: In this study, aggregation refers to the process of merging each classifier's performance scores into a single global score. For the said purpose, an aggregation operator was used. Normally, we consider weighted average as the aggregation operator; however, in this situation, we employed the Choquet fuzzy integral [39]. This approach has already been performed in a variety of pattern identification applications due to its benefit of capturing the unpredictability that exists in the decision values obtained as extra data via classifier fusion, which is lacking in traditional ensemble approaches. As a result, fuzzy measures are the extension of aggregation operations on a set of confidence values. Weighted values are assigned to distinct classifier combinations in these fuzzy measures. To use the Choquet fuzzy approach, we must first establish the fuzzy measure values that define the strength of each classifier. We include three different types of interaction in this section: are two subsets of classifiers and mutually exclusive. f (S) indicates the strength of a set of classifiers.
After obtaining a complete set of fuzzy measure scores and relevant scores for each classifier, we can utilize the Choquet fuzzy method to compute the classifiers' aggregated global score. Let g () generate the fuzzy measure score of a collection of classifiers D = (d 1 , d 2 , d 3 . . . .., d n ), and P = (p 1 , p 2 , p 3 . . . .., p n ) are the individual classifier performance scores in D. Assume L i is a subset of D, such that where i is in between 1 and n. This indicates that, if i = 1 then L 1 = (d 1 ), and if i = 3, then , then the aggregated score may be computed using the Choquet fuzzy integral, as shown in Equation (3): We compute the value of λ for determination of the fuzzy membership values using Equation (4): By obtaining the roots of the characteristic equation, we can find the value of λ. By repeating Equation (5), the fuzzy membership value of each classifier combination can be determined: We have p classifiers that are used to categorize q classes when dealing with classifier combinations. Let x ij denote the confidence score of the jth class of the ith classifier. Choquet integral is now utilized for each j to produce a fuzzy confidence score. In this scenario, g({b i }) represents each classifier's fuzzy membership scores, which are empirically In this study, however, we set the fuzzy measure g({b i }) = w i /w 1 + w 2 + w 3 . . . . + w p , where w i indicates the validation accuracy of the ith classifier. Even after the values of fuzzy measures have been set, the Choquet fuzzy integral can modify the weightage of each classifier by considering the decision scores supplied by the rest of the classifiers. This makes the system dynamic, as opposed to other assembly techniques that employ the majority voting and sugeno integral system.
We must first obtain fuzzy measure values for each classifier before we can use the Choquet fuzzy ensemble. The fuzzy measure values indicate the strength of each classifier and all possible classifiers. Algorithm 1 depicts the key phases of the proposed methodology.
Majority Voting: MV is a basic but effective ensemble learning approach that has been effectively utilized in a variety of industries. The fundamental concept behind MV is to use the majority voting rule to aggregate the output results from many data sources [40]. The majority voting rule is divided into three types: (i) unanimous voting, in which all classifiers must agree on a prediction; (ii) simple majority voting, in which the prediction must be supported by at least one more than half of the classifiers; and (iii) plurality or majority voting, in which the highest number of votes is considered for the ensemble decision whether the sum of those exceeds 50%. In the case of independent classifier outputs, the majority voting rule combiner always improves prediction performance. The MV technique works by accumulating the labels associated with each classifier in the input image. The final label is then assigned to the class with the most labels (votes) among the classifiers. If two classes receive equal votes, we choose to assign the final label to the class with the lowest index.
Sugeno Integral: Takagi-Sugeno [41] is a fuzzy inference technique that creates a systematic strategy for generating fuzzy rules from a given input-output dataset. The inputs are blurry, but the outcome is clear. Takagi-Sugeno computes the crisp output using a weighted average. This method is computationally more efficient and may be used in conjunction with optimization and adaptive approaches.

Methodology
Herein, we present a model that employs the deep transfer learning approach to classify CXR images as COVID-positive, negative or pneumonia. This approach first employs pre-trained transfer learning models from VGG16, GoogleNet, and Xception as feature extractors, then uses classifiers to classify the features. The classifier outputs are subsequently merged using Majority Voting, Sugeno Integral, and Choquet fuzzy integral, whereby each classifier's fuzzy measures are paired with an empirically established ratio. We used two conventional ensemble techniques (Majority Voting and Sugeno Integral) to make a comparison of the Choquet fuzzy ensemble approach for COVID-19 patient identification based on the CXR dataset. We evaluated these models for consistent performance comparison. Figure 2 depicts the proposed ensemble model. Sugeno Integral: Takagi-Sugeno [41] is a fuzzy inference technique that creates a systematic strategy for generating fuzzy rules from a given input-output dataset. The inputs are blurry, but the outcome is clear. Takagi-Sugeno computes the crisp output using a weighted average. This method is computationally more efficient and may be used in conjunction with optimization and adaptive approaches.

Methodology
Herein, we present a model that employs the deep transfer learning approach to classify CXR images as COVID-positive, negative or pneumonia. This approach first employs pre-trained transfer learning models from VGG16, GoogleNet, and Xception as feature extractors, then uses classifiers to classify the features. The classifier outputs are subsequently merged using Majority Voting, Sugeno Integral, and Choquet fuzzy integral, whereby each classifier's fuzzy measures are paired with an empirically established ratio. We used two conventional ensemble techniques (Majority Voting and Sugeno Integral) to make a comparison of the Choquet fuzzy ensemble approach for COVID-19 patient identification based on the CXR dataset. We evaluated these models for consistent performance comparison. Figure 2 depicts the proposed ensemble model.

Algorithm 1 Detection of COVID-19 +ve and -ve Patients.
Step 1: As input, use CXR images (COVID-positive, negative or pneumonia images) Step 2: Resize the size of the training pictures to 224 × 224 × 3 pixels Step 3: Load pre-trained models like VGG16, GoogleNet and Xception. Let the feature maps for VGG16, GoogleNet and Xception are denoted as FM1, FM2 and FM3 respectively Step 4: In training phase of VGG16, we use GlobalAveragePooling2D followed by FCL with dropout 0.5, FCL with ReLU 98 and finally FCL with softmax as output layer Step 5: In training phase of GoogleNet and Xception, we use FCL with dropout 0.5 followed by FCL with ReLU 98 and finally FCL with softmax as output layer Step 6: Create the classifiers CF1, CF2 and CF3 by layering two FC layers on top of feature maps and a softmax layer.
Step 7: Divide all of the images in the dataset into training and testing sets in proportions of 80% and 20%, respectively.
Step 8: Using the training dataset, train the classifiers CF1, CF2 and CF3 Step 9: Using the validation set, produce validation accuracies for the w 1 , w 2 , w 3 feature maps Step 10: Determine the fuzzy measure values for the k th classifier is w i /w 1 + w 2 + w 3 Step 11: Resize the size of the testing pictures to 224 × 224 × 3 pixels.
Step 12: Based on the testing samples, use the learned classifiers to create prediction scores P.
Step 13: Using fuzzy measures F, apply Choquet Fuzzy to the prediction scores P to get the final prediction scores.
Step 14: Use final prediction scores to determine if a particular sample is COVID-19 positive, negative or pneumonia.
2D Global Average Pooling is employed on the spatial dimensions until each spatial dimension is equal to one. To avoid overfitting, FCL with a dropout of 0.5 is utilized. FCL combined with ReLU 98 is then utilized to enhance neural networks by accelerating training. The FCL function with the softmax function is used to normalize the outputs between 0 and 1.

Experimental Setup
The entire experiment was carried out on Google Colaboratory using a Python3 engine, a GPU as a backend, and a system RAM of 12 GB. TensorFlow 1.x was employed. For image processing, we loaded the picture as a color image with three RGB channels and transformed it into a NumPy array. On the CXR dataset, we trained all three transfer learning models with an identical set of hyperparameters. During the entry procedure, the images were scaled to 224 × 224 pixels with various dimensions. For training purposes, we setthe learning rate to 2 × 10 −4 with 60 epochs, which are small enough to prevent the fitting problem of the transfer learning models. An Adam optimizer was used as a compilation, and FCL with ReLU 98 was employed as the activation function for feature extraction.

Dataset Description
We obtained the COVID-19 CXR image dataset from Kaggle [42], which includes equal number of COVID positive, pneumonia and normal CXR images. Each class contains 2313 images. The active COVID-19, pneumonia and normal patient CXR images were used to train the Tensor Flow and Keras deep learning models to detect whether or not a person has COVID-19. The data were divided into 80% for the training set and 20% for the test sets, with the identical sets utilized for all models. Figures 3 and 4 show the accuracy and loss graphs of the transfer learning models (VGG16, GoogleNet, and Xception). Specifically, Figure 3 indicates that Xception learns faster than GoogleNet and VGG16 and excelled in the competition in terms of accuracy. Early halting and model checkpoints were used to determine the best model and stop training if there was no substantial gain in accuracy. The ImageNet weights for the models were set. In Figure 4, the loss for the above three models is depicted, whereby the loss rate of VGG16 was initially very high but gradually declined, while the loss rate of Xception exhibited a smooth descent. For all three models, the loss was <0.1. We compared the performance of the Choquet fuzzy integral-based ensemble strategy for coronavirus detection to two traditional ensemble approaches (Majority Voting and Sugeno Integral) using the current dataset.

Results and Outcomes
for the test sets, with the identical sets utilized for all models. Figures 3 and 4 show the accuracy and loss graphs of the transfer learning models (VGG16, GoogleNet, and Xception). Specifically, Figure 3 indicates that Xception learns faster than GoogleNet and VGG16 and excelled in the competition in terms of accuracy. Early halting and model checkpoints were used to determine the best model and stop training if there was no substantial gain in accuracy. The ImageNet weights for the models were set. In Figure 4, the loss for the above three models is depicted, whereby the loss rate of VGG16 was initially very high but gradually declined, while the loss rate of Xception exhibited a smooth descent. For all three models, the loss was <0.1. We compared the performance of the Choquet fuzzy integral-based ensemble strategy for coronavirus detection to two traditional ensemble approaches (Majority Voting and Sugeno Integral) using the current dataset.

Performance Analysis
The performance of each model was evaluated in terms of recall, precision, F1-Score, sensitivity, and specificity, and the results are represented in Table 1. Confusion matrices of the VGG16, GoogleNet, and Xception-associated fuzzy ensemble techniques are illustrated in Figures 5-7, respectively. Performance comparisons of various transfer learning for the test sets, with the identical sets utilized for all models. Figures 3 and 4 show the accuracy and loss graphs of the transfer learning models (VGG16, GoogleNet, and Xception). Specifically, Figure 3 indicates that Xception learns faster than GoogleNet and VGG16 and excelled in the competition in terms of accuracy. Early halting and model checkpoints were used to determine the best model and stop training if there was no substantial gain in accuracy. The ImageNet weights for the models were set. In Figure 4, the loss for the above three models is depicted, whereby the loss rate of VGG16 was initially very high but gradually declined, while the loss rate of Xception exhibited a smooth descent. For all three models, the loss was <0.1. We compared the performance of the Choquet fuzzy integral-based ensemble strategy for coronavirus detection to two traditional ensemble approaches (Majority Voting and Sugeno Integral) using the current dataset.

Performance Analysis
The performance of each model was evaluated in terms of recall, precision, F1-Score, sensitivity, and specificity, and the results are represented in Table 1. Confusion matrices of the VGG16, GoogleNet, and Xception-associated fuzzy ensemble techniques are illustrated in Figures 5-7, respectively. Performance comparisons of various transfer learning

Performance Analysis
The performance of each model was evaluated in terms of recall, precision, F1-Score, sensitivity, and specificity, and the results are represented in Table 1. Confusion matrices of the VGG16, GoogleNet, and Xception-associated fuzzy ensemble techniques are illustrated in Figures 5-7, respectively. Performance comparisons of various transfer learning models with the proposed model using CXR images are presented in Table 2 lowing equations were used to calculate the performance metrics:

Conclusions and Future Work
With the growing threat of new coronavirus strains throughout the world, early diagnosis of COVID-19 is critical especially due to the global scarcity of healthcare facilities. In this work, we propose a hybrid fuzzy ensemble-based method for detecting COVID-19-positive patients using CXR images. Ensemble learning can improve the overall performance by combining the prominent properties of its constituent models, resulting in better predictions than the individual contributing models. Moreover, ensemble models can enhance performance by lowering the variance of prediction errors by adding some bias to compete with base learners. In the proposed work, we used various transfer learning models, which were separately trained to make independent predictions. The models were then integrated to forecast a class value, utilizing a novel approach combining Majority Voting, Sugeno Integral, and Choquet Fuzzy. A publicly accessible database containing 6939 CXR pictures was employed to assess the effectiveness of the proposed method. The data were divided into training and test sets in proportions of 80% and 20%, respectively. The proposed method, Xception + Choquet Fuzzy ensemble methods, obtained a better accuracy of 99.57% compared to other transfer learning models. The suggested methodology failed to categorize 0.43% of test data, which may be due to the relatively poor performance of the individual transfer learning models. As a result, creating or selecting better individual transfer learning models may help to amend this issue. In addition, only three basic transfer learning models were applied in the aforementioned ensemble method. Therefore, adding more classifier techniques might be an additional option to improve the current method. To demonstrate its robustness across various domains, the proposed model might be applied to various classification-based image analyses. Such advancements will aid radiologists to develop a graphical user interface (GUI) for recognizing COVID-19 to keep up with the disease's various stages of progression and imagistic patterns.