An IoT-Based Deep Learning Framework for Real-Time Detection of COVID-19 through Chest X-ray Images

Abstract

Over the next decade, Internet of Things (IoT) and the high-speed 5G network will be crucial in enabling remote access to the healthcare system for easy and fast diagnosis. In this paper, an IoT-based deep learning computer-aided diagnosis (CAD) framework is proposed for online and real-time COVID-19 identification. The proposed work first fine-tuned the five state-of-the-art deep CNN models such as Xception, ResNet50, DenseNet201, MobileNet, and VGG19 and then combined these models into a majority voting deep ensemble CNN (DECNN) model in order to detect COVID-19 accurately. The findings demonstrate that the suggested framework, with a test accuracy of 98%, outperforms other relevant state-of-the-art methodologies in terms of overall performance. The proposed CAD framework has the potential to serve as a decision support system for general clinicians and rural health workers in order to diagnose COVID-19 at an early stage.

1. Introduction

The coronavirus disease of 2019, also known as COVID-19, was first reported on 31 December 2019, in Wuhan city of China and quickly spread worldwide [1]. The COVID-19 pandemic has resulted in approximately 6,459,684 deaths and more than 596,873,121 confirmed cases of COVID-19 infection as of August 2022 [2]. Along with physical health, COVID-19 also had a toll on mental health of people [3]. This has led to a major concern. The pandemic’s management is complicated by the daily increase of positive COVID-19 cases and incorrect diagnoses. Due to the urgent need to diagnose COVID-19 disease, the medical industry is on the lookout for new tools and strategies to track and manage the virus’s spread [4]. The Internet of Medical Things (IoMT) and the high-speed 5G network, in combination with artificial intelligence and deep learning techniques, could help the healthcare system for quick diagnosis procedures in order to monitor COVID-19 disease efficiently [5,6,7]. The IoMT has substantially shifted traditional healthcare systems to IoT-based digitized healthcare infrastructure [8]. In order to release the loads of healthcare, especially in a pandemic scenario, the IoMT provides real-time services in the healthcare system [9].

The real-time reverse transcription-polymerase chain reaction (RT-PCR) test is the most used technique for diagnosing COVID-19 [10]. However, this method of identification requires a lot of time, and the outcomes might contain a lot of false-negative mistakes. Many doctors regard chest X-rays to be one of the most basic diagnostic techniques [11]. It is inexpensive and useful in identifying pulmonary infections such as pneumonia, tuberculosis, early lung cancer, and now COVID-19. Accurate diagnosis of COVID-19 using X-ray images takes specialized expertise and experience. However, in the case of COVID-19 pandemic, the ratio of medical experts who can manually make this diagnosis to the number of patients is inadequate. Computer-aided diagnosis (CAD) systems might be an effective solution to address this deficiency during the COVID-19 pandemic [12]. In this paper, an Internet of Medical Things (IoMT)-based deep learning CAD framework is presented for COVID-19 detection, prescreening, and prevention through chest X-ray images. This CAD system will perform radiological diagnosis automatically after receiving chest X-ray images from the remote and rural healthcare centers. As a result, medical experts will be able to remotely review and monitor patients with minimal expense.

1.1. Objective

The central motive of the proposed framework is to design an IoMT-based deep learning healthcare application that can make the initial decision for COVID-19 detection. The proposed study has the following main objectives:

• To present an IoMT based framework for early detection and control of COVID-19 in remote and rural areas.
• To introduce an IoMT framework based on the light-weight deep ensemble CNN architectures for COVID-19 diagnosis using chest X-ray images.
• To evaluate the performance of the proposed ensemble model using a benchmark chest X-ray image dataset.

1.2. Contributions

The main contributions of the proposed work are summarized as follows:

• An IoMT-based architecture is introduced for the early detection of COVID-19. The IoT devices (X-ray machine) first collect chest X-ray images and then directly feed into a deep learning architecture for the detection of COVID-19.
• Proposed a light-weight deep ensemble CNN architecture with fine-tuned models such as Xception, ResNet50, DenseNet201, MobileNet, and VGG19 to improve the training efficiency and guarantee significant model accuracy.
• Conducted extensive experiments based on the benchmark chest X-ray image dataset and have a comparative evaluation with other deep learning approaches for COVID-19 detection.
• Integrated the proposed framework with computer-aided diagnosis (CAD) systems (GUI tools/Mobile App) in order to automatically screen COVID-19 from low-contrast X-ray images. As a result, dependence on radiologists to detect COVID-19 is also reduced.
• The proposed framework can be used for detection of bacterial and viral pneumonia from chest X-ray images.

The remainder of this paper is structured as follows: the different related studies made in this field are described in the next section. The deep ensemble CNN approach for COVID-19 detection is presented in the Materials and Methods section. Evaluations of the suggested framework’s performance are included in the Results and Discussion section. The summary of the entire effort and the future scope are included in the Conclusion section.

2. Related Work

In the midst of the continuing COVID-19 epidemic, IoMT offers a number of automated services and facilities in order to support patient monitoring [13]. To monitor and assist control the virus’s spread, researchers are experimenting with new methods and technologies [14,15,16]. For instance, Ahmad et. al. presented a mobile technology based solution for early exposure and prompt medical intervention to stop the spread of COVID-19 [17]. Awotunde et al. [18] design an intelligent Edge-IoMT-based system for fighting COVID-19 pandemic. Singh et al. [19] provided an example of using the IoMT technique to combat the continuing COVID-19 epidemic while treating orthopedic patients.

Another IoT-based system was developed by Bhardwaj et al. [20] for earlier diagnosis and treatment COVID-19 patients by real-time tracking of a person’s temperature, heart rate, blood pressure, and oxygen saturation etc. Shibly et al. have presented a deep learning based approach for detecting COVID-19 using chest X-ray images [21]. In the literature, the authors employed faster regions with CNNs (Faster-RCNN) with VGG-16 (visual geometry group) based CNN model with Faster-RCNN architecture [22]. Turkoglu [23] opted for the transfer learning approach using the AlexNet model for feature extraction. They selected only the most effective features using Relief feature selection methodology from all layers of AlexNet and fed those effective features into the SVM classifier for COVID-19 detection. Narin et al. [24] evaluated five pre-trained convolutional neural network models such as InceptionV3, ResNet50, ResNet101, ResNet152, and Inception-ResNetV2 for detection of COVID-19 disease, based on CXR images. In the work, the authors reported that the ResNet50 model provides the highest accuracy among used five models. Singh et al. [25] presented a CNN model to detect COVID-19 patients using a multi-objective adaptive differential evolution (MADE) approach.

Moura et al. [26] utilized six pre-trained CNN architectures (DenseNet-121, DenseNet-169, ResNet-18, ResNet-34, VGG-16, and VGG-19) for the classification of CXR images in 3 different categories: COVID-19, pneumonia, and healthy cases.

It can be said that the accuracy of diagnosis in the research stated above and a large number of others is not very noteworthy and can yet be enhanced. Compared with previous related works, the significant contribution of the presented work is the use of IoMT based framework and a majority voting deep ensemble CNN (DECNN) model in order to achieve better results in real-time COVID-19 detection.

3. Materials and Methods

The IoT-based architecture for COVID-19 identification is presented in this section. Figure 1 illustrates the overall IoT-based framework. The presented framework is composed of three smaller systems. The first is the X-ray image collection unit which collects a patient’s chest X-ray (CRX) images using X-ray machines. The second is the data processing unit which is located in the computational cloud. The third includes the physicians, nurses, and the patients, who have been granted access to the system by smartphones, computers, or any other devices.

The collected CRX images are sent to the computational cloud for processing by the deep ensemble CNN model in order to classify the X-ray images into COVID-19, normal, and pneumonia.

3.1. Structure of the CNN

A convolutional neural network (CNN) is a deep learning model that has a great potential to produce successful results, especially in the area of image analysis and classification. CNNs have significantly increased their capacity to carry out important medical diagnosis. CNN is made up of multilayered neural networks in which at least one of its layers use convolution operation. As illustrated in Figure 2, a convolutional neural network is made up of following components: (i) a convolution layer, (ii) a pooling or subsampling layer, (iii) an activation function, and (iv) fully connected layer.

(i) Convolution layer: The convolution layer performs convolutional filtering by a small sliding filter on an input image. Convolutional filtering on the 2D matrix representing the image (I) of size MxN with the smaller 2D filter w of size mxm is given in Equation (1):

$$Y_{i,j}={\displaystyle \sum }_{p=0}^{m-1}{\displaystyle \sum }_{q=0}^{m-1}{I}_{i+p,j+q}{w}_{p,q}$$

(1)

The size of $Y_{i,j}$ is given as $\left(M-2\left|\frac{m}{2}\right|\right)\times \left(N-2\left|\frac{m}{2}\right|\right)$.

(ii) Pooling layer: This layer performs pooling operation with a stride factor of “k” on a “k-by-k” square window sliding on the 2D feature map to obtain one single value of the features with certain scheme (max by max pooling or average by average pooling). As a result, it effectively reduces the dimension of the feature maps while preserving their channel count.

(iii) Activation function: The aim of the activation function is to increase the non-linearity of the model’s output. Commonly used activation function is the rectified linear unit (ReLU), which is defined in Equation (2):

ReLU(x) = max(x, 0)

(2)

Additionally, when doing classification, the softmax function is generally applied in the last layer. The Equation (3) defines softmax function where k represents the number of labels involved in classification.

$$\mathrm{softmax}\left(x_i\right)=\frac{e^{x_i}}{{{\displaystyle \sum }}_{j=0}^k{e}^{x_k}}$$

(3)

(iv) Fully connected (FC) layer: commonly, fully connected layer is located at the end of each CNN model. The purpose of FC layer is to compile the features that the earlier convolution layers had collected, and then to produce a forecast. As shown in Figure 2, the output of the fully connected layer represents the final CNN output.

3.2. Loss Functions

A loss function is a measure of the effectiveness of a deep learning model. Loss function is used in the output layer in order to reduce the error between the predicted and the actual output using the CNN learning technique. One of the ways to minimize the loss is categorical cross entropy function [27]. The categorical cross entropy loss function (L) is computed using Equation (4).

$$L=-\frac{1}{N}{\displaystyle \sum }_{i=1}^N{y}_i\log \widehat{y_i}$$

(4)

where N denotes the number of classes in the problem, $y_i$ is the corresponding target value.

Some state-of-the-art deep CNN models for image classification, such as VGG, ResNet, and so on, are introduced in

Table 1. Brief overview of some state-of-the-art CNN architectures.

	VGG16	Resnet50	MobileNetV2	DenseNet201	Xception
Input Size	224 × 224	224 × 224	224 × 224	224 × 224	299 × 299
Depth	16	50	53	201	71
Year	2014	2015	2018	2017	2017
No. of Parameters	135 M	24 M	2 M	20 M	22.9 M

.

3.3. X-ray Image Dataset

In our study, we have used anterior-to-posterior (PA)/posterior-to-anterior (AP) view of CXR images since radiologists frequently use this view of radiography for clinical diagnosis. The dataset used in our work is collected from Kaggle [28,29]. It consists of 12,443 images and these images are divided into three different classes—“Normal”, “Covid positive”, and “Viral Pneumonia”. There are 906 Covid positive images, 1345 viral pneumonia images and 10,192 normal images. These datasets are further split into training and testing data with a percentage ratio of 80% training data and 20% testing data. Moreover, we have also tested the proposed framework on another dataset [30].

3.4. Pre-Processing and Data Augmentation

As the part of data preprocessing, bilateral filter [31] has been used to extract more features from the images. The bilateral filter is a useful tool to smooth images by removing noises while preserving edges. Subsequently, different flips and rotation are applied while executing data augmentation in order to aid more images in the training phase for better learning.

3.5. Detection of COVID-19 Using Deep Ensemble CNN (DECNN)

An ensemble method is a way to combine different learning approaches into a single predictive model. In order to develop a real-time automated system for COVID-19 detection from CRX images, deep ensemble CNN (DECNN) models have been trained and tested. As shown in Figure 3, five different models were employed in the proposed ensemble framework—Xception, ResNet50, DenseNet201, MobileNetV2, and VGG16—all of which were pre-trained on ImageNet and then fine-tuned using the chest X-ray image dataset.

The ensemble approach adopted for the proposed work is the majority voting classifier-based ensemble (MVCE) model. MVCE learning model makes use of a variety of machine learning algorithms to discover the correct label for data and to forecast which option will receive the most votes. Algorithm 1 demonstrates the proposed algorithm.

Algorithm 1: COVID19_detection (Proposed deep CNN algorithm)

Input: Chest-X-ray images  Output: The trained model that classifies the CXR images  Steps  1. Import a set of pre-trained models  E = {Xception, Resnet50, Dense201, MobileNetV2 and VGG16}.  2. For each model in E  3.  Replace the last fully connected layer of each model by a layer  of dimension (3 × 1)  4.  for epochs = 1 to 20  5.    model.fit (train_data, validation_data)  6. Return Class (COVID19/Normal/Pneumonia)

4. Rsults and Discussion

This section presents the evaluation and visualization results of the proposed ensemble model.

4.1. Tools Used

Deep CNN model is implemented over the deep learning library TensorFlow 2.2.0. The Google Colab GPU (Tesla K80 12GB GDDR5 VRAM) platform is utilized for training and testing purposes.

4.2. Performance Metrics

The performance has been evaluated using four metrics—precision, accuracy, F1-score, and recall. Confusion matrix is shown in the Figure 4. Given the number of true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN), these metrics are evaluated based on the confusion matrix as follows:

Accuracy: It is defined as the proportion of right predictions to the total number of predictions made on a given set of data as given below:

$$\mathrm{Accuracy}=\frac{\mathrm{TP}+\mathrm{TN}}{\left(\mathrm{TP}+\mathrm{FN}\right)+\left(\mathrm{FP}+\mathrm{TN}\right)}$$

Precision: It is the ratio of successfully predicted positive data points to the number of data points correctly predicted as positive by the classifier, as expressed by the following formula:

$$\mathrm{Precision}=\frac{\mathrm{TP}}{\left(\mathrm{TP}+\mathrm{FP}\right)}$$

Recall (Sensitivity): It is the proportion of accurately anticipated positive data points to the total number of positive data points. It is defined as:

$$\mathrm{Recall}=\frac{\mathrm{TP}}{\left(\mathrm{TP}+\mathrm{FN}\right)}$$

F1-score: It is used to determine the accuracy of a test set. The harmonic mean of precision and sensitivity is the F1 score.

$$\mathrm{F}1\text{-}\mathrm{score}=\frac{2\times \mathrm{Recall}\times \mathrm{Precision}}{\mathrm{Recall}+\mathrm{Precision}}$$

The training and validation accuracy and loss are presented in Figure 5, Figure 6, Figure 7, Figure 8 and Figure 9. It can be observed that the training and validation accuracy rates considerably increased at the first epoch. However, training and validation loss are not decreasing steadily except ResNet50.

A simple GUI-based COVID-19 detection tool has been developed for online and real-time detection and monitoring. Figure 10 and Figure 11 demonstrates the GUI-based and Mobile App-based COVID-19 detection tool.

Table 2. Experimental results.

Class	Precision	Recall	F1-Score
COVID-19	0.95	0.99	0.97
Normal	1.00	0.97	0.98
Viral Pneumonia	0.81	0.98	0.95

shows the precision, recall and F1-score of the proposed deep CNN for 3-class problems.

The accuracy results of the proposed deep ensemble CNN model are compared with other existing state-of-the-art methods, as provided in

Table 3. Comparison of the proposed work with other DL-based state-of-the-art methods.

Previous Study	Year	Type of Image	Model Used	Accuracy (%)			GUI Tool/Mobile App for Real Time Testing
				2 (COVID-19/Normal)	3 (COVID-19/Normal/Pneumonia)	3 (Bacterial Pneumonia, Normal, Viral Pneumonia)
Narin et al. [24]	2021	Chest X-ray	Deep CNN ResNet-50	96.1	×	×	No
Khanna et al. [34]	2021	Chest X-ray	Hybrid LSTM-CNN	96.46	×	×	No
Chakraborty et al. [32]	2022	Chest X-ray	Transfer Learning(VGG-19)	×	97.11	×	No
Nasiri and Hasani [33]	2022	Chest X-ray	DenseNet169 & XGBoost	98.24	89.70	×	No
Proposed System	2022	Chest X-ray	Deep ensemble CNN (Xception, ResNet50, DenseNet201, MobileNetV2 and VGG16)	98.89	97.16	96.21	Yes

. It can be observed that the proposed DECNN model outperformed with respect to two important aspects which are missing in other recent works reported in Table 3. First, most of the works considered 2-class problems. A transfer learning-based model for 3-class problems in COVID-19 identification has been created by Chakraborty et al. [32], although their accuracy is worse than ours. Moreover, Nasiri and Hasani [33] consider 3-class problems along with 2-class, the accuracy of 3-class problems is significantly low (less than 90%). Second, no work has reported implementation of GUI tools/Mobile App to investigate COVID-19 in real-time.

5. Conclusions

In this paper, we present an Internet of Things (IoT) based CAD framework for the online and real-time COVID-19 detection through CXR images applying deep ensemble CNN in order to identify COVID19, normal, pneumonia patients. The proposed scheme achieves a better overall performance in accurately detecting the COVID-19 compared to the other existing state-of-the-art methods as shown by performance simulation. This is due to the fact that the proposed IoT based framework utilizes deep ensemble CNN models. As a result, the presented framework has the potential to serve as a decision support system for general practitioners and rural health workers for the early detection of COVID-19. However, there are certain shortcomings in the presented work that need additional implementation. The proposed system is tested only on pre-captured CXR images. In future, the proposed work will be tested using real-time captured CXR images.