IoMT with Deep CNN: AI-Based Intelligent Support System for Pandemic Diseases

Abstract

The Internet of Medical Things (IoMT) is an extended version of the Internet of Things (IoT). It mainly concentrates on the integration of medical things for servicing needy people who cannot get medical services easily, especially rural area people and aged peoples living alone. The main objective of this work is to design a real time interactive system for providing medical services to the needy who do not have a sufficient medical infrastructure. With the help of this system, people will get medical services at their end with minimal medical infrastructure and less treatment cost. However, the designed system could be upgraded to address the family of SARs viruses, and for experimentation, we have taken COVID-19 as a test case. The proposed system comprises of many modules, such as the user interface, analytics, cloud, etc. The proposed user interface is designed for interactive data collection. At the initial stage, it collects preliminary medical information, such as the pulse oxygen rate and RT-PCR results. With the help of a pulse oximeter, they could get the pulse oxygen level. With the help of swap test kit, they could find COVID-19 positivity. That information is uploaded as preliminary information to the designed proposed system via the designed UI. If the system identifies the COVID positivity, it requests that the person upload X-ray/CT images for ranking the severity of the disease. The system is designed for multi-model data. Hence, it can deal with X-ray, CT images, and textual data (RT-PCR results). Once X-ray/CT images are collected via the designed UI, those images are forwarded to the designed AI module for analytics. The proposed AI system is designed for multi-disease classification. It classifies the patients affected with COVID-19 or pneumonia or any other viral infection. It also measures the intensity level of lung infection for providing suitable treatment to the patients. Numerous deep convolution neural network (DCNN) architectures are available for medical image classification. We used ResNet-50, ResNet-100, ResNet-101, VGG 16, and VGG 19 for better classification. From the experimentation, it observed that ResNet101 and VGG 19 outperform, with an accuracy of 97% for CT images. ResNet101 outperforms with an accuracy of 98% for X-ray images. For obtaining enhanced accuracy, we used a major voting classifier. It combines all the classifiers result and presents the majority voted one. It results in reduced classifier bias. Finally, the proposed system presents an automatic test summary report textually. It can be accessed via user-friendly graphical user interface (GUI). It results in a reduced report generation time and individual bias.

1. Introduction

In late 2019, many people were affected by unexplained pneumonia cases that spread quickly across the country and globally. This unknown novel virus created a lot of problems in the human body, such as acute breathing distress, multi-organ failure, cough, smell loss, etc. [1,2]. After long research, the WHO announced that pneumonia was caused by a novel coronavirus infection. It was identified as an international public health emergency in January 2020 by the WHO. Due to the rapid spreading rate of novel coronavirus, more than 618 million of peoples were affected, and 4.9 million of peoples died globally. If we had enough medical infrastructure for early screening, we could limit the mortality rate drastically. The early detection of patients suspected of being infected plays a vital role in the prevention and fight against new cases of corona pneumonia. The arrival of the new COVID-19 delta strain makes the situation worsen, as rates of transmission are notably high. Furthermore, all the countries became stuck in choosing the vaccine. The only opportunity that we have to minimize the spread (and stop it early) is to correctly diagnose the infected individuals. Numerous medical and technical research was conducted for precise diagnosis. Finally, the medical community announced RT-PCR test for finding COVID positivity. Reverse transcription polymerase chain reaction (RT-PCR) is used to determine COVID positivity. Even though RT-PCR is considered a standard diagnosis tool by World Health Organization (WHO), it has an accuracy of only about 80%. The difficulty with RT-PCR is that it yields a negative result if it is performed after the life span of the COVID-19 virus. In such cases, to determine the severity of the infection (stage), CT (computed tomography) and X-ray imaging are commonly utilized in conjunction with RT-PCR to ascertain the intensity level (stage) of the disease and the treatment plan. Some of the challenges in getting timely and accurate treatments are (i) infection interpretation by the radiologist is a very introspective endeavor, invariably prone to individual bias and therapeutic practices. (ii) A lack of medical infrastructure, particularly in rural areas, prevents patients from receiving timely care. In this regard, numerous technical research is performed in deep learning approaches for pneumonia, viral, and COVID-19 prediction and classification. They have taken CT images and X-ray images for training and testing their system. Researchers experimented with lots of AI techniques for better image classification. Though AI techniques, especially neural network architectures, yield better image classification and prediction, for obtaining better results, it needs massive volume of images. In the proposed system, we used deep convolution neural network architectures for image classification. The challenge which we have faced is data collection. As the collected data is not sufficient for getting better accuracy, we used Keras image data generator for generating synthetic data. With the help of data augmentation, we generated sufficient data. It helps to get better classification results. Some researchers are working on evolution and impacts of the diseases. In [3], the authors proposed a logical framework for studying the evolution of neurological diseases using answer set programming (ASP). They have performed exploratory research on ANN and ASP for the evolution of neurological diseases. From the experimentation, it is observed that the combination of ANN and ASP helped to explore the connectome and test the impact of its variations of the neurological diseases. Though, deep convolution neural network architectures provide better accuracy for image classification. Some of the researchers working on quantum convolution neural networks for better classification. To improvise the accuracy of the convolution neural network architectures further, recently many researchers [4,5,6,7] have worked on integration of quantum computing with neural network architectures, which give increased accuracy, especially for image classification. From the study, it was observed that all the systems that were developed were treatment support systems. It would be useful for the medical community in treating the patients and simplifying their tasks. It is not able to address the problems of society directly. We presented an automated system for assisting the people who are affected with COVID-19. As it can generate rapid results, we can stop the spreading rate drastically. It results in reducing the mortality rate with minimal medical infrastructure. The Internet of Things (IoT) is a field of automation. As automation gives precise results, all the fields from retail business to satellite communication are moving towards automation. It becomes an integral part of in every walk of life, and the medical field could no longer be exempted from automation [8]. Though, health care systems are more sensitive, and it completely depends on medical experts’ advice in giving treatments and taking clinical decisions, emerging technologies such as artificial intelligence (AI) break the barrier, due to its accuracy [9]. Furthermore, because of its self-learning characteristics, an AI-based system can also predict the epidemiological pattern. Future forecasting is one of the important features of an AI-based system. It majorly depends on the previous history of clinical data. However, our conventional medical system uses hospital management systems to manage the health records of the patients [10]. These systems create huge amounts of data in varied forms. However, this data is seldom visited and remains untapped. Hence, in this scenario, efforts are necessary to make intelligent decisions. The diagnosis of the diseases using different features or symptoms is complex. One of the effective solutions is to use artificial intelligence techniques to reduce data ambiguity and provide good decision making [11]. In general, the Internet of Things connects different objects (different attributes) together for detailed analysis. It results in the automation of decision making. The Internet of Medical Things (IoMT) is an extended version of IoT. It mainly concentrates on the integration of medical things for servicing the needy people who cannot get medical services easily, especially rural area people and aged peoples living alone. The IoMT creates a platform for collecting medical attributes directly from people (patients) with the help of some sensors or a user interface that will be analyzed by using AI-based techniques [12]. It will give the proper guidelines to the patients and assistance to the medical experts for giving timely treatment to save their lives. In our country, the medical infrastructure is not ample. Especially during the peak period of COVID spread, most of the people lost their lives because of a shortage of medical infrastructure, such as medical equipment’s, doctors, healthcare takers, etc. If we had an option to provide the health care support to the remote/needy people with the help of IoMT, we could have saved most of the lives. Hence, we designed an automated system for pandemic disease prediction for early alarming. The rest of the paper is organized as follows. The related work on IoMT and AI approaches for image classification is discussed in Section 2. The proposed IoMT system design for pandemic diseases is discussed in Section 3. A detailed experimentation for image classification is performed in Section 4. Section 5 discusses conclusion of the work.

2. Related Work

COVID-19 is a pandemic disease that spread globally since late December 2019, which has already caused huge mortality and morbidity among millions of people worldwide [13]. Treating the impact of COVID-19 virus in the lungs is a challenging task. For treating this case, CT images/X-ray images are taken [14,15,16]. Based on the CT images, medical experts can rank lung infection and the treatment provided. Hence, CT/X-ray images play a vital role in treatment selection. In a conventional medical system, a radiologist plays a vital role in diagnosing CT images or X-ray images, etc. [17,18]. Based on their report, treatment is given by the practitioners. However, infection interpretation by a radiologist is a highly inclined task. It may be influenced by individual bias and lack of medical experience. The Internet of Medical Things is designed to automate medical services. Wireless body area network (WBAN) is a field of IoMT in which sensors are attached to the human body. With the help of the sensors attached, we can measure and monitor human vitals from anywhere in the world. IoMT is an integration of multiple medical devices for providing accurate and cost-effective services, especially for remote people. It is generally used for continuous health monitoring in which sensors are deployed in the human body for measuring medical attributes, such as heart rate, oxygen level, ECG, blood pressure, breathing rate, heart rate, body temperature, etc., as shown in Figure 1. Based on the measured value, treatment support will be given to the needy people with the help of the internet and advanced AI-based techniques. IoMT helps society by providing timely and accurate medical services for saving their lives [19,20,21,22]. As the IoMT deals with a massive volume of medical records, researchers used cloud storage for storing the information [23]. IoMT is used in different medical applications, as shown in Table 1. It is discussed as follows. In [24], the authors designed a cost-effective and accurate IoMT module for the disease obstructive sleep apnea. It is a breathing disorder during sleep. SpO2 sensor is used for measuring the oxygen blood level and heart rate continuously. The measured data communicated to the cloud for extensive medical analysis. Based on the analysis report, warning will be given to remote patients via a mobile app.

Figure 1. Overview of Internet of Medical Things.

Table 1. Summary of the work in the view of IoMT.

Summary of the Work in the View of IoMT
Title	Techniques Used	Strength
IoMT cloud-based real time sleep apnea detection scheme using SpO2 estimation supported by heart rate variability	The sensing unit involved SpO2 for measuring oxygen blood level, and heart rate. The rules are used for analysis. Mobile app is used as actuation unit.	The designed system continuously monitors the oxygen level and heart rate of the person who is having breathing disorder during sleeping time with the help of sensors attached to the human body (wearables) and to alert the people during sleeping time.
IoMT for orthopedic in COVID-19 pandemic: roles, challenges, and applications	Cloud-based consultation.	The designed system addresses the problem of orthopedic patients during lockdown period and offers telemedicine services using cloud and internet services.
A novel method of clinical nursing under the medical internet of things technology	Sensing unit involves temperature and pulse rate sensors for measuring human vitals. Analysis is performed by medical experts.	The designed system continuously monitors the human body which is communicated to the medical community via internet and cloud technologies. If they find any abnormality, it is notified to the concerned via phone or mobile app. It reduces the treatment cost and burden of the health care takers.
Scope for improvements (from the study, it is observed that we can automate anything if we have a device for real time data collection, techniques for analysis, and a medium (user interface, mobile app, etc.) for controlling the event.
Proposed System	Sensing unit involves UI for data collection, pulse oximeter for oxygen, pulse rate measurement, RT-PCR kit for knowing COVID positivity Analysis involves deep convolution network architectures, such as ResNet-50, ResNet-100, ResNet-101, VGG 16, and VGG 19. After analysis, test summary report and assistance are communicated via designed UI.	Proposed system collects the information via the designed UI. Then, the collected information is analyzed using deep convolution network architectures. We used major voting classifier for getting precise results. It also avoids the classifier bias. After analysis, automatic test summary report is communicated via the same UI. It reduces the summary generation time. It helps to reduce the spreading rate. It also gives better assistance to the needy.

Due to the shortage of medical infrastructure during the COVID-19 pandemic situation, most of them are not able to take regular check-ups especially orthopedic patients. To address the issue, the authors proposed the IoMT system for orthopedic diseases [25]. It gives cloud-based services and telemedicine to the patients. In [26], the authors proposed an IoMT system for clinical nursing, in which the temperature and pulse rate sensors are attached to the ward’s body. The sensor layer connected to the clinical nurse via internet and cloud technology for monitoring the patients. It lessens the burden of the healthcare takers.

In general, IoMT integrates a sensing part for collecting the medical attributes from the patient, as well as an analysis part for making automatic decisions.

In the proposed system, we used deep convolution neural network architectures for CT/X-ray image analysis/classification. Many researchers experimented with different deep neural network architectures for detailed and accurate COVID-19 diagnosis, as shown in Table 2. It is discussed in detail as follows. Ahsan et al. [27] presented AI-based approach for detecting COVID-19. They experimented with six different deep convolution neural network architectures, such as VGG-16, Mobile Net V2, Inception ResNetV2, ResNet50, ResNet101, and VGG19 for getting better accuracy. They have used both CT and X-ray images to get enhanced results. From the experimentation, it is observed that VGG 16 outperforms X-ray images with an accuracy 95%, and MobileNetV2 outperforms CT images with an accuracy 98.5%. Danilo et al. [28] performed an exploratory study on transfer learning capabilities for pneumonia classification. They have used chest X-rays for training and testing the model. Two different types of datasets of chest X-ray images were used for experimentation. For a detailed comparison, 12 different pre-trained models were tested with different datasets. The dataset included the chest X-ray images of healthy, viral, pneumonia, and bacterial affected. A total of 6330 images were taken for experimentation. From the results, it was observed that DenseNet provided better results. Yong et al. [29] proposed a semi-supervised approach for detecting COVID-19 using chest CT scans. It is concentrated on reducing the impact of overfitting. MixMatch rules were used for data augmentation. They also concentrated on data enhancement methods for analyzing the area which was difficult to distinguish. Liang Sun [30] proposed an adaptive feature selection method for choosing contrasting features for the analysis of COVID-19. It majorly concentrated on high-level feature selection. It is the process of extracting the location-specific features, such as where it did spread and how much it has spread. After that, the affected regions were segmented. They considered various features that are used interlobular septal deepening for classification. A computer-aided diagnosing method determined only a bilateral lobe. In this approach, lung lobes, bilateral lungs, and pulmonary segments were considered for better segregation. It also calculated distribution. To reduce the redundancy of the features, the XGBoost and random forest methods were used. The dataset included 1495 COVID-19 and 1027 pneumonia patient records. It achieves 91.79% accuracy. Edelson Damasceno Carvalho [31] designed the system for diagnosis of COVID-19. CT images were taken as input. CNN and XGBoost were used for classification. This approach involved the (i) generation of CT photographs, (ii) eradication of a feature which uses CNN, (iii) class of images which make use of XGBoost, and (iv) affirmation of outcomes, which include metrics typically utilized in computer-aided diagnosis (CAD) systems. It used CNN for the utilization of features in CT images and XGBoost for grouping. A total of 708 CT images were collected. In that dataset, it had 312 COVID-19 and 396 non-COVID records. It focused mostly on binary classification. In general, we need a massive volume of CT images to train the deep convolution neural network architecture. The 708 CT images were not sufficient to train CNN. They used pre-trained models for feature extraction and the extreme gradient boosting (XGBoost) for classification. This technique produced 95.07% accuracy. Guotai Wang [32] proposed an automated system for extracting pneumonia lesions from CT images. The splitting of images is essential for accurate prediction. More specifically, deep learning helps in performing such a kind of splitting of the images. However, it requires a large volume of high-quality annotations. If the quantity of the dataset is minimal, then training CNN seems to be backbreaking. Hence, a novel noise robust framework was proposed, which learns from the noisy labels for segmentation tasks. It presents the discrepancy between foreground and background pixels and labels which are noisy in general. In order to address the above issue, an advanced COVID-19 pneumonia lesion segmentation network (COPLE-Net) is used to increase information gain. It combines the uses of max pooling and average pooling as down sampling. It minimizes information loss, compared to simple max pooling. For experimentation, it utilizes the CT images of 558 corona patients. Xinggang Wang [33] presented an enfeebled supervised system for coronavirus segregation and the localization of lesions from chest CTs. It works extremely well on the extraction of images part. For every patient, the lung vicinity became segmented by utilizing a pre-trained UNet; then, the 3-D lung vicinity became segmented and fed right into a 3-D deep neural technique to find out the chance of occurrence of coronavirus. The 499 CT scan images had been utilized for training. For testing, 131 CT volumes had been used. A total of 1.93 s was taken by the system to determine each patient’s CT volume, which utilized a dedicated graphics processing unit. The authors have adapted a 3-D deep convolution neural community to identify COVID-19 from CT volumes. It directed CT extent and 3-D lung as input. The 3-D lung masks were generated through a pre-trained U-Net. DeCoVNet consists of three phases, namely a network system to preserve rich local visual information, a 3D residual block to extract the features, and a progressive classifier that classifies whether the person is affected with COVID-19 or not. El-Sayed [34] proposed inventive feature selection and voting classifier algorithms especially for coronaviruses. Diagnosis plays a vital role in providing timely and suitable treatment. In this approach, numerous deep learning algorithms (PSO, CNN, and guided whale optimization algorithm) were experimented. A novel voting approach was implemented to overcome individual classifier incompatibilities. Sadman Sakib et al. [35] proposed a deep learning-based chest radiograph type system for accurately distinguishing COVID-19 cases. The novel dataset was prepared including normal, pneumonia, and COVID-19 cases. Posteroanterior (PA) chest view is chosen as a feature for classification, which improves the quantity of the dataset. Data augmentation for radiographic images algorithm is used to generate the augmented data. For the classification of COVID-19, Siva Ramakrishnan [36] employed excellent deep learning ensemble techniques. As a dataset, it uses chest X-rays. The dataset combines X-ray images of bacterial pneumonia, COVID, and viral infection. Numerous pruning models are combined through ensemble techniques. It results in better classification with an accuracy of 99.01%. For recognizing COVID-19 from CT images, Arnab Kumar Mishra et al. [37] employed deep convolution neural network architecture. A decision fusion-based strategy was applied in this case. To obtain the final prediction, it checks several deep learning models. The goal of this model is to diminish the bias of individual approaches via a major voting approach. They experimented with different models, i.e., VGG 16, Inception V3, ResNet50, DenseNet 121, and DenseNet 201. They utilized Specificity, Accuracy, Sensitivity, Precision, Recall, and the F1-Score as the performance metrices. Dilbag Singh [38] multi-objective differential evolution-based convolution neural networks were implemented to classify the patients affected with COVID-19. CT images were taken as a dataset. It focused only on binary classification. A multi-objective differential evolution-based convolution neural network was used for classification. In this approach, stride, kernel size, activation functions, kernel type, padding, hidden layer, etc., are considered as parameters to train the machine. Numerous deep convolution neural network architectures, such as CNN, ANN, and ANFIS, were experimented to obtain an accurate classification. Chirag Goel [39] proposed deep network architectures for finding the infections caused by COVID-19 using CT images. Random forest algorithm uses the features derived from an auto-encoder and grey level co-occurrence matrix for the efficient and fast detection of COVID-19. The dataset included 2482 CT images, among which 1252 images were those of the victims who tested positive for the COVID-19 infection, and 1230 images were from individuals who tested negative for the coronavirus. Auto-encoders are used to recreate the given image. Encoders are used for dimensionality reduction. It also tries to reduce over fitting. GLCM is utilized for texture evaluation, since it can estimate the quality of the image, with respect to second-order statistics. The pixel relationship is studied by using gray level co-occurrence matrix. Rachna Jain [40] presented a model for COVID-19 exposure and analysis on chest X-ray images. Post anterior (PA) vision of chest X-ray images were considered to classify the diseases. Inception V3, Xception, and ResNet models were furnished for comparative analysis. The dataset included 6432 chest X-ray samples, in which were 5467 applied for preparation, and 965 images were used for validation. In this study, the Xception model earned a better accuracy (97.97%) than other models. From the literature survey, it was observed that many works were performed to identify a person who is affected by COVID-19. There is no specific AI-based IoMT system that is designed to reach, guide, and measure the severity of level of patients who are affected with COVID-19 remotely. It inspires us to design and develop a system for providing better assistance. The proposed AI system concentrated on binary classification and multi-level classification.

Table 2. Summary of the work in the view of image analysis.

Summary of the Work in the View of Image Analysis
Title	Techniques Used	Strength
Detection of COVID-19 Patients from CT Scan and Chest X-ray Data Using Modified MobileNetV2 and LIME	Experimented VGG-16, MobileNet V2, Inception ResNet V2, ResNet 50, ResNet101, VGG 19.	The proposed approach used CT and X-ray images for detecting COVID-19. VGG16 obtains 95% accuracy for X-ray images, MobileNetV2 obtains 98.5 for CT images.
Study on Transfer Learning Capabilities for Pneumonia Classification in Chest X-ray Images	12 different pre trained models are tested. Two different chest X-ray image datasets are used. 6330 images are taken for experimentation.	The study was performed for pneumonia classification. DenseNet yields better result.
A Semi-Supervised Learning Approach for COVID-19 Detection from Chest CT Scans	MixMatch Rules.	It is focused on reducing the impact of overfitting and enhancement.
A Logic-Based Framework Leveraging Neural Networks for Studying the Evolution of Neurological Disorders	Answer Set Programming.	It is designed for learning the evolution of neurological diseases. It makes the use of ANN and ASP for better analysis.
Quantum Convolutional Neural Networks. Experimental Quantum Advantage with Quantum Coupon Collector, Breaking the Rate-Loss Bound of Quantum Key Distribution with Asynchronous Two-Photon Interference	CNN and Quantum Computing.	Proposed System combined the features of CNN and quantum computing for better image classification.
Diagnosis of COVID-19 in CT Image Using CNN and XGBoost	CNN and XGBoost. 708 CT images considered for experimentation.	The system is designed for diagnosis of COVID-19. CNN is used for feature selection and XGBoost is used for grouping. It achieves 95.07% accuracy.
A Noise-Robust Framework for Automatic Segmentation of COVID-19 Pneumonia Lesions from CT Images	558 CT images are taken for experimentation. Combination of Max pooling and average pooling is used for down sampling.	Advanced COVID-19 pneumonia lesion segmentation network (COPLE-Net) is proposed for extracting pneumonia lesions.
Classification of COVID-19 Patients from Chest CT Images Using Multi-Objective Differential Evolution-Based Convolutional Neural Networks	CT images are taken as dataset. Focused on binary classification. CNN, ANN, and ANFIS are experimented for classification.	It is designed for COVID-19 classification.
Scope for improvements—from the study, it was observed that numerous AI techniques and convolution neural network architectures are available for better image classification. Some of the researchers took X-ray images, some of them took CT images, and some of them took both for better classification. However, all are support system for medical practitioners.
Proposed System	Major voting classifier is used for reducing the classifier bias. Outcomes of ResNet 50, ResNet100, ResNet101, VGG 16, and VGG 19 are given as input to the major voting classifier. It selects the majority voted one. Image data generator used for synthetic data generation.	Data analysis is one of the modules used in making the system automated. Pre-trained models are used for image analyses. Totally, 7455 CT images and 8900 X-ray images are considered for experimentation.

In the first step, IoMT treatment support system classifies patients as COVID or non-COVID. In the second step, multilevel classification is used to measure the severity. It is to be produced by the system with the help of AI techniques. Finally, a textual summary report is sent to the remote/needy people via the designed GUI

3. Proposed System

We have designed an AI-based IoMT treatment support system for COVID-19, as shown in Figure 2. It is not only designed to fix the people who are affected with COVID-19 or not. It can find the SARS family of diseases. It also ranks the severity level of the infection.

Figure 2. Architecture of Proposed IoMT Treatment Support System for COVID-19.

The main objectives of the proposed system are listed as follows:

(1)

Leverage the culmination of cutting-edge technologies to address the vital problems in the health care sector for the common public.

(2)

Design of intelligent data acquisition system.

(3)

Acquire heterogeneous raw data (CT images, RT-PCR results, pulse oxygen rate, X-ray imaging) from various sources, either directly and indirectly, and integrate them contextually to achieve a unified formatted data.

(4)

Design and develop a cloud platform for unified formatted data.

(5)

Design a system for multi-diseases classification (COVID-19/pneumonia/viral infections).

(6)

Develop a model to measure severity level of the lungs to provide suitable treatment to save their lives.

(7)

Present test summary report textually. It can be accessed by a user-friendly GUI.

Figure 3 shows the workflow of the proposed system in a detailed manner. By considering the above objectives, we have designed an AI-based intelligent support system for pandemic diseases. It includes the following phases, as listed below:

Figure 3. Workflow of the proposed IoMT Treatment Support System for COVID–19.

(a)

Intelligent data acquisition system;

(b)

Cloud platform for storing the data;

(c)

Data pre-processing;

(d)

Synthetic data generation;

(e)

Proposed AI-based system for disease classification.

3.1. Intelligent Data Acquisition System

The proposed data acquisition system provides a platform for obtaining medical details directly from the needy/remote people who do not have enough medical infrastructure. We have designed a graphical user interface (GUI), as in Figure 4. It is a common platform in which they can upload their medical details and they can obtain the guidelines and medical assistance.

Figure 4. Graphical User Interface design for data acquisition and report generation.

The proposed system is mainly designed for COVID-19. We have incorporated some basic intelligence into the proposed data acquisition system. At the outset, the user needs to register in the portal with basic details, such as the name of the patient, location, contact number, Aadhaar card details, etc. Once they have registered in the portal, they will be given login credentials. They can upload their details, such as RT-PCR test results and pulse oxygen rate. Then, the intelligent data acquisition system analyzes the data uploaded in the portal. In general, it deals with three cases, as follows.

(1)

Case 1: RT-PCR test results are negative and pulse oxygen rate is not less than the threshold value (90%), then it generated the test report as “Non-COVID-Viral Infection”.

(2)

Case 2: If RT-PCR is negative and pulse oxygen rate is less than the threshold value (90%), it gives a suggestion to upload the CT images/X-ray images based on the available medical infrastructure.

(3)

Case 3: If RT-PCR is positive, it gives a suggestion to take CT scan or X-ray for analyzing intensity level of the diseases. The designed user interface is connected with Google Cloud for detailed medical analysis.

3.2. Cloud Platform

The medical details, which were collected from the intelligent data acquisition system, are stored in cloud storage for detailed medical analysis. Nowadays, we are working with different cloud storage, such as Amazon S3 storage, Google cloud, etc., and Amazon S3 storage provides free storage up to 5 GB. If we need more storage, then we need to pay. In Google cloud, we can store up to 15 GB free of cost. As the Google cloud gives more storage, compared to AWS, free of cost, we have chosen the Google cloud for our implementation. Cloud storage deals with heterogeneous data, such as RT-PCR test results, pulse oxygen rate, X-ray images, and CT images, as shown in Figure 5. RT-PCR and pulse oxygen rate are text data. X-ray images and CT images are image data. As it includes different type of data, it needs data segregation. We are not able to apply the same technique for text data and image data. Hence, data isolation plays a vital role in this research work.

Figure 5. Google Cloud storage for the proposed IoMT system.

3.3. Data Preprocessing

The proposed IoMT system uses AI techniques for detailed medical analysis. Deep convolution neural network architectures are used for training the proposed AI system. It uses a heterogeneous clinical dataset. For training the proposed AI system, clinical data, such as pulse oxygen rate, RT-PCR, X-ray images, and CT images, are collected from historical data sources (ICMR, DBT, Medical Colleges, Scan Centre etc.) and indirect data sources (Kaggle, UCI, GitHub, etc.) as shown in Figure 6 and Figure 7. The raw data from the above data sources is fed to data pre-processing system, which performs the mapping of data in different formats to the prescribed format for analytics. The formatted data is again processed to remove noise and outliers and handle missing data.

Figure 6. Sample CT image Dataset.

Figure 7. Sample X-ray Image Dataset.

3.4. Synthetic Data Generation

We collected the clinical dataset from direct and indirect sources. It consists of 1700 patient records. For training deep convolution neural network architectures, the dataset we collected was not sufficient. Hence, we used Keras Image Data Generator for synthetic data generation. With the help of an image data generator, we generated 7455 synthetic records. Under an image data generator, we used simple techniques such as rotation, flipping, zooming, horizontal shift, and vertical shift, to generate new images. For testing, anonymous real time data were used, which were collected from hospitals and scan centres.

3.5. Proposed AI-Based System for Disease Classification

We have designed AI-based disease diagnosis system for generating a detailed test summary report to the needy/remote peoples in a textual format. The main objective of this module is to generate the medical diagnosis report automatically. It obtains the input from the Google cloud, where the users upload their medical details. We carried out two kinds of data analysis, namely preliminary examination and detailed medical report generation. The preliminary examination is performed by the intelligent GUI with the help of client-side programming. At the beginning, it asks the user to enter RT-PCR test results and pulse oxygen rate. If the system found that the person is affected with a normal viral infection, it just generates the message “Normal—Viral infection only”. If the client-side programming found any abnormality in the preliminary test, it asks the user to enter X-ray/CT images based on the available infrastructure in their location. CT scan centres may not be available in the rural area. Hence, the proposed AI system deals with X-ray and CT images. The proposed AI system focuses on classification techniques to examine patterns and rank the severity of lung infection based on filtered, pre-processed infectious disease data (pure GGO, halo sign, reversed halo sign, bronchial wall thickening septal thickening and/or reticulation, crazy-paving pattern, consolidation, and reticular and/or interlobular septal thickening). The extraction of the clinical attributes of CT images needed for ranking lung infection is performed based on radiologist experience and historical data. A classification system uses more than one deep convolution neural network architecture to obtain better classification. In this paper, we experimented with numerous deep convolution network architectures, such as ResNet 50, ResNet 100, ResNet 101, VGG 16, and VGG 19. Furthermore, a major voting classifier is used for addressing individual model bias. It takes the classification result from majority classifiers, as shown in Figure 8. Finally, the majority will be taken as the classification result. We collected two kinds of datasets, such as X-ray and CT images, for training the machine. They were collected through various sources, such as Kaggle, github, UCI, DBT, hospitals, scan centres, etc. The size of the dataset 1700 and 2200 in CT and X-ray images respectively. However, the collected information is not sufficient to train the machine. Hence, we generated synthetic data using the Keras image data generator. As a result, we obtained 7455 and 8900 in CT and X-ray images. We also obtained increased accuracy, as shown in Table 3 and Table 4. It also includes the affirmation support system. It is a feedback system to improve the accuracy of historical data. This was achieved through medical expert’s suggestions provided to the test results generated by the machine, as shown in Figure 9. The secondary data collected from medical experts can be added to the existing historical data for future use.

Figure 8. Working Principle of Major Voting Classifier.

Table 3. Performance analysis of deep convolution neural network architectures on CT images.

Dataset Size	Type of the Image	Algorithm Used	Classification Report
1700 (images collected from public resource) + 7455 (synthetic data)	CT	ResNet 50	Precision	78%
			Recall	79%
			F1-Score	82%
			Accuracy	78%
1700 (images collected from public resource) + 7455 (synthetic data)	CT	ResNet 100	Precision	92%
			Recall	89%
			F1-Score	93%
			Accuracy	93%
1700 (images collected from public resource) + 7455 (synthetic data)	CT	ResNet 101	Precision	97%
			Recall	92%
			F1-Score	96%
			Accuracy	97%
1700 (images collected from public resource) + 7455 (synthetic data)	CT	VGG 16	Precision	89%
			Recall	86%
			F1-Score	89%
			Accuracy	81%
1700 (images collected from public resource) + 7455 (synthetic data) public resource) + 7455 (synthetic data)	CT	VGG 19	Precision	97%
			Recall	90%
			F1-Score	95%
			Accuracy	97%

Table 4. Performance analysis of deep convolution neural network architectures on X-ray images.

Dataset Size	Type of the Image	Algorithm Used	Classification Report
2200 (images collected from public resource) + 8900 (synthetic data)	X-ray	ResNet 50	Precision	82%
			Recall	81%
			F1-Score	83%
			Accuracy	80%
2200 (images collected from public resource) + 8900 (synthetic data)	X-ray	ResNet 100	Precision	95%
			Recall	89%
			F1-Score	94%
			Accuracy	95%
2200 (images collected from public resource) + 8900 (synthetic data)	X-ray	ResNet 101	Precision	98%
			Recall	92%
			F1-Score	96%
			Accuracy	98%
2200 (images collected from public resource) + 8900 (synthetic data)	X-ray	VGG 19	Precision	96%
			Recall	95%
			F1-Score	96%
			Accuracy	95%
2200 (images collected from public resource) + 8900 (synthetic data)	X-ray	VGG 16	Precision	85%
			Recall	81%
			F1-Score	82%
			Accuracy	81%

Figure 9. Working principle of Affirmation Support System.

Once the medical diagnosis performed by the proposed AI system, the test summary report will be generated in textual form. It is communicated to the end user via the designed GUI. It results in a reduced report generation time with minimal medical infrastructure.

4. Experimentation for SARs Virus Classification

Two kinds of datasets, such as CT images and X-ray images, were chosen for the proposed AI system experimentation. The dataset was divided into two parts: training data and testing data. X-ray and CT images were gathered from open sources, such as Kaggle and GitHub. CT scans of pneumonia, COVID-19, and virally infected people were included in the data collection. For training, we used public resources. For testing purposes, real images which we have collected from hospitals were used. Precision, recall, F1-score, and accuracy (see Equations from (1) to (4)) were taken as performance metrics for the proposed AI system evaluation. The performance metrics were computed by using the formula shown below.

$$\mathrm{Precision}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}$$

(1)

$$\mathrm{Recall}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$$

(2)

$$\mathrm{F}1=\frac{\mathrm{TP}}{\mathrm{TP}+\left(0.5\right) \left(\mathrm{FP}+\mathrm{FN}\right)}$$

(3)

$$\mathrm{Accuracy}=\frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}}$$

(4)

where TP—true positive; FP—false positive; FN—false negative; TN—true negative.

For obtaining better accuracy, we experimented with five deep convolution network architectures, such as ResNet 50, ResNet 100, ResNet 101, VGG-19, and VGG-16. With the help of pre-trained models, we can obtain better accuracy with minimal datasets. Pre-trained models are trained models. Hence, it does not require more information. Furthermore, we have implemented a major voting classifier. It takes ‘n’ number of classifier results as input. From that, it will take majority voting, as shown in Figure 8. It results in better accuracy. It also lessens individual classifier bias. In addition to the performance metrics, we have generated heat map for CT and X-ray images. With the help of heat map, we could understand the pulmonary amalgamation in the lungs. It helps to measure intensity level of the severity. Finally, textual reports generated are communicated to the needy/remote people via GUI.

4.1. Experimental Results on CT Images

We have designed an AI-based medical diagnosis system. For obtaining improved accuracy, we have trained the proposed AI system with heterogeneous dataset. The initial screening was performed by designed intelligent GUI. It collects RT-PCR test results and pulse oxygen rate from the public. Based on the initial screening, the GUI collects further details for detailed medical analysis. Initial screening is achieved by front-end programming. In this section, we discussed the AI-based medical diagnosis system on CT images. Dataset was collected from public sources, hospitals, and scan centres. It includes CT images of COVID, pneumonia, normal, and viral infections. Totally, 1700 images were collected. As we have used deep convolution neural network architectures, such as ResNet 50, ResNet 100, ResNet101, VGG 16, and VGG 19, for experimentation, the dataset which we collected was not sufficient to train the machine. Hence, we generated synthetic data with the help of Keras image data generator. As a result of synthetic data generator, we obtained 7455 synthetic images. For testing the model, 100 real time images were used. Major voting classifier takes the result of all the classifiers as input; majority was taken as final classification result. Detailed performance analysis of deep convolution network architectures is given in Table 3. In addition to that, a heat map was generated for analyzing pulmonary amalgamation, as shown in Figure 10.

Figure 10. Heap map and super imposed image for the study of pulmonary amalgamation.

4.2. Experimental Results on X-ray Images

The proposed IoMT treatment support system provides an option to upload CT/X-ray images in the designed GUI. The needy/remote location people can upload CT/X-ray images based on the available facility in their area. In the designed system, the proposed IoMT system will generate the test summary report, instead of radiologist. It results in a reduced report generation time with minimal medical infrastructure. In this section, we discussed the AI-based medical diagnosis system on X-ray images. Dataset was collected from public sources, hospitals, and scan centres. It includes X-ray images of COVID, pneumonia, and normal and viral infections. Totally, 2200 images were collected. As we have used deep convolution neural network architectures, such as ResNet 50, ResNet 100, ResNet101, VGG 16, and VGG 19, for experimentation, the dataset which we have collected was not sufficient to train the machine. Hence, we generated synthetic data with the help of Keras image data generator. As a result of synthetic data generator, we obtained 8900 synthetic images. For testing the model, 226 real time images were used. Major voting classifier takes the result of all the classifiers as input; majority was taken as final classification result. Detailed performance analysis of deep convolution network architectures is given in Table 4. In addition to that, a heat map was generated for analyzing pulmonary amalgamation, as shown in Figure 11.

Figure 11. Heap map and super imposed image for the study of pulmonary amalgamation.

5. Conclusions

In this paper, we have designed an AI-based IoMT treatment support system for COVID-19, in which we have designed a GUI for interfacing the needy/remote location people with the health care system. It provides an option to the public to upload their medical records, such as pulse oxygen rate and RT-PCR results. Based on the initial screening performed by the intelligence GUI, further details will be collected from the people. After that, the collected information is stored in the Google cloud for detailed medical analysis. For detailed medical analysis, we have experimented with numerous deep convolution neural network architectures, such as ResNet50, ResNet 100 ResNet101, VGG 16, and VGG 19. To address individual classifier bias, a major voting classifier is used. A heat map is generated for obtaining pulmonary amalgamation details. Based on that, we ranked the severity level. Finally, a textual report is generated. It is communicated to the concerned via the same GUI that was designed for data acquisition. In future, the designed system can be extended to predict similar kinds of diseases (SARs family), as well. This is achieved by incorporating the epidemiological pattern of the diseases by artificial intelligence methods.