Next Article in Journal
Supercritical Carbon Dioxide Extraction of Phenolic Compounds from Potato (Solanum tuberosum) Peels
Next Article in Special Issue
A Roadmap towards Breast Cancer Therapies Supported by Explainable Artificial Intelligence
Previous Article in Journal
Equivalent Electrical Circuit Modeling of CNT-Based Transparent Electrodes
 
 
Review

COVID-19 Detection Empowered with Machine Learning and Deep Learning Techniques: A Systematic Review

School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu 611756, China
*
Author to whom correspondence should be addressed.
Academic Editor: Kao-Shing Hwang
Appl. Sci. 2021, 11(8), 3414; https://doi.org/10.3390/app11083414
Received: 19 March 2021 / Revised: 4 April 2021 / Accepted: 8 April 2021 / Published: 10 April 2021
(This article belongs to the Special Issue Machine Learning in Medical Applications (Extended Version))

Abstract

COVID-19 has infected 223 countries and caused 2.8 million deaths worldwide (at the time of writing this article), and the death rate is increasing continuously. Early diagnosis of COVID patients is a critical challenge for medical practitioners, governments, organizations, and countries to overcome the rapid spread of the deadly virus in any geographical area. In this situation, the previous epidemic evidence on Machine Learning (ML) and Deep Learning (DL) techniques encouraged the researchers to play a significant role in detecting COVID-19. Similarly, the rising scope of ML/DL methodologies in the medical domain also advocates its significant role in COVID-19 detection. This systematic review presents ML and DL techniques practiced in this era to predict, diagnose, classify, and detect the coronavirus. In this study, the data was retrieved from three prevalent full-text archives, i.e., Science Direct, Web of Science, and PubMed, using the search code strategy on 16 March 2021. Using professional assessment, among 961 articles retrieved by an initial query, only 40 articles focusing on ML/DL-based COVID-19 detection schemes were selected. Findings have been presented as a country-wise distribution of publications, article frequency, various data collection, analyzed datasets, sample sizes, and applied ML/DL techniques. Precisely, this study reveals that ML/DL technique accuracy lay between 80% to 100% when detecting COVID-19. The RT-PCR-based model with Support Vector Machine (SVM) exhibited the lowest accuracy (80%), whereas the X-ray-based model achieved the highest accuracy (99.7%) using a deep convolutional neural network. However, current studies have shown that an anal swab test is super accurate to detect the virus. Moreover, this review addresses the limitations of COVID-19 detection along with the detailed discussion of the prevailing challenges and future research directions, which eventually highlight outstanding issues.
Keywords: COVID-19; detection; machine learning; deep learning; diagnosis; anal swab; systematic review; computer-aided diagnosis COVID-19; detection; machine learning; deep learning; diagnosis; anal swab; systematic review; computer-aided diagnosis

1. Introduction

COVID-19 is a contagious disease reverted by the SARS virus and announced as a global pandemic by the World Health Organization (WHO) in March 2020. The COVID-19 massive outbreak has infected 223 countries, with more than 129 million infected patients and 2.8 million deaths globally [1,2]. The infected cases and death rate is increasing rapidly, as shown in Figure 1. The vital factor is to diagnose the infected cases at an early stage to control this natural pandemic. However, advanced intelligent prediction systems and accurate modeling of techniques have made a precious contribution in managing and planning health resources against the virus. The initial stage diagnosis of the virus is also useful for appropriate patient isolation, fast canulization of chronic patients in specific hospitals, and observing virus spread. However, the diagnosis of COVID-19 is quite challenging given the economic issues raised among developed and under-developed countries due to the high cost of diagnostic tests [3].
One of the most critical reasons behind the rapid spread of COVID-19 is the shortage of clinical diagnosis methods [4], such as reverse transcription-polymerase chain reaction (RT-PCR) [5], serologic tests, and viral throat swab testing (oropharyngeal) [6], etc. Currently, China has started anal swab testing to enhance the detection rate and overcome the transmission of the deadly virus. Researchers have found that anal swab testing accuracy is higher than other diagnostic swab methods [7], but this method is still not convenient for massive sampling. These methods are quite expensive and not readily available in many countries and have been used only to detect the existence of the virus [8]. Therefore, researchers proposed some other techniques such as Computed Tomography (C.T.) scans and X-ray to assess the screening and intensity of COVID-19 infection (in hospital admitted cases). These tests are more suitable for providing fast and accurate results [9]. It is still challenging to use chest C.T. scans and chest radiographs (X-rays) because they may not correctly distinguish between COVID-19 infection and other lung diseases [10,11].
However, the above-mentioned conventional methods require medical experts for the judgment of diagnostic tests. To automate this type of diagnosis, a computer-aided diagnosis (CAD) system provides vital solutions to researchers detecting coronavirus by using classification approaches [12]. CAD has now become the center of research. In the case of COVID-19, CAD employs ML/DL techniques to evaluate all kinds of patient data, either images or clinical data, and predict the patient’s condition. This evaluation improves diagnosis and support for medical experts in decision making and improves diagnosis [13]. Henceforward, the CAD system is considered a robust tool for radiologists to improve diagnostic methods such as chest radiographs, C.T. Images, etc. Thus, ML and DL techniques have been introduced to develop expert systems due to their significant role in healthcare to deploy clinical decision support systems [14]. ML and DL are not only supportive for the classification of medical diseases such as breast cancer [15], tuberculosis [16], pneumonia [17], and muscle diseases [18] but also useful for detecting, diagnosing, and classifying COVID-19 [19,20]. ML and DL are emerging technologies in the field of medicine, and due to their capability to generate highly accurate predictive results, these methods can play an essential role in renovating the healthcare system [21]. However, due to the enormous variety of these techniques, it is difficult to prefer an adequate ML/DL strategy that provides effective results [22,23] to diagnose COVID-19.
Moreover, most of these techniques are characterized by less accuracy and computational efficiency [24]. Due to these issues, it is a big challenge to evaluate the optimal technique having the most accurate results [25]. In other words, it is very crucial to present a prediction system that can precisely detect and diagnose the virus. It will also be a challenge for ML techniques to detect public health risks in advance for improving the prediction, prevention, and detection of future epidemiologic risks. In literature, several ML/DL techniques have been proposed as intelligent systems to provide highly accurate results in a short time for the prediction of COVID-19 [26,27,28]. In this review, our primary focus is to accomplish the following two tasks: (i) the analysis of different ML/DL techniques that have been recently published related to the detection of coronavirus and (ii) consideration of upcoming research challenges. Furthermore, novel efforts have been made to achieve the following objectives:
Determining the significance of available datasets from literature used for the prediction of COVID-19.
To analyze the ML and DL techniques that were applied to detect the COVID-19.
Identification of challenges and future research directions related to the implications of ML/DL techniques for COVID detection.
The study can be categorized as follows. Section 2 describes the procedures used to evaluate the systematic literature on the topic. Results and relevant discussion are explained in Section 3. Then introduce the anal swab testing in Section 4. Section 5 discusses the upcoming challenges and research directions of ML/DL related to COVID-19. In the end, the conclusions and future directions of the study are presented.

2. Review Methodology

The methodology for the current study was adopted from the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (P.R.I.S.M.A.) statement [29].

2.1. Selection of Digital Archives

For this systematic review, the data was collected from three full-text digital archives, including Science Direct (S.D.), Web of Science (WoS), and PubMed. Google Scholar and medRxiv were employed for preprint studies. S.D. provides access to exceptionally reliable journals in the field of engineering, medical, and computer technology. WoS is a comprehensive source of social science, liberal arts, and multi-disciplinary applications. PubMed accesses MEDLINE, which is the most suitable database for medical and bio-engineering studies [30]. Google scholar is a reliable source in every research field, such as biomedical engineering, computer science, and health technology. These databases were selected due to their adequate scope and originality of the studies. All hypothetical features of coronavirus are included in these databases as a diagnosis of the virus needs more attention than other infections.

2.2. Search Code Strategy

The original research articles available from the beginning of time were collected from the databases (S.D., WoS, PubMed) on 16 March 2021. Many queries were used to enhance the search-related quality to various ML and DL techniques for detecting COVID-19. Important searching keywords used for the data retrieval were as follows: (a) SD database search code is TOPIC: ((“Coronavirus” OR “COVID-19”) AND (“Detection” OR “diagnosis” OR “classification”) AND (“Machine learning” OR “Deep learning”)); (b) PubMed database search code is TOPIC: ((“Novel Coronavirus” OR “COVID-19” OR “coronavirus” OR “2019-nCoV” OR “new coronavirus” OR “SARS-CoV-2”) AND (“Detection” OR “diagnosis” OR “prediction” OR “prognosis”) AND (“Machine learning” OR “Deep learning”));(c) WoS database search code is TOPIC:(“Novel Coronavirus” OR “COVID-19” OR ”coronavirus” OR “2019-nCoV” OR “new coronavirus” OR “SARS-CoV-2”) AND (“Detection” OR “diagnosis” OR “prediction” OR “prognosis” OR “Analysis” OR “Classification”) AND (“Machine learning” OR “Deep learning”). The published papers with these terms found in titles, abstracts, and keywords were retrieved. Moreover, only original research papers related to the detection of COVID-19 through ML/DL techniques were extracted, and the review articles and books were eliminated from the collection.

Machine and Deep Learning Techniques

Remarkable progress has been made in using different ML algorithms applied to medical datasets for detecting different diseases. ML is a mathematical and statistical technique that gives computers the ability to learn from data and elaborate a convoluted framework. DL consists of a group of algorithms applied to develop an expert system that can identify problems and yield predictions. These ML and DL techniques produce intelligence into a computer that can elicit the patterns relative to specific data and then proceeds for automatic reasoning [31]. Many ML/DL algorithms have been applied to predict, detect, and diagnose COVID-19 [32].
In this article, we have presented an analysis of these ML/DL techniques used to detect COVID-19. Below is a brief description of these classifiers.
SVM is a supervised ML classifier, a specified set of training examples given to SVM that learn the hyperplane, isolate the instances from each class, and magnify the edges among data instances and hyperplane. K-NN is also a supervised algorithm. It classifies the unknown sample by measuring its distance from other training samples and calculates K-smallest distances. The unknown sample output class label is given by most represented in these K classes [33].
A Decision Tree classifier consists of one root node, numerous internal nodes, and several terminal nodes. D.T. represents a tree structure in which every leaf node is related to a group of features, and a branch displays a value. The primary purpose of the D.T. algorithm is to build a tree for the whole data and provide refine results on each leaf. A Random Forest classifier ensembles various decision trees for prediction, which are developed by grasping numerous classification trees simultaneously R.F. performance is more superior to a single tree [34]. This classifier can be employed efficiently, and it accomplishes predictions for many higher dimensional datasets with greater accuracy.
CNN and D.N.N. is a deep learning technique consisting of several layers such as input, output, and hidden layers. These layers modify the data with convolution filters. Currently, researchers are employing these deep learning techniques to detect COVID-19 [31].
It is investigated in the literature that these models achieved greater accuracy in detecting viral pneumonia, bacterial pneumonia, and COVID-19 [26], respectively. ML/DL represents a tremendous innovation in automatic diagnostic classification systems; therefore, these classifiers present a suitable choice to be employed in diagnosing COVID-19.

2.3. Eligibility Criteria and Article Screening

The literature selection method was started by applying search code queries on selected three digital databases, as shown in Figure 2. In the initial query, 961 articles were found; however, by removing duplicate articles from these, 886 articles have remained. Screening has been conducted based on title, abstract, and keywords, then 795 articles were excluded. We had sorted out 91 articles for complete text studies, and after studying each article, only 40 high-quality articles were selected for this systematic review.

2.4. Data Segregation and Categorization

The data were extracted and categorized to investigate the adaptability of the COVID in terms of exposure, diagnosis, and classification through ML/DL techniques. The distribution of the retrieved data with the countries involved in the study of COVID-19 detection through ML/DL techniques is presented in Figure 3.
It can be noticed that China was the most prolific country with 15 publications related to the MLT-based detection of COVID-19. Some other influential contributing countries (and their publications) were Turkey (5), India (3), USA (2), Iran (2), and Egypt (2). The rising scope of the ML/DL techniques can play a significant role in controlling the spread of the novel coronavirus globally. Furthermore, different ML and DL algorithms such as classification, regression, and prognosis were presented to analyze these technique’s efficiency. Figure 4. highlights the number of articles related to these algorithms utilized for the detection of COVID-19. It can be observed that most of the studies employed Convolutional Neural Network (CNN) algorithm and Deep Neural Network (D.N.N.) algorithm. The utilization frequencies of these techniques were 18 and 13, respectively. Only five investigations used Support Vector Machine (SVM) and Random Forest (R.F.) algorithms to detect the COVID, whereas the number of investigations related to Long Short-Term Memory (L.S.T.M.), Decision Tree (D.T.), and K-Nearest Neighbors (K-NN) algorithms was 4, 2, and 1, accordingly.
The research articles retrieved for the current systematic review and the available datasets that used ML/DL techniques (such as SVM, K-NN, R.F., CNN, and D.N.N.) for the diagnosis of COVID-19 elaborate in Section 3.

3. Results and Discussion

3.1. Analysis of Extracted Data

Table 1 summarizes the datasets used in the previous studies presented along with test data (i.e., Chest X-rays and C.T. Images, RT-PCR, and Clinical data), types of infected patients, and references. In some articles, public databases such as Kaggle and GitHub repository were utilized, whereas, in other studies, the private data collected from hospitals and universities were investigated. The diagnosis of COVID-19 is crucial as careful investigations and data processing are required using specific medical expertise. It was revealed that in most of the cases, the data from C.T. and X-ray images were studied, whereas a few investigations were conducted using RT-PCR and clinical blood test data to diagnose COVID-19 by using ML/DL techniques.
Generally, there are two methods of prediction of COVID-19 images, binary classification and multiclass classification. Binary classification is used to detect COVID-19 positive and negative cases. However, this classification method is inaccurate due to the misclassification of COVID-19 images with other types of lung diseases (viral pneumonia, bacterial pneumonia). To solve this issue, researchers have differentiated the images of COVID-19 viral pneumonia, bacterial pneumonia, fungal pneumonia, SARS, MERS, influenza, tuberculosis, and images of healthy people by classifying them using the method of multiclass classification. The accuracy of multiple classifiers is better than binary classifiers in detecting COVID-19 images. The total number of C.T. scans and X-ray images used to categorize the COVID Positive, Normal, Pneumonia, and other Lung patients were reported in the literature and are mentioned in Table 1.

3.2. Investigation on Classification Performance

Table 2 summarizes the state-of-art COVID-19 prediction algorithms to test the data (Radiological (chest X-rays and C.T. images), RT-PCR, and Clinical data) along with the highest prediction results of the selected previous studies. Only the best-obtained results of different ML/DL techniques on C.T., X-ray images, RT-PCR, and clinical blood test data were mentioned.
In RT-PCR, samples collected from a person’s body, such as nose or throat, are investigated to detect the presence of the virus. The reliability of RT-PCR is not suitable owing to a high false rate, due to which it cannot give accurate results [73]. The availability of RT-PCR is relatively short around the world. Moreover, it includes a lengthy procedure for detection and is quite expensive. RT-PCR based model revealed with the least accuracy of 80% in recent research [59] and the maximum A.U.C. of 0.84 for another analysis [58]. The clinical blood test-based model was found with the lowest accuracy of 84.24% in the previous study [72] and the highest A.U.C. of 0.97 for another study [66]. Comparatively, the clinical blood test-based model obtained the highest A.U.C. of 0.97 [71] than RT-PCR-based models.
In the C.T. scan method, a final chest image is captured by combining the images taken from various angles, while a C.T. scan requires a short time, but it is quite expensive. It can be observed that the C.T. image-based model obtained a minimum A.U.C. of 0.86 in the previous study [60] and maximum accuracy of 99.96% as per the investigation [50].
X-rays are used to create images of the chest. An X-ray is economically affordable in most areas. Therefore, most of the medical experts utilized X-ray images instead of C.T. images. An X-ray image-based model was observed with the lowest accuracy of 86.7% in a previous study [40] and the highest accuracy of 99.7% for another study [55].
Currently, ML/DL has played a significant role in boosting many CAD systems’ diagnostic efficiency for various medical applications such as diagnosing and detecting different pulmonary diseases. Recent studies have shown that DL techniques have proven to be very efficient for CAD systems in radiography [48]. Hence, C.T. scan and X-ray both are widely used imaging methodologies for detecting various diseases and COVID-19. While X-rays need less data memory, processing time requires a low radiation dose than a C.T. scan [74]. Thus, CAD systems present vital solutions to boost and support radiologist workflow in predicting COVID-19 using low-dose X-ray images and overcoming limitations.
The literature has investigated that some researchers employed C.T. and X-ray image-based combined models, which were observed with the lowest accuracy of 96.61% in a previous study [62] and the highest accuracy of 98% for another study [35]. It can be stated that radiological X-ray image-based models worked better than the C.T. image-based models by examining Table 2 because the highest accuracy of X-ray-based models is 99.7% by using D.C.N.N. [55] to classify COVID-19. Thus, the C.T. and X-ray images are optimal for detecting coronavirus, but medical experts are required for the RT-PCR test, C.T. scan, and X-ray techniques.
It was observed that CNN and D.N.N. were the most considerable classification techniques for detecting COVID-19, followed by SVM, Random Forest followed by K-NN, and L.S.T.M. Moreover, compared to other ML/DL techniques, CNN was the most widely utilized classifier for the diagnosis of COVID-19. D.C.N.N. was the most accurate for the detection of COVID-19, but its usage was relatively short. Hence it needs to be explored further in future research for the detection and diagnosis of COVID-19. The study revealed that ML/DL-based approaches can significantly promote intelligent diagnosis systems, which are promising for healthcare professionals to make fast and reliable detection of the virus. It will also eliminate the manual flaws during the diagnosis by physicians and radiologists. Moreover, it will be a step towards time-efficient and accurate diagnoses to facilitate both hospitals and the patients.

4. State-of-the-Art COVID-19 Detection Using Anal Swab-Based Diagnosis

The outbreak of COVID-19 has adversely impacted the whole world with a massive death rate. In late December 2019, China was the epidemic center just before the spring festival [75]. However, with this perspective, the Chinese government has taken strict precautions to overcome the spread of the virus. Some of the best strategies deployed by China to combat this pandemic are given below.
  • Stop traveling from in and out of China to control the transmission.
  • Developed quarantine centers for suspected cases to get the best treatment.
  • China developed mobile apps for tracking suspected, confirmed cases, and interaction with individuals having COVID-19 symptoms.
  • Develop public awareness regarding self-protection, epidemiologic investigation, cleaning, and disinfecting the environment.
  • The government installed many intelligent based systems to monitor the public temperature, such as airports, metro stations, hospitals, communities.
There are different criteria for suspected, confirmed, asymptomatic, mild, and critical cases. All these cases were diagnosed and cured by following management criteria to diagnose COVID-19. Moreover, China’s current positive cases have dropped to a few or even zero [76]. However, the pandemic situation goes on critically in other countries. Noticeably, China’s policy implemented special preventive and diagnostic measures to control the deadly virus transmission and spread with more individuals testing. Oropharyngeal, nasopharyngeal swabs are primarily used for nucleic acid sampling [77].
According to Li Tong (Deputy Chief Physician of the department of respiratory and infectious disease of Beijing You’an Hospital), oropharyngeal swab tests are widely used and most appropriate for large-scale sampling. Nevertheless, the nasopharyngeal swab test accuracy is higher than the oropharyngeal swab tests, but the nasopharyngeal swab test is not convenient as oropharyngeal, and a person feels uncomfortable while sampling. On the flip side, China has started anal swab testing in Beijing to enhance infected people’s detection rate and minimize unpredictable diagnoses. The sample collection method for anal swabs is to insert a cotton-tipped swab about 1-2 inches into the rectum to detect the virus. However, the sample collection of the anal swab is not so appropriate as of oropharyngeal swab. Therefore, anal swab sample collection is limited to high-risk areas such as quarantine centers. It was observed that the virus lives a long time in the anus than the upper respiratory tract. Therefore, the anal swab tests remain positive in infected cases after giving negative results in the oropharyngeal swab [7]. Noticeably, it is concluded that anal swab tests could accurately identify the virus than any other swab tests. The significant purpose of anal swabs is to increase the detection rate and increase accuracy in identifying COVID-19 positive cases.

5. Challenges and Future Research Directions

Several challenges are required to be addressed in applying ML/DL techniques to detect COVID-19. Various kinds of diagnosis and detection strategies have been developed to control the spread of COVID-19 based pandemic, but the knowledge related to the applications of various ML/DL techniques to detect the virus is still insufficient. ML/DL approaches need a massive amount of data for computational models to discover and achieve information that is very limited in the case of COVID-19. The ML/DL techniques can only provide better results and valuable solutions to deal with the pandemic if a considerable amount of clinical data and medical images are available. To solve the issue, researchers need to work on appropriate ML/DL techniques that can provide better results using small datasets [78] such as L.S.T.M. [79].
Moreover, Multi-Criteria Decision Analysis (M.C.D.A.) [80] can be considered a practical solution that enables decision-making to deal with the issues related to COVID-19. As Li et al. [26] specify, radiology images give optimal results in differentiating coronavirus from other kinds of viral pneumonia, but it still needs to be more efficient to recognize imaging features accurately. In some cases, the presence of noisy, unclear, and inaccurate datasets can lead to misclassification of radiological images (C.T. and X-ray), which can be a significant difficulty in diagnosing COVID. Many complications hinder the training of ML/DL algorithms, such as data redundancy and missing values. Henceforth, researchers utilized different kinds of datasets to improve the training of algorithms to detect the virus. It will enable the radiologists and doctors to diagnose the C.T. and X-ray images more efficiently and promote awareness of viruses. Therefore, a standard framework is needed for future research to assess and analyze these issues. ML can be applied for preparator and feedback activities to tackle worldwide critical situations. For example, ML applications can be used to develop the most efficient robotic and automatic setup for sanitation, distributing food, medicine, and taking care of patients in hospitals. In the computer and biomedical engineering domain, ML has been partially utilized to find out the novel drug compounds against coronavirus [81,82].
Moreover, the usage of data science applications like ultrasound and Magnetic Resource Imaging (M.R.I.) for the detection of COVID-19 was limited. The ultrasound scans are as useful as chest C.T. scans, but these are not reported to be utilized to detect COVID-19. Some researchers [83] employed M.R.I to detect COVID-19, but due to the shortage of data, the M.R.I. technique’s feasibility and effectiveness are still unexplained. Generating well-defined datasets for the effective use of these methods to detect COVID-19 is another challenge. Moreover, upcoming research must be focused on the classification of COVID-19 by different symptoms of illness (such as fever, cough, congestion, sore throat, runny nose, diarrhea, and breathing difficulty) for precise and fast diagnosis in chronic patients (organ transplant). It is essential to utilize advanced ML/DL techniques to accurately evaluate infected patients and the corresponding death rate in the current scenario. One of the critical issues to be solved is the lack of interest by medical experts in examining medical images and clinical data. Furthermore, the manual classification of COVID-19 is a complicated and time-consuming process that demands the utilization of intelligent supervised ML/DL algorithms [84,85].

6. Conclusions

The global outbreak of novel coronavirus has affected millions of lives and demands various cost-effective diagnostic tests about the presence of COVID-19 infection. Fortunately, the current era of advanced technologies with ML/DL techniques has improved various medical aspects of human life and detect chronic and contagious diseases. There is a need for robust research to overcome the spread of this deadly virus by using ML-/DL-based intelligent models and taking preventive measures. In the recent systematic review, the analysis of ML/DL algorithms reported in the literature related to COVID-19 prediction, classification, and detection strategies has been presented. Several researchers have utilized X-ray, CT images, RT-PCR, and clinical blood data to assist the prognosis and anomalies of COVID-19. The analysis was performed to select appropriate ML/DL techniques to predict and diagnose the virus using radiological and clinical datasets. It is revealed that the highest accuracy of various ML/DL techniques such as CNN, D.N.N, SVM, K-NN, and R.F is 99%, 99.7%, 99.68%, 93.41%, and 95%, respectively, to detect COVID-19. These ML/DL techniques have achieved astounding performance results in every domain, along with medical research and radiology. However, DL has become dominant in various complicated tasks such as image classification and detection. Being familiar with these models’ key advantages will assist radiologist’s diagnosis research and develop an automated medical diagnosis decision support system for medical health experts. Finally, it can be concluded that ML and DL techniques played a significant role in the prediction, classification, screening, and minimizing the spread of the COVID-19 pandemic.

Author Contributions

Conceptualization, A.R. and H.X.; methodology, A.R., H.X., and M.A.I.; validation, A.R., H.X., M.A.I., and I.A.; formal analysis, M.A.I.; investigation, A.R., H.X., and M.A.I.; resources, H.X.; writing-original draft, A.R.; writing-review and editing, A.R., H.X., M.A.I., and I.A.; visualization, A.R., and I.A.; supervision, H.X. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the China Scholarship Council, China, grant number 201907005026.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Conflicts of Interest

The authors report no conflict of interest.

Abbreviations

The following abbreviations are used in this manuscript.
MLMachine Learning
MLTMachine Learning Techniques
CTComputer Tomography
CADComputer-Aided Diagnosis
CNNConvolutional Neural Network
D.N.NDeep Neural Network
DLDeep Learning
DTDecision Tree
RFRandom Forest
SVMSupport Vector Machine
SARSSevere Acute Respiratory Syndrome
K-NNK-Nearest Neighbors
L.S.T.M.Long Short-Term Memory
SDScience Direct
W.O.SWeb of Science
X-rayX-radiation
M.R.IMagnetic Resource Imaging
MERSMiddle East Respiratory syndrome
RT-PCRReverse Transcription Polymerase Chain Reaction
A.U.C.Area under the Receiver Operating Characteristic Curve

References

  1. Salman, F.M.; Abu-Naser, S.S.; Alajrami, E.; Abu-Nasser, B.S.; Alashqar, B.A. Covid-19 detection using artificial intelligence. IJAER 2020, 4, 18–25. [Google Scholar]
  2. Coronavirus. (COVID-19) Cases Statistics and Research Our World in Data. Available online: https://ourworldindata.org/covid-cases (accessed on 3 April 2021).
  3. Apostolopoulos, I.D.; Mpesiana, T.A. Covid-19: Automatic detection from X-ray images utilizing transfer learning with convolutional neural networks. Phys. Eng. Sci. Med. 2020, 43, 635–640. [Google Scholar] [CrossRef][Green Version]
  4. Ai, T.; Yang, Z.; Hou, H.; Zhan, C.; Chen, C.; Lv, W.; Tao, Q.; Sun, Z.; Xia, L.J.R. Correlation of chest C.T. and RT-PCR testing for coronavirus disease 2019 (COVID-19) in China: A report of 1014 cases. Radiology 2020, 296, E32–E40. [Google Scholar] [CrossRef][Green Version]
  5. Fang, Y.; Zhang, H.; Xie, J.; Lin, M.; Ying, L.; Pang, P.; Ji, W.J.R. Sensitivity of chest C.T. for COVID-19: Comparison to RT-PCR. Radiology 2020, 296, E115–E117. [Google Scholar] [CrossRef]
  6. Schwartz, D.A. An analysis of 38 pregnant women with COVID-19, their newborn infants, and maternal-fetal transmission of SARS-CoV-2: Maternal coronavirus infections and pregnancy outcomes. Arch. Pathol. Lab. Med. 2020, 144, 799–805. [Google Scholar] [CrossRef][Green Version]
  7. Tong, L. China Daily Website-Connecting China Connecting the World. Available online: http://www.chinadaily.com.cn/ (accessed on 15 March 2021).
  8. Bai, H.X.; Hsieh, B.; Xiong, Z.; Halsey, K.; Choi, J.W.; Tran, T.M.L.; Pan, I.; Shi, L.-B.; Wang, D.-C.; Mei, J.J.R. Performance of radiologists in differentiating COVID-19 from non-COVID-19 viral pneumonia at chest CT. Radiology 2020, 296, E46–E54. [Google Scholar] [CrossRef]
  9. Shi, F.; Wang, J.; Shi, J.; Wu, Z.; Wang, Q.; Tang, Z.; He, K.; Shi, Y.; Shen, D.J. Review of artificial intelligence techniques in imaging data acquisition, segmentation and diagnosis for covid-19. IEEE Rev. Biomed. Eng. 2020, 14, 4–15. [Google Scholar] [CrossRef][Green Version]
  10. Li, M.; Lei, P.; Zeng, B.; Li, Z.; Yu, P.; Fan, B.; Wang, C.; Li, Z.; Zhou, J.; Hu, S.; et al. Coronavirus disease (COVID-19): Spectrum of C.T. findings and temporal progression of the disease. Acad. Radiol. 2020, 27, 603–608. [Google Scholar] [CrossRef][Green Version]
  11. Ozturk, T.; Talo, M.; Yildirim, E.A.; Baloglu, U.B.; Yildirim, O.; Acharya, U.R. Automated detection of COVID-19 cases using deep neural networks with X-ray images. Comput. Biol. Med. 2020, 121, 103792. [Google Scholar] [CrossRef]
  12. Karar, M.E.; Hemdan, E.E.-D.; Shouman, M.A. Cascaded deep learning classifiers for computer-aided diagnosis of COVID-19 and pneumonia diseases in X-ray scans. Complex Intell. Syst. 2021, 7, 235–247. [Google Scholar] [CrossRef]
  13. Hizukuri, A.; Nakayama, R.; Nara, M.; Suzuki, M.; Namba, K. Computer-Aided Diagnosis Scheme for Distinguishing Between Benign and Malignant Masses on Breast DCE-MRI Images Using Deep Convolutional Neural Network with Bayesian Optimization. J. Digit. Imaging 2021, 34, 116–123. [Google Scholar] [CrossRef]
  14. Latif, S.; Qadir, J.; Farooq, S.; Imran, M.A. How 5g wireless (and concomitant technologies) will revolutionize healthcare? Future Internet 2017, 9, 93. [Google Scholar] [CrossRef][Green Version]
  15. Khan, S.; Islam, N.; Jan, Z.; Din, I.U.; Rodrigues, J.J.C. A novel deep learning based framework for the detection and classification of breast cancer using transfer learning. Pattern Recognit. Lett. 2019, 125, 1–6. [Google Scholar] [CrossRef]
  16. Lakhani, P.; Sundaram, B. Deep learning at chest radiography: Automated classification of pulmonary tuberculosis by using convolutional neural networks. Radiology 2017, 284, 574–582. [Google Scholar] [CrossRef]
  17. Toğaçar, M.; Ergen, B.; Cömert, Z.; Özyurt, F. A deep feature learning model for pneumonia detection applying a combination of mRMR feature selection and machine learning models. IRBM 2020, 41, 212–222. [Google Scholar] [CrossRef]
  18. Burlina, P.; Billings, S.; Joshi, N.; Albayda, J. Automated diagnosis of myositis from muscle ultrasound: Exploring the use of machine learning and deep learning methods. PLoS ONE 2017, 12, e0184059. [Google Scholar] [CrossRef]
  19. Canayaz, M. MH-COVIDNet: Diagnosis of COVID-19 using deep neural networks and meta-heuristic-based feature selection on X-ray images. Biomed. Signal Process. Control 2021, 64, 102257. [Google Scholar] [CrossRef]
  20. Sedik, A.; Iliyasu, A.M.; El-Rahiem, A.; Abdel Samea, M.E.; Abdel-Raheem, A.; Hammad, M.; Peng, J.; El-Samie, A.; Fathi, E.; El-Latif, A.A.A. Deploying machine and deep learning models for efficient data-augmented detection of COVID-19 infections. Viruses 2020, 12, 769. [Google Scholar] [CrossRef]
  21. Chen, X.-W.; Lin, X. Big data deep learning: Challenges and perspectives. IEEE Access 2014, 2, 514–525. [Google Scholar] [CrossRef]
  22. Alsalem, M.; Zaidan, A.; Zaidan, B.; Albahri, O.; Alamoodi, A.; Albahri, A.; Mohsin, A.; Mohammed, K.I. Multiclass benchmarking framework for automated acute Leukaemia detection and classification based on B.W.M. and group-VIKOR. J. Med. Syst. 2019, 43, 1–32. [Google Scholar] [CrossRef]
  23. Khan, M.A.; Abbas, S.; Khan, K.M.; Al Ghamdi, M.A.; Rehman, A. Intelligent forecasting model of COVID-19 novel coronavirus outbreak empowered with deep extreme learning machine. Comput. Mater. Contin. 2020, 64, 1329–1342. [Google Scholar] [CrossRef]
  24. Zaidan, A.; Zaidan, B.; Alsalem, M.; Albahri, O.; Albahri, A.; Qahtan, M.Y. Applications. Multi-agent learning neural network and Bayesian model for real-time IoT skin detectors: A new evaluation and benchmarking methodology. Neural Comput. Appl. 2020, 32, 8315–8366. [Google Scholar] [CrossRef]
  25. Yas, Q.M.; Zaidan, A.; Zaidan, B.; Rahmatullah, B.; Karim, H.A. Comprehensive insights into evaluation and benchmarking of real-time skin detectors: Review, open issues & challenges, and recommended solutions. Measurement 2018, 114, 243–260. [Google Scholar]
  26. Li, L.; Qin, L.; Xu, Z.; Yin, Y.; Wang, X.; Kong, B.; Bai, J.; Lu, Y.; Fang, Z.; Song, Q. Artificial intelligence distinguishes COVID-19 from community acquired pneumonia on chest CT. Radiology 2020, 296, E65–E71. [Google Scholar] [CrossRef]
  27. Long, J.B.; Ehrenfeld, J.M. The role of augmented intelligence (A.I.) in detecting and preventing the spread of novel coronavirus. J. Med Syst. 2020, 44, 59. [Google Scholar] [CrossRef][Green Version]
  28. Otoom, M.; Otoum, N.; Alzubaidi, M.A.; Etoom, Y.; Banihani, R. An IoT-based framework for early identification and monitoring of COVID-19 cases. Biomed. Signal Process. Control 2020, 62, 102149. [Google Scholar] [CrossRef]
  29. Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Int. J. Surg. 2010, 8, 336–341. [Google Scholar] [CrossRef][Green Version]
  30. Albahri, A.; Hamid, R.A.; Alwan, J.K.; Al-Qays, Z.; Zaidan, A.; Zaidan, B.; Albahri, A.; AlAmoodi, A.; Khlaf, J.M.; Almahdi, E.M. Role of biological data mining and machine learning techniques in detecting and diagnosing the novel coronavirus (COVID-19): A systematic review. J. Med. Syst. 2020, 44, 1–11. [Google Scholar] [CrossRef]
  31. Thenault, R.; Kaulanjan, K.; Darde, T.; Rioux-Leclercq, N.; Bensalah, K.; Mermier, M.; Khene, Z.-E.; Peyronnet, B.; Shariat, S.; Pradere, B.; et al. The Application of Artificial Intelligence in Prostate Cancer Management—What Improvements Can Be Expected? A Systematic Review. Appl. Sci. 2020, 10, 6428. [Google Scholar] [CrossRef]
  32. Kooi, T.; Litjens, G.; Van Ginneken, B.; Gubern-Mérida, A.; Sánchez, C.I.; Mann, R.; den Heeten, A.; Karssemeijer, N. Large scale deep learning for computer aided detection of mammographic lesions. Med. Image Anal. 2017, 35, 303–312. [Google Scholar] [CrossRef]
  33. Yassin, N.I.; Omran, S.; El Houby, E.M.; Allam, H. Machine learning techniques for breast cancer computer aided diagnosis using different image modalities: A systematic review. Comput. Methods Progr. Biomed. 2018, 156, 25–45. [Google Scholar] [CrossRef]
  34. Pehrson, L.M.; Nielsen, M.B.; Ammitzbøl, L.C. Automatic pulmonary nodule detection applying deep learning or machine learning algorithms to the LIDC-IDRI database: A systematic review. Diagnostics 2019, 9, 29. [Google Scholar] [CrossRef] [PubMed][Green Version]
  35. Narin, A.; Kaya, C.; Pamuk, Z. Automatic detection of coronavirus disease (covid-19) using X-ray images and deep convolutional neural networks. arXiv 2020, arXiv:2003.10849. [Google Scholar]
  36. Khan, A.I.; Shah, J.L.; Bhat, M.M. CoroNet: A deep neural network for detection and diagnosis of COVID-19 from chest X-ray images. Comput. Methods Programs Biomed. 2020, 196, 105581. [Google Scholar] [CrossRef]
  37. Brunese, L.; Mercaldo, F.; Reginelli, A.; Santone, A. Explainable deep learning for pulmonary disease and coronavirus COVID-19 detection from X-rays. Comput. Methods Progr. Biomed. 2020, 196, 105608. [Google Scholar] [CrossRef]
  38. Pham, T.D. Classification of COVID-19 chest X-rays with deep learning: New models or fine tuning? Health Inf. Sci. Syst. 2021, 9, 1–11. [Google Scholar] [CrossRef]
  39. Ardakani, A.A.; Kanafi, A.R.; Acharya, U.R.; Khadem, N.; Mohammadi, A. Application of deep learning technique to manage COVID-19 in routine clinical practice using C.T. images: Results of 10 convolutional neural networks. Comput. Biol. Med. 2020, 121, 103795. [Google Scholar] [CrossRef] [PubMed]
  40. Xu, X.; Jiang, X.; Ma, C.; Du, P.; Li, X.; Lv, S.; Yu, L.; Ni, Q.; Chen, Y.; Su, J.; et al. A deep learning system to screen novel coronavirus disease 2019 pneumonia. Engineering 2020, 6, 1122–1129. [Google Scholar] [CrossRef]
  41. Zheng, C.; Deng, X.; Fu, Q.; Zhou, Q.; Feng, J.; Ma, H.; Liu, W.; Wang, X. Deep learning-based detection for COVID-19 from chest C.T. using weak label. MedRxiv 2020, 39, 2615–2625. [Google Scholar]
  42. Wang, S.; Kang, B.; Ma, J.; Zeng, X.; Xiao, M.; Guo, J.; Cai, M.; Yang, J.; Li, Y.; Meng, X.; et al. A deep learning algorithm using C.T. images to screen for Corona Virus Disease (COVID-19). Eur. Radiol. 2021, 31, 1–9. [Google Scholar]
  43. Song, Y.; Zheng, S.; Li, L.; Zhang, X.; Zhang, X.; Huang, Z.; Chen, J.; Zhao, H.; Jie, Y.; Wang, R.; et al. Deep learning enables accurate diagnosis of novel coronavirus (COVID-19) with C.T. images. IEEE/ACM Trans. Comput. Biol. Bioinform. 2021. [Google Scholar] [CrossRef] [PubMed]
  44. Ardakani, A.A.; Acharya, U.R.; Habibollahi, S.; Mohammadi, A. COVIDiag: A clinical CAD system to diagnose COVID-19 pneumonia based on C.T. findings. Eur. Radiol. 2021, 31, 121–130. [Google Scholar] [CrossRef] [PubMed]
  45. Wang, L.; Lin, Z.Q.; Wong, A. Covid-net: A tailored deep convolutional neural network design for detection of covid-19 cases from chest X-ray images. Sci. Rep. 2020, 10, 1–12. [Google Scholar] [CrossRef]
  46. Sethy, P.K.; Behera, S.K.; Ratha, P.K.; Biswas, P. Detection of coronavirus disease (COVID-19) based on deep features and support vector machine. Electr. Electron. Eng. 2020. [Google Scholar] [CrossRef]
  47. Hemdan, E.E.-D.; Shouman, M.A.; Karar, M.E. Covidx-net: A framework of deep learning classifiers to diagnose covid-19 in X-ray images. arXiv 2020, arXiv:2003.11055. [Google Scholar]
  48. Narayanan, B.N.; Hardie, R.C.; Krishnaraja, V.; Karam, C.; Davuluru, V.S.P. Transfer-to-Transfer Learning Approach for Computer Aided Detection of COVID-19 in Chest Radiographs. AI 2020, 1, 539–557. [Google Scholar] [CrossRef]
  49. Gozes, O.; Frid-Adar, M.; Greenspan, H.; Browning, P.D.; Zhang, H.; Ji, W.; Bernheim, A.; Siegel, E. Rapid ai development cycle for the coronavirus (covid-19) pandemic: Initial results for automated detection & patient monitoring using deep learning ct image analysis. arXiv 2020, arXiv:2003.05037. [Google Scholar]
  50. Barstugan, M.; Ozkaya, U.; Ozturk, S. Coronavirus (covid-19) classification using ct images by machine learning methods. arXiv 2020, arXiv:2003.09424. [Google Scholar]
  51. Ghoshal, B.; Tucker, A. Estimating uncertainty and interpretability in deep learning for coronavirus (COVID-19) detection. arXiv 2020, arXiv:2003.10769. [Google Scholar]
  52. Bai, X.; Fang, C.; Zhou, Y.; Bai, S.; Liu, Z.; Xia, L.; Chen, Q.; Xu, Y.; Xia, T.; Gong, S.; et al. Predicting COVID-19 malignant progression with A.I. techniques. medRxiv 2020. [Google Scholar] [CrossRef][Green Version]
  53. Jin, C.; Chen, W.; Cao, Y.; Xu, Z.; Tan, Z.; Zhang, X.; Deng, L.; Zheng, C.; Zhou, J.; Shi, H.; et al. Development and evaluation of an artificial intelligence system for COVID-19 diagnosis. Nat. Commun. 2020, 11, 1–14. [Google Scholar] [CrossRef] [PubMed]
  54. Jin, S.; Wang, B.; Xu, H.; Luo, C.; Wei, L.; Zhao, W.; Hou, X.; Ma, W.; Xu, Z.; Zheng, Z.; et al. AI-assisted CT imaging analysis for COVID-19 screening: Building and deploying a medical A.I. system in four weeks. medRxiv 2020. [Google Scholar] [CrossRef][Green Version]
  55. Chowdhury, M.E.; Rahman, T.; Khandakar, A.; Mazhar, R.; Kadir, M.A.; Mahbub, Z.B.; Islam, K.R.; Khan, M.S.; Iqbal, A.; Al Emadi, N.; et al. Can AI help in screening viral and COVID-19 pneumonia? IEEE Access 2020, 8, 132665–132676. [Google Scholar] [CrossRef]
  56. Maghdid, H.S.; Asaad, A.T.; Ghafoor, K.Z.; Sadiq, A.S.; Khan, M.K. Diagnosing COVID-19 pneumonia from X-ray and C.T. images using deep learning and transfer learning algorithms. arXiv 2020, arXiv:2004.00038. [Google Scholar]
  57. Zhang, J.; Xie, Y.; Li, Y.; Shen, C.; Xia, Y. Covid-19 screening on chest X-ray images using deep learning based anomaly detection. arXiv 2020, arXiv:2003.12338. [Google Scholar]
  58. de Moraes, B.A.F.; Miraglia, J.L.; Donato, T.H.R.; Filho, A.D.P.C. COVID-19 diagnosis prediction in emergency care patients: A machine learning approach. MedRxiv 2020. [Google Scholar] [CrossRef][Green Version]
  59. Jiang, X.; Coffee, M.; Bari, A.; Wang, J.; Jiang, X.; Huang, J.; Shi, J.; Dai, J.; Cai, J.; Zhang, T.; et al. Towards an artificial intelligence framework for data-driven prediction of coronavirus clinical severity. Comput. Mater. Contin. 2020, 63, 537–551. [Google Scholar] [CrossRef]
  60. Wang, S.; Zha, Y.; Li, W.; Wu, Q.; Li, X.; Niu, M.; Wang, M.; Qiu, X.; Li, H.; Yu, H.; et al. A fully automatic deep learning system for COVID-19 diagnostic and prognostic analysis. Eur. Respir. J. 2020, 56. [Google Scholar] [CrossRef]
  61. Zhao, J.; Zhang, Y.; He, X.; Xie, P. Covid-ct-dataset: A ct scan dataset about covid-19. arXiv 2020, arXiv:2003.13865. [Google Scholar]
  62. Asnaoui, K.E.; Chawki, Y.; Idri, A. Automated methods for detection and classification pneumonia based on X-ray images using deep learning. arXiv 2020, arXiv:2003.14363. [Google Scholar]
  63. Bhandary, A.; Prabhu, G.A.; Rajinikanth, V.; Thanaraj, K.P.; Satapathy, S.C.; Robbins, D.E.; Shasky, C.; Zhang, Y.-D.; Tavares, J.M.R.; Raja, N.S.M. Deep-learning framework to detect lung abnormality–A study with chest X-ray and lung C.T. scan images. Pattern Recognit. Lett. 2020, 129, 271–278. [Google Scholar] [CrossRef]
  64. Farooq, M.; Hafeez, A. Covid-resnet: A deep learning framework for screening of covid19 from radiographs. arXiv 2020, arXiv:2003.14395. [Google Scholar]
  65. Abbas, A.; Abdelsamea, M.M.; Gaber, M.M. Classification of COVID-19 in chest X-ray images using DeTraC deep convolutional neural network. Appl. Intell. 2021, 51, 854–864. [Google Scholar] [CrossRef]
  66. Karim, M.; Döhmen, T.; Rebholz-Schuhmann, D.; Decker, S.; Cochez, M.; Beyan, O. Deepcovidexplainer: Explainable covid-19 predictions based on chest X-ray images. arXiv 2020, arXiv:2004.04582. [Google Scholar]
  67. Sitaula, C.; Hossain, M.B. Attention-based VGG-16 model for COVID-19 chest X-ray image classification. Appl. Intell. 2020. [Google Scholar] [CrossRef]
  68. Keles, A.; Keles, M.B.; Keles, A. COV19-CNNet and COV19-ResNet: Diagnostic inference Engines for early detection of COVID-19. Cogn. Comput. 2021, 13, 1–11. [Google Scholar]
  69. Chandra, T.B.; Verma, K.; Singh, B.K.; Jain, D.; Netam, S.S. Coronavirus disease (COVID-19) detection in Chest X-Ray images using majority voting based classifier ensemble. Expert Syst. Appl. 2021, 165, 113909. [Google Scholar] [CrossRef]
  70. Wu, J.; Zhang, P.; Zhang, L.; Meng, W.; Li, J.; Tong, C.; Li, Y.; Cai, J.; Yang, Z.; Zhu, J. Rapid and accurate identification of COVID-19 infection through machine learning based on clinical available blood test results. medRxiv 2020. [Google Scholar] [CrossRef]
  71. Kukar, M.; Gunčar, G.; Vovko, T.; Podnar, S.; Černelč, P.; Brvar, M.; Zalaznik, M.; Notar, M.; Moškon, S.; Notar, M. COVID-19 diagnosis by routine blood tests using machine learning. arXiv 2020, arXiv:2006.03476. [Google Scholar]
  72. Arpaci, I.; Huang, S.; Al-Emran, M.; Al-Kabi, M.N.; Peng, M. Predicting the COVID-19 infection with fourteen clinical features using machine learning classification algorithms. Multimed. Tools Appl. 2021, 80, 1–15. [Google Scholar]
  73. Das, N.N.; Kumar, N.; Kaur, M.; Kumar, V.; Singh, D. Automated deep transfer learning-based approach for detection of COVID-19 infection in chest X-rays. IRBM 2020. [Google Scholar] [CrossRef]
  74. Wu, M.-H.; Chen, K.-Y.; Shih, S.-R.; Ho, M.-C.; Tai, H.-C.; Chang, K.-J.; Chen, A.; Chen, C.-N. Multi-reader multi-case study for performance evaluation of high-risk thyroid ultrasound with computer-aided detection. Cancers 2020, 12, 373. [Google Scholar] [CrossRef][Green Version]
  75. Riou, J.; Althaus, C.L. Pattern of early human-to-human transmission of Wuhan 2019 novel coronavirus (2019-nCoV), December 2019 to January 2020. Eurosurveillance 2020, 25, 2000058. [Google Scholar] [CrossRef]
  76. Ki, M. Epidemiologic characteristics of early cases with 2019 novel coronavirus (2019-nCoV) disease in Korea. Epidemiol. Health 2020, 42, e2020007. [Google Scholar] [CrossRef]
  77. Xu, T.-L.; Ao, M.-Y.; Zhou, X.; Zhu, W.-F.; Nie, H.-Y.; Fang, J.-H.; Sun, X.; Zheng, B.; Chen, X.-F. China’s practice to prevent and control COVID-19 in the context of large population movement. Infect. Dis. Poverty 2020, 9, 1–14. [Google Scholar] [CrossRef] [PubMed]
  78. Oh, Y.; Park, S.; Ye, J.C. Deep learning covid-19 features on cxr using limited training data sets. IEEE Trans. Med. Imaging 2020, 39, 2688–2700. [Google Scholar] [CrossRef]
  79. Swapnarekha, H.; Behera, H.S.; Nayak, J.; Naik, B. Role of intelligent computing in COVID-19 prognosis: A state-of-the-art review. Chaos Solitons Fractals 2020, 138, 109947. [Google Scholar] [CrossRef] [PubMed]
  80. Mohammed, K.; Zaidan, A.; Zaidan, B.; Albahri, O.; Alsalem, M.; Albahri, A.; Hadi, A.; Hashim, M. Real-time remote-health monitoring systems: A review on patients prioritisation for multiple-chronic diseases, taxonomy analysis, concerns and solution procedure. J. Med. Syst. 2019, 43, 1–21. [Google Scholar] [CrossRef] [PubMed]
  81. Heo, L.; Feig, M. Modeling of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) proteins by machine learning and physics-based refinement. BioRxiv 2020. [Google Scholar] [CrossRef][Green Version]
  82. Zhavoronkov, A.; Zagribelnyy, B.; Zhebrak, A.; Aladinskiy, V.; Terentiev, V.; Vanhaelen, Q.; Bezrukov, D.S.; Polykovskiy, D.; Shayakhmetov, R.; Filimonov, A. Potential non-covalent SARS-CoV-2 3C-like protease inhibitors designed using generative deep learning approaches and reviewed by human medicinal chemist in virtual reality. ChemRxiv 2020. [Google Scholar] [CrossRef]
  83. Poyiadji, N.; Shahin, G.; Noujaim, D.; Stone, M.; Patel, S.; Griffith, B. COVID-19–associated acute hemorrhagic necrotizing encephalopathy: Imaging features. Radiology 2020, 296, E119–E120. [Google Scholar] [CrossRef] [PubMed][Green Version]
  84. Abbas, A.; Abdelsamea, M.M.; Gaber, M. 4S-DT: Self Supervised Super Sample Decomposition for Transfer learning with application to COVID-19 detection. arXiv 2020, arXiv:2007.11450. [Google Scholar]
  85. Stochiţoiu, R.D.; Rebedea, T.; Popescu, I.; Leordeanu, M. A self-supervised neural-analytic method to predict the evolution of covid-19 in Romania. arXiv 2020, arXiv:2006.12926. [Google Scholar]
Figure 1. (a) Daily new confirmed COVID-19 cases Worldwide (b). COVID-19 death rate statistics among top ten infected countries last updated 17 March 2021 [2]. (Sources: Center for Systems Science and Engineering (C.S.S.E.) at Johns Hopkins University, Baltimore, MD, USA).
Figure 1. (a) Daily new confirmed COVID-19 cases Worldwide (b). COVID-19 death rate statistics among top ten infected countries last updated 17 March 2021 [2]. (Sources: Center for Systems Science and Engineering (C.S.S.E.) at Johns Hopkins University, Baltimore, MD, USA).
Applsci 11 03414 g001
Figure 2. Search and selection strategy for systematic review.
Figure 2. Search and selection strategy for systematic review.
Applsci 11 03414 g002
Figure 3. Distribution of published articles (ML/DL techniques for COVID-19 prediction).
Figure 3. Distribution of published articles (ML/DL techniques for COVID-19 prediction).
Applsci 11 03414 g003
Figure 4. ML/DL techniques used in this review articles.
Figure 4. ML/DL techniques used in this review articles.
Applsci 11 03414 g004
Figure 5. Percentage utilization of various diagnostic techniques for the detection of COVID-19.
Figure 5. Percentage utilization of various diagnostic techniques for the detection of COVID-19.
Applsci 11 03414 g005
Table 1. Summary of the datasets used in the selected papers.
Table 1. Summary of the datasets used in the selected papers.
ReferenceTest DataCOVID-19
Positive
Pneumonia/Other InfectionHealthyTotalSources/Links
[35]Chest X-ray
CT Images
50* NA50100https://github.com/ieee8023/covid-chestxray-dataset
https://www.kaggle.com/paultimothymooney/chest-xray
[36]Chest X-ray2846573101251https://github.com/ieee8023/covid-chestxray-dataset
https://www.kaggle.com/paultimothymooney/chest-xray
[37]Chest X-ray250275335206523https://github.com/ieee8023/covid-chestxray-dataset
https://github.com/muhammedtalo/COVID-19
https://www.kaggle.com/nih-chest-xrays/sample
[38]Chest X-ray274205113413666https://www.kaggle.com/tawsifurrahman/covid19-radiography-database
https://github.com/agchung/Figure1-COVID-chestxray-dataset
https://github.com/ieee8023/covid-chestxray-dataset
[26]CT Images1296173513254356https://github.com/bkong999/COVNet
[39]CT Images510510* NA1020Alexion, Toshiba Medical System, Japan
[11]Chest X-ray25050010001750https://github.com/ieee8023/COVID-chestxray-dataset
[40]CT Images219224175618Hospital of Zhejiang University
Hospital of Wenzhou
Hospital of Wenling
[41]CT Images313* NA229542Union Hospital Tongji Medical College, Huazhong University of Science and Technology, China.
[42]CT Images325740* NA1065Xi’an Jiaotong University First Affiliated Hospital
Nanchang University First Hospital
Xi’an No.8 Hospital of Xi’an Medical College, China
[43]CT Images777** NS7081485The Third Affiliated Hospital and Sun Yat-Sen Memorial Hospital, Sun Yat-sen University Guangzhou
Renmin Hospital of Wuhan University, China
[44]CT Images306* NA306612University of Medical Science (I.U.M.S.), Iran
[3]Chest X-ray2247005041428https://github.com/ieee8023/covid-chestxray-dataset
https://www.kaggle.com/andrewmvd/convid19-X-rays
[45]Chest X-ray535526806613,645https://github.com/agchung/Figure1-COVID-chestxray-dataset
https://github.com/agchung/Actualmed-COVID-chestxray-dataset
https://www.kaggle.com/tawsifurrahman/covid19-radiography-database
https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/data
[46]Chest X-ray127127127381https://openi.nlm.nih.gov/
www.kaggle.com
www.github.com
[47]Chest X-ray25* NA2550https://github.com/ieee8023/covid-chestxray-dataset
https://www.pyimagesearch.com/category/medical/
[48]Chest X-ray239 + 2265 (BIMCV)4273 + 95115839311Japanese Radiological Scientific Technology (J.R.S.T.)
Shenzhen Dataset, Montgomery Dataset University of Montreal, Valencian Region Medical Image Bank (B.I.M.C.V.)
https://github.com/ieee8023/covid-chestxray-dataset
[49]CT Images565249157http://www.chainz.cn/Hospital in Wenzhou, ChinaEI-Camino Hospital, USA
[50]CT Images
(abdominal)
53** NS* NA150https://www.sirm.org/en/
[51]Chest X-ray6827861583+ 15045941https://github.com/ieee8023/covid-chestxray-dataset
[52]CT Images133** N.S.* NA199Wuhan Pulmonary Hospital, china
[53]Chest X-ray andCT Images2228330823817917https://github.com/ChenWWWeixiang/diagnosis_covid19
https://tianchi.aliyun.com/competition/entrance/231601/information
Wuhan Union Hospital and Jianghan Mobile cabin Hospital, China
[54]CT Images877* NA5411418Beijing Tsinghua Changgung Hospital, China, Wuhan No.7 Hospital, Wuhan Leishenshan Hospital, Zhongnan Hospital of Wuhan University, Wuhan, Tianyou Hospital Affiliated to Wuhan University of Science and Technology, Wuhan, China
[55]Chest X-ray423148515793487https://www.kaggle.com/tawsifurrahman/covid19-radiography-database
https://www.sirm.org/category/senza-categoria/covid-19/
https://github.com/ieee8023/covid-chestxray-dataset
https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia
[56]Chest X-ray
and
CT Images
288** NS238526https://www.bsti.org.uk/training-and-education/covid-19-bsti-imaging-database/
https://radiopaedia.org/articles/normal-chest-imaging-examples?lang=gb
https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia/metadata
[57]Chest X-ray100** NS14311531https://github.com/ieee8023/covid-chestxray-dataset
[58]RT-PCR102** NS* NA235Hospital Israelita Albert Einstein, Brazil
[59]RT-PCR** NS** NS* NA53Wenzhou Central Hospital and Cangnan People’s Hospital Wenzhou, China
[60]Chest CT
Images
9244448* NA5372Renmin Hospital of Wuhan University, Henan Provincial People’s hospital, First Hospital of China Medical University First Affiliated Hospital of Anhui Medical University Beijing Youan Hospital of Capital Medical University, Huangshi Central Hospital,
[61]Chest CT
Images
349* NA463812https://www.sirm.org/
https://radiopaedia.org/
https://www.kaggle.com/tawsifurrahman/covid19-radiography-database/
https://coronacases.org/
https://www.eurorad.org/
[62]Chest X-ray
CT Images
** NS427315835856[63]
[64]Chest X-ray459316601636https://github.com/lindawangg/COVID-Net
https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/data
[65]Chest X-ray105* NA80185https://github.com/ieee8023/covid-chestxray-dataset
[66]Chest X-ray6801845 + 3457997715,959https://www.kaggle.com/c/rsna-pneumonia-detection-challenge
https://www.kaggle.com/tawsifurrahman/covid19-radiography-database
https://github.com/ieee8023/covid-chestxray-dataset
https://github.com/muhammedtalo/COVID-19
[67]Chest X-ray534115713103001https://github.com/ieee8023/covid-chestxray-dataset
https://www.kaggle.com/paultimothymooney/chest-Xray
[68]Chest X-ray210350350910https://www.kaggle.com/tawsifurrahman/covid19-radiography-database
[69]Chest X-ray6966966962088https://github.com/ieee8023/covid-chestxray-dataset
[70]Clinical Blood TestSuspected Covid-19
105
148* NA253Gansu Provincial Hospital, Lanzhou Pulmonary Hospital,
The First Hospital of Lanzhou University, The First People’s Hospital of Lanzhou City, Lanzhou University Second Hospital, China
[71]Clinical Blood Test1605333* NA5493University Medical Centre Ljubljana (U.M.C.L.), Slovenia
[72]Clinical Blood Test82* NA32114Taizhou Hospital Zhejiang, China
* Not Applied (N.A.) represents those kinds of images that were not utilized; ** Not Specified (N.S.) shows the images that were utilized but did not explain the number clearly.In Table 1, some significant available data sources were also presented and utilized by other researchers to explore more valuable solutions to deal with this pandemic. The datasets using X-ray images were higher in number than those with C.T. images. The percentage contribution of different diagnostic methods used for COVID-19 detection is presented in Figure 5. It can be noticed that X-ray image-based detection was the most utilized diagnostic method, contributing 47% of the total. C.T. image-based models were the second-largest diagnostic methods accounting for 30%, whereas the approaches with the least utilization for diagnosis of the virus were X-ray and C.T. image-based combined models (10%), clinical blood test (8%), and RT-PCR data-based models (5%).
Table 2. Summary of state-of-art COVID-19 prediction algorithms.
Table 2. Summary of state-of-art COVID-19 prediction algorithms.
ReferenceTest TypeML/DL TechniquesPrediction
Results
CountryCited by No of Papers
[26]CT ImagesCNN, COVNetAUC 0.96China553
[39]CT ImagesCNNs, ResNet-101 & XceptionAUC of 0.99
Sensitivity 98.02% Specificity 99.51%
Iran120
[41]CT Images3-D DNN, DeCoVNetAccuracy
0.90
China205
[42]C.T. ImagesInception Transfer learning model establish the algorithmAccuracy of 89.5% with Specificity of 0.88 and Sensitivity of 0.87China376
[43]C.T. ImagesD.N.N.,
DRE-Net
A.U.C. of 0.99
Sensitivity of 0.93
China198
[49]CT Images2D and 3D deep learning (Resnet-50-2D) and AI ModelsAUC of 0.99
Sensitive 92.2%
Specificity 92.2%
China306
[50]C.T. ImagesClassification
Stage 1 SVM, Stage 2 GLCM, GLSZ MDWT
Accuracy of 99.68%Turkey103
[52]C.T. ImagesMultilayer perceptron and LSTMAUC of 0.954China48
[54]CT ImagesCombined model 3D UNet++ and RestNet-50AUC of 0.991
Sensitivity of 0.974 and specificity of 0.922
China99
[60]C.T. Images3D-DNN,
COVID-19Net
AUC 0.86 Sensitivity of 79.35% and specificity of 71.43%China95
[61]C.T. ImagesCNN, Multi-task learning, self-supervised learning, DenseNet-169Accuracy of 0.89 and AUC 0.90China175
[44]C.T. ImagesFive classifiers, Decision tree, k-nearest neighbor, naïve Bayes, support vector machine, Proposed COVIDiag modelAccuracy of 91.4% sensitivity of 93.24%, and specificity of 90.32%Iran7
[36]X-ray ImagesCNN, CoroNetOverall Accuracy
89.6%
India159
[37]X-ray ImagesCNN
VGG16
Average accuracy
0.97%
Italy69
[11]X-ray ImagesCNN
DarkCovidNet
Accuracy of 98.08%Turkey417
[40]CT Images3-D CNN
ResNet-18
Overall Accuracy
86.7%
China443
[3]X-ray ImagesCNN, MobileNet v2Accuracy of 96.78%
Sensitive 98.66%
Specificity 96.46%
Greece480
[45]X-ray ImagesD.N.N., VGG-19
ResNet-50, COVID-Net
Accuracy of
93.3%
Canada558
[46]X-ray ImagesCNN, RestNet50 + SVMAccuracy of
95.33%
India250
[47]X-ray ImagesD-CNN, VGG19,
DenseNet201
Classification with F1-scores of 0.89 and 0.91Egypt251
[68]X-ray ImagesCNN, COV19-ResNet, COV19-CNNetAccuracy of 97.61%Turkey1
[48]X-ray ImagesResNet50, DenseNet201, Inception-v3, and XceptionAUC 0.996,
Overall sensitivity of 0.94
USA2
[51]X-ray ImagesCNN, Bayesian ResNet50V2Accuracy of 89.92%UK104
[55]X-ray ImagesDCNN, CheXNet +
DenseNet-201
Accuracy of 99.7%
precision, sensitivity of 99.7% and specificity is 99.55%
Qatar161
[57]X-ray ImagesCNN, Classification
Grad-CAM
AUC 95.13%, the sensitivity of 90%, and specificity of 87.84%China161
[64]X-ray ImagesCNN, COVID-ResNetAccuracy of 96.23%USA125
[65]X-ray ImagesD-CNN, DeTraCAccuracy of 95.12% Sensitivity of 97.91% and Specificity of 91.87%Egypt161
[66]X-ray ImagesD.N.N., Deep COVID ExplainerPositive Predictive Value 96.12% and recall of 94.3%Germany34
[67]X-ray ImagesVGG-16
VGG-19
Accuracy of 87.49%Australia5
[38]X-ray ImagesCNN, AlexNet,
GoogleNet,
SqueezeNet
Overall accuracy 99%Saudi Arabia4
[69]X-ray ImagesClassification Models
K.N.N., ANN, D.T., SVM,
Overall accuracy of 93.41%India8
[35]Chest X-ray
CT Images
CNN, RestNet50, Inception V3, Inception-RestNetV2Accuracy of 98%Turkey408
[53]Chest X-ray
CT Images
D-CNN, A.I. systemA.U.C. of 97.91%
The sensitivity of 90.19% and specificity of 95.76%
China97
[56]Chest X-ray
CT Images
Simple 2-D CNN Model and pre-trained AlexNet with transfer learningAccuracy of 98% on X-ray images and 94.1% on C.T. imagesIraq80
[62]Chest X-ray
CT Images
D-CNN, Resnet50,
MobileNet_V2, Inception_Resnet_V2
Accuracy of 96.61%Morocco39
[58]RT-PCRSupport Vector Machine, Random Forests, Neural Networks, Logistic RegressionAUC 0.847, Sensitivity 0.67, Specificity 0.85Brazil31
[59]RT-PCRSupport Vector Machine, K.N.N., Decision Tree, Random ForestAccuracy of 80%China147
[70]Clinical Blood TestRandom Forests
(R.F.)
Accuracy of 0.9512, Sensitivity 0.9697, and Specificity 0.9595China27
[71]Clinical Blood TestCRISP-DM,
Random Forest,
Deep Neural Network,
Extreme gradient Boosting Machine (XGBoost)
A.U.C. of 0.97, Sensitivity 81.9%, and specificity of 97.9%Switzerland5
[72]Clinical Blood TestSix Classification Model
BayesNet, logistic-regression, lazy-classifier, meta-classifier, classification via regression, decision-tree (J48)
Accuracy of 84.24%Turkey0
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Back to TopTop