A Meta-Analysis of Computerized Tomography-Based Radiomics for the Diagnosis of COVID-19 and Viral Pneumonia

Introduction: Coronavirus disease 2019 (COVID-19) led to a global pandemic. Although reverse transcription polymerase chain reaction (RT-PCR) of viral nucleic acid is the gold standard for COVID-19 diagnosis, its sensitivity was found to not be high enough in many reports. As radiomics-based diagnosis research has recently emerged, we aimed to use computerized tomography (CT)-based radiomics models to differentiate COVID-19 pneumonia from other viral pneumonia infections. Materials and methods: This study was performed according to the preferred reporting items for systematic review and meta-analysis diagnostic test accuracy studies (PRISMA-DTA) guidelines. The Pubmed, Cochrane, and Embase databases were searched. The pooled sensitivity and pooled specificity were calculated. A summary receiver operating characteristic (sROC) curve was constructed. The study quality was evaluated based on the radiomics quality score. Results: A total of 10,300 patients were involved in this meta-analysis. The radiomics quality score ranged from 13 to 16 (maximum score: 36). The pooled sensitivity was 0.885 (95% CI: 0.818–0.929), and the pooled specificity was 0.811 (95% CI: 0.667–0.902). The pooled AUC was 906. Conclusion: Our meta-analysis showed that CT-based radiomics feature models can successfully differentiate COVID-19 from other viral pneumonias.


Introduction
Coronavirus disease 2019 (COVID- 19) led to a global pandemic featuring a highly contagious disease which has led to millions of deaths worldwide. Although reverse transcription polymerase chain reaction (RT-PCR) of viral nucleic acid is the gold standard for COVID-19 diagnosis [1], its sensitivity was found to not be high enough in many reports [2][3][4].
On the other hand, chest x-ray (CXR) and computerized tomography (CT) are helpful in the diagnosis of COVID-19 pneumonia [5][6][7]. According to recent experience, lung image findings are observed earlier than clinical manifestations, highlighting the importance of image exams for screening pneumonias [8]. Early diagnosis can also reduce disease transmission risk and prevent the endemic of COVID-19 [9]. CXR is convenient, easily accessible, and can avoid cross contamination between users. Although the sensitivity of CXR is lower than CT in diagnosing COVID-19 pneumonia. With artificial intelligence machine learning model assistance, CXR can achieve an improved COVID-19 diagnostic rate [10][11][12].
However, the power of generalization is low when the machine learning model was trained on sets of heterogeneous images. The absence of an adequate evaluation protocol also limited the artificial intelligence assistance in CXR and its utility in clinical settings [13]. Nonetheless, CXR is useful to monitor disease progression in unstable patients.
Articles written in English; 2.
Full text available.
The exclusion criteria were as follows: 1. Studies using only deep learning features; 2.
Conference papers or studies with only the abstract available.

Workflow of the Radiomics Study
In the selected studies, the radiomics-based machine learning process was similar. First, the CT images acquisition and region-of-interest segmentation were performed by radiologists. Next, the radiomics feature extraction, model training and cross validation were performed by artificial intelligence. Finally, the data analysis and clinical application was evaluated by clinicians.

Data Extraction
The main outcome was set as the highest area under the curve (AUC) in the validation dataset. In the absence of an external validation set, we chose the value from the cross-validation process or hold-out dataset. We also extracted the true positive (TP), false positive (FP), true negative (TN), and false negative (FN) values from the literature. Furthermore, we gathered the name of the first author, the nation of the first author, the publication year of the study, the region of interest (ROI), the patient number, and other characteristics from the selected studies.

Statistical Analysis
Pooled analysis was carried out using a random effects model. The pooled sensitivity and pooled specificity were calculated. A forest plot and summary receiver operating characteristic (ROC) plot were created. The heterogeneity was assessed using the chi-square test. The statistical analysis was performed with the R language using R studio.

Bias Assessment
The publication bias was assessed using a funnel plot. Egger's test was only conducted if the number of included articles was more than 10. The statistical analysis was performed with the R language using R studio.

Quality Assessment
The RQS (radiomics quality score) was used to evaluate the chosen studies [29]. Two authors independently scored the table. Any inconsistencies between the authors were resolved by consensus.

Literature Collection
First, we collected a total of 348 articles. After duplicate removal, 283 publications were selected for meticulous evaluation. After assessment of the title and abstracts, ten publications were selected, and their full texts were retrieved. One observational study [30] without radiomics application in the research and two observational studies [31,32] with a repetitive patient population were excluded. After the article selection process, seven articles were used in the qualitative analysis [33][34][35][36][37][38][39], and six articles were further used in the meta-analysis. The PRISMA flowchart is provided in Figure 1. Details of the selected studies are listed in Table 1. Only the six studies were used in the meta-analysis [33][34][35][36][37][38]. Wang's study [39] was excluded due to the predictions being made on the basis of the CT slice number. A total of 10,300 patients were involved in this meta-analysis. Among them, 3587 patients had COVID-19 pneumonia.

Quality Assessment and Workflow of the Radiomics Study
The RQS table is provided in Table 2. The score range of included studies was 13 to 16 (maximum score: 36). None of the evaluated articles performed a phantom study, discussed biological correlates, conducted the study in a prospective design, or performed a cost-effectiveness analysis. The radiomics-based machine learning process workflow was shown in Figure 2.

Quality Assessment and Workflow of the Radiomics Study
The RQS table is provided in Table 2. The score range of included studies was 13 to 16 (maximum score: 36). None of the evaluated articles performed a phantom study, discussed biological correlates, conducted the study in a prospective design, or performed a cost-effectiveness analysis. The radiomics-based machine learning process workflow was shown in Figure 2.

Statistical Analysis
Only six studies were included in the meta-analysis. The pooled sensitivity was 0.885 (95% CI: 0.818-0.929), and the pooled specificity was 0.811 (95% CI: 0.667-0.902). The heterogeneity of sensitivity was low (p = 0.434), whereas the heterogeneity of specificity was high (p = 0.000661). The pooled AUC was 0.906. The forest plots for sensitivity and specificity are provided in Figure 3. The sROC curve is shown in Figure 4.

Statistical Analysis
Only six studies were included in the meta-analysis. The pooled sensitivity was 0.885 (95% CI: 0.818-0.929), and the pooled specificity was 0.811 (95% CI: 0.667-0.902). The heterogeneity of sensitivity was low (p = 0.434), whereas the heterogeneity of specificity was high (p = 0.000661). The pooled AUC was 0.906. The forest plots for sensitivity and specificity are provided in Figure 3. The sROC curve is shown in Figure 4.

Bias Assessment
As shown in Figure 5, the publication bias was assessed using a funnel plot. The funnel plot result was symmetrical, indicating no obvious publication bias in this meta-analysis.

Review of Prediction Feature
According to the International Symposium on Biomedical Imaging (ISBI), radiomics features can be classified as shape-based features, first-order features, gray-level run-length matrix (GLRM) features, gray-level co-occurrence matrix (GLCM) features, gray-level distance-zone matrix (GLDZM) features, gray-level size-zone matrix (GLSZM) features, neighborhood gray tone difference matrix (NGTDM) features, or neighboring gray-level dependence matrix (NGLDM) features [40]. Three studies used shape-based

Bias Assessment
As shown in Figure 5, the publication bias was assessed using a funnel plot. The funnel plot result was symmetrical, indicating no obvious publication bias in this meta-analysis.

Bias Assessment
As shown in Figure 5, the publication bias was assessed using a funnel plot. The funnel plot result was symmetrical, indicating no obvious publication bias in this meta-analysis.

Review of Prediction Feature
According to the International Symposium on Biomedical Imaging (ISBI), radiomics features can be classified as shape-based features, first-order features, gray-level run-length matrix (GLRM) features, gray-level co-occurrence matrix (GLCM) features, gray-level distance-zone matrix (GLDZM) features, gray-level size-zone matrix (GLSZM) features, neighborhood gray tone difference matrix (NGTDM) features, or neighboring gray-level dependence matrix (NGLDM) features [40]. Three studies used shape-based

Review of Prediction Feature
According to the International Symposium on Biomedical Imaging (ISBI), radiomics features can be classified as shape-based features, first-order features, gray-level run-length matrix (GLRM) features, gray-level co-occurrence matrix (GLCM) features, gray-level distance-zone matrix (GLDZM) features, gray-level size-zone matrix (GLSZM) features, neighborhood gray tone difference matrix (NGTDM) features, or neighboring gray-level dependence matrix (NGLDM) features [40]. Three studies used shape-based features, while all studies used first-order and second-order features. The details of used features are listed in Table 3. The number of the studies in which the radiomics type was used was shown in Figure 6. features, while all studies used first-order and second-order features. The details of used features are listed in Table 3. The number of the studies in which the radiomics type was used was shown in Figure 6. First-order, GLCM Lesion distribution, pleural effusion, maximum lesion range, mediastinal and hilar lymph node enlargement, Huang [36] China, 2020 Shape-based, first-order, GLCM, GLDM *, GLSZM, GLRM Halo sign, ground glass opacity (GGO), intralobular interstitial thickening (IIT) Chen [37] China, 2020 Shape-based, first-order, GLSZM Number of lesions with pleural thickening, white blood cell count, platelet count, number of lesions with crazy paving appearance Liu [38] China 2021 first order, GLCM, GLDM*, GLRM age, lesion distribution, neutrophil ratio, CT score, lymphocyte count Note: GLRM, gray-level run-length matrix; GLCM, gray-level co-occurrence matrix; GLDZM, gray-level distance-zone matrix; GLSZM, gray-level size-zone matrix; NGTDM, neighborhood gray tone difference matrix; NGLDM, neighboring gray-level dependence matrix. * The gray-level dependence matrix (GLDM) is not listed by the International Symposium on Biomedical Imaging (ISBI).

Review of Prediction Models
Four studies used least absolute shrinkage and selection operator (LASSO) regression, one study used logistic regression, and one study used support vector machine (SVM) models with a radial basis function kernel. The details of the used models are listed in Table 4.

Review of Prediction Models
Four studies used least absolute shrinkage and selection operator (LASSO) regression, one study used logistic regression, and one study used support vector machine (SVM) models with a radial basis function kernel. The details of the used models are listed in Table 4.

Discussion
This meta-analysis is the first to explore CT-based radiomics features for the differentiation of COVID-19 from other viral pneumonias. The advantage of conducting this meta-analysis is that a large number of patients were included. A total of 10,300 patients were included in this meta-analysis, increasing the robustness of our results. The prediction performance was fascinating with a pooled AUC of 0.906.
Among the studies that included viral pneumonia comparisons, two studies included only influenza infections [33,34]. Other studies included influenza and other viral pneumonia infections [35][36][37][38][39]. Considering that influenza also represents a highly contagious disease with a high prevalence of adult viral pneumonias [41,42]. We included the two studies which compared COVID-19 with influenza pneumonia in this meta-analysis [33,34].
The sensitivity of RT-PCR for COVID-19 diagnosis varied from 59 to 71% depending on viral load and test sample quality [2,5]. That is to say, an RT-PCR negative result was still shown in some COVID-19 infected patients [5,43]. Therefore, chest CT played a crucial role in the early diagnosis of COVID-19 pneumonia for the RT-PCR negative patients [44]. The identification of COVID-19 pneumonia on chest CT depended on radiologists' interpretation. However, radiologists qualified only moderate sensitivity in differentiation COVID-19 from other viral pneumonia on chest CT [14]. With artificial intelligence assistance, radiologists achieved higher sensitivity in diagnosis of COVID-19 pneumonia [3,45]. In this meta-analysis comparing COVID-19 with other viral pneumonia under CT-based radiomics assistance, the pooled sensitivity was 0.885.
The radiomics quality scores in the included studies ranged from 13 to 16 points. The maximum RQS is 36 points. However, none of the collected studies were conducted prospectively, which led to a loss of 7 points. Thus, future studies should be conducted prospectively to achieve better-quality results.
Among the six studies included in the meta-analysis, four of them used LASSO regression. LASSO regression is a commonly used feature selection algorithm in the data science discipline. It is a logistic regression method with L1 regularization, which renders the prediction model more prone to noise, thus increasing its robustness [46]. One study used traditional logistic regression, whereas another used SVM. The SVM algorithm works well in the high-dimensional space, making it popular in machine learning.
First-order features, shape-based features, and second-order features were used in the prediction models. The power of radiomics features was displayed in many cancers [47][48][49]. However, this meta-analysis showed that radiomics features are useful not only in cancer, but also in other diseases, such as COVID-19 pneumonia.
The limitation of this meta-analysis is that all the studies were retrospective and conducted in China. However, there was no other suitable article that met the inclusion criteria in the three databases (Pubmed, Cochrane Library, and Embase) search process by two authors. One selected article was conducted in China but some of the involved patients were collected in the USA [39]. In the future, prospective and multinational studies should be conducted to validate the effectiveness of radiomics in COVID-19 detection using CT scans.

Conclusions
Our meta-analysis showed that CT-based radiomics feature models can successfully be used to differentiate COVID-19 from other viral pneumonias. Informed Consent Statement: Patient consent was waived due to the chosen studies that did not involve patient consent.