Next Article in Journal
Cholesterol-Induced Metabolic Reprogramming in Breast Cancer Cells Is Mediated via the ERRα Pathway
Next Article in Special Issue
Differentiating Glioblastomas from Solitary Brain Metastases: An Update on the Current Literature of Advanced Imaging Modalities
Previous Article in Journal
Surgery after Induction Targeted Therapy and Immunotherapy for Lung Cancer
Previous Article in Special Issue
Use of PET Imaging in Neuro-Oncological Surgery
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Systematic Review

Accuracy of Machine Learning Algorithms for the Classification of Molecular Features of Gliomas on MRI: A Systematic Literature Review and Meta-Analysis

1
Department of Medical Imaging, Radboud University Medical Center, Radboud University, 6500HB Nijmegen, The Netherlands
2
Clinic of Radiology, University Hospital Münster, WWU University of Münster, 48149 Münster, Germany
3
Department of Pathology, Radboud University Medical Center, Radboud University, 6500HB Nijmegen, The Netherlands
4
Department of Neurosurgery, Radboud University Medical Center, Radboud University, 6500HB Nijmegen, The Netherlands
*
Author to whom correspondence should be addressed.
Cancers 2021, 13(11), 2606; https://doi.org/10.3390/cancers13112606
Submission received: 6 April 2021 / Revised: 17 May 2021 / Accepted: 18 May 2021 / Published: 26 May 2021
(This article belongs to the Special Issue Advanced Neuroimaging Approaches for Malignant Brain Tumors)

Abstract

:

Simple Summary

Glioma prognosis and treatment are based on histopathological characteristics and molecular profile. Following the World Health Organization (WHO) guidelines (2016), the most important molecular diagnostic markers include IDH1/2-genotype and 1p/19q codeletion status, although more recent publications also include ARTX genotype and TERT- and MGMT promoter methylation. Machine learning algorithms (MLAs), however, were described to successfully determine these molecular characteristics non-invasively by using magnetic resonance imaging (MRI) data. The aim of this review and meta-analysis was to define the diagnostic accuracy of MLAs with regard to these different molecular markers. We found high accuracies of MLAs to predict each individual molecular marker, with IDH1/2-genotype being the most investigated and the most accurate. Radiogenomics could therefore be a promising tool for discriminating genetically determined gliomas in a non-invasive fashion. Although encouraging results are presented here, large-scale, prospective trials with external validation groups are warranted.

Abstract

Treatment planning and prognosis in glioma treatment are based on the classification into low- and high-grade oligodendroglioma or astrocytoma, which is mainly based on molecular characteristics (IDH1/2- and 1p/19q codeletion status). It would be of great value if this classification could be made reliably before surgery, without biopsy. Machine learning algorithms (MLAs) could play a role in achieving this by enabling glioma characterization on magnetic resonance imaging (MRI) data without invasive tissue sampling. The aim of this study is to provide a performance evaluation and meta-analysis of various MLAs for glioma characterization. Systematic literature search and meta-analysis were performed on the aggregated data, after which subgroup analyses for several target conditions were conducted. This study is registered with PROSPERO, CRD42020191033. We identified 724 studies; 60 and 17 studies were eligible to be included in the systematic review and meta-analysis, respectively. Meta-analysis showed excellent accuracy for all subgroups, with the classification of 1p/19q codeletion status scoring significantly poorer than other subgroups (AUC: 0.748, p = 0.132). There was considerable heterogeneity among some of the included studies. Although promising results were found with regard to the ability of MLA-tools to be used for the non-invasive classification of gliomas, large-scale, prospective trials with external validation are warranted in the future.

Graphical Abstract

1. Introduction

The most common primary brain tumor—glioma—is a rare cancer, but it is invariably fatal despite surgery, chemotherapy, and radiotherapy. While primary central nervous system tumors account for only 2% of primary tumors, they cause 7% of the years of life lost from cancer before age 70 [1,2,3]. Current glioma classification is based on the 2016 World Health Organization (WHO) guidelines, which differentiates subtypes of gliomas based on the presence or absence of isocitrate dehydrogenase (IDH) mutation and 1p/19q codeletion status. In addition to the mutation status, cytologic features and degrees of malignancy after hematoxylin and eosin (H&E) staining are also evaluated (Figure 1). Over the years, various other molecular biomarkers have been reported in the scientific literature, which led the European Association of Neuro-Oncology (EANO) to consider it necessary to update its guideline for the management of adult patients with gliomas [4].
Improved differentiation between the different subtypes of oligodendroglial tumors and astrocytic tumors based on neuroimaging would be beneficial, as this would facilitate the treatment planning, such as the extent of the resection margins and radiotherapy field [5]. Molecular characteristics of glioma have been shown to represent hallmark features that help clinicians to accurately define the nature of the neoplasm. For example, primary glioblastomas are characterized by a distinct pattern of genetic aberrations when compared with secondary glioblastomas, which develop by degeneration of pre-existing lower-grade gliomas [6]. Moreover, molecular characteristics are known to impact the effectiveness of certain treatment options and can therefore help to identify the most suitable treatment strategy for each patient individually [7,8]. Finally, the different subtypes of glioma are known to have different survival rates [9,10]. With regard to prognosis, patients suffering from a grade II glioma with an oligodendroglial origin have a 5-year survival rate of 81%, whereas those suffering from a grade II astrocytic glioma have a 5-year survival rate of 40%. When classified as WHO grade III, oligodendroglial tumors have better 5-year survival rates as compared to astrocytic tumors (43% vs. 20%, respectively). The patients suffering from glioblastoma (grade IV) have the poorest outcomes, with a 5-year survival rate of 5.5% [11]. In terms of treatment, preoperative distinguishing of oligodendroglial tumors from astrocytic tumors would be beneficial in facilitating the planning, extent of the resection, and the radiotherapy field [5]. Unfortunately, no visual features have yet been proven accurately enough to circumvent histopathological assessment after neurosurgical intervention. Application of machine learning algorithms (MLAs), however, could be helpful in the non-invasive characterization of gliomas [12].
As previously predicted, MLAs are increasingly becoming a critical component of advanced software systems in radiology [13,14]. MLAs concern medical imaging analysis carried out by (automatic) feature selection, followed by automatic classification. These processes detect complex patterns in images elusive to the eyes of neuroradiologists and make predictions that surpass human intelligence and human-level performance. In general, input data for MLAs consist of the imaging data themselves (e.g., different MRI sequences) and/or the segmentation of the regions of interest. Output data, on the other hand, are the desired parameters that should be extracted from the imaging data [13,14,15]. In general, the dataset is divided into two different sets: the training and the test set. The training set is used to train the performance of the MLA, indicating that the MLA is attempting to elucidate an often complex relationship between input data and output data. The test set is then used to test the actual performance of the data on a new dataset, indicating that the network has not yet been able to train on these data. The term “test set” is often used interchangeably with “validation” set. Nevertheless, only a small amount of MLAs are actually validated on a completely different, external dataset, which significantly hampers the further development of the integration of MLAs in daily practice [15].
With regard to the use of MLAs in neuro-oncology imaging, various reports on the use of MLAs, using a broad range of extracted features on magnetic resonance imaging (MRI), showed promising results with regard to the prediction of molecular markers and genetic alterations (e.g., IDH genotype, 1p/19q codeletion status, P53 mutations, MGMT promoter mutation, TERT promoter mutation, BRAF status, EGFR receptor mutations) [3,12,16,17,18,19]. However, one of the limitations of this type of research is the relatively limited amount of data in each study, which could possibly be overcome by a systematic review and meta-analysis of the aggregated study results [20]. The purpose of this study was to provide such an overview and perform a meta-analysis of the accuracy of MLAs in predicting gliomas’ genotype.

2. Materials and Methods

2.1. Guidelines and Registration

A systematic review and meta-analysis were conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement [21]. Prior to the initiation of the review, the study protocol was registered in the international open-access Prospective Register of Systematic Reviews (PROSPERO) under the number CRD42020191033.

2.2. Search Strategy, Inclusion Criteria, and Exclusion Criteria

Papers describing the use of MLAs for the binary classification of molecular characteristics of gliomas were reviewed. Databases searched for literature included Medline (accessed through PubMed), EMBASE, and the Cochrane Library. Searches were conducted from 1 April 2020 to 24 January 2021. Search strings are made available in the Supplementary Methods S1 and were included when they discussed the use of classification MLA methodologies on MR images in glioma patients. Next, papers must report results as mean accuracy and/or mean area under the receiver operator characteristics curve (AUC). Papers were excluded from this review when they discussed findings in animal-based studies or in non-human samples. In addition, MLA models needed to be at least internally validated. Letters, preprints, scientific reports, and reviews were included. After the removal of duplicates, the remaining papers were systematically screened on title and abstract by two researchers (E.v.K. and D.H.) independently. Non-consensus papers were identified and discussed by two researchers (E.v.K. and D.H.) to resolve disagreements and to reach consensus. Formal quality assessment tools are still lacking for this type of research [19], although a version of the TRIPOD statement tailored to machine learning methods has been announced [22].
Standardized tables were used to acquire the information of interest from the included articles. Data extracted from each study were (a) first author and year of publication, (b) size of the training set, (c) mean age of participants in the training set, (d) sex of participants in the training set, (e) size of the validation set, (f) whether there was an external validation, (g) study design, (h) architecture of the MLA algorithm(s), (i) target condition, (j) performance of the algorithm(s). Performance of the classification tools was expressed in accuracy, AUC, sensitivity, and specificity for both the training and the validation set. For studies performing external validation, externally validated data were displayed. Extracted data were cross-checked afterward, and discrepancies were resolved.

2.3. Statistical Analysis

Meta-analysis was conducted on the papers, which included the AUC ± standard deviation (SD) using a random-effects model to estimate the performance of the included MLA methodologies. For inclusion in quantitative analysis, studies must have reported a standard deviation, 95% confidence interval (CI), or standard error, along with the AUC-value. For the meta-analysis, the standard deviation was derived from the standard error or 95%-CI for studies not reporting the standard deviation [23]. If not provided, corresponding authors were contacted with the request to provide the necessary data to be included in the meta-analysis. Results of all appropriate studies were combined to meta-analyze the aggregated data. Then, meta-analyses were conducted on different subgroups of target conditions in order to estimate the accuracy of the algorithm for each condition separately. To be included in subgroup analysis, an additional criterium for the included studies was to describe a specific target condition (e.g., IDH and 1p/19q). Meta-analysis was performed with the use of IBM SPSS Statistics (IBM Corp. Released 2017. IBM SPSS Statistics for Windows, Version 25.0. Armonk, NY: IBM Corp.) and OpenMeta[Analyst] software (MetaAnalyst, Brown University EPBC [24]), which is the visual front-end for the R package (version 12.11.14) [25]. Results were displayed in forest plots. AUC-values of subgroups were compared by looking at the 95% confidence intervals and whether there was any overlap. The Higgins I2-test was used to test for heterogeneity between included studies with I2 > 75% deemed as considerable heterogeneity [23]. The Egger regression analysis was carried out to test for publication bias [26].

3. Results

A total of 1094 publications were initially retrieved through literature searches, and 724 remained after the removal of duplicates. After title and abstract screening, 215 full-text articles were assessed for eligibility. After a full-text assessment, 60 and 17 studies were included for the systematic review and meta-analysis, respectively (Figure 2). A total of 155 studies were excluded based on the full-text assessment for the following reasons: 66 studies described the segmentation of tumor-area instead of the classification of glioma; 23 studies did not report AUC as a performance metric; 14 studies discussed the classification of texture features instead of molecular characteristics of glioma; 14 studies had an incomplete description of the published data and/or methods; 14 studies did not describe an MLA model, but instead a number of combined features; 13 studies reported a too specific target condition (e.g., H3K27M and Ki-67); 5 studies used imaging techniques other than MR imaging; 3 studies had no internal validation-group; 3 studies included other brain tumors besides glioma.

3.1. Review of the Included Studies

A total of 60 studies [16,17,18,19,20,21,22,23,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74] were included after the eligibility assessment. Table 1 shows the participant demographics and study characteristics. The performance evaluation of the MLA algorithms in terms of accuracy, AUC, sensitivity, and specificity for the validation set is displayed in Table 1, organized by target condition subgroups. For 12 of the included studies, an external out-of-sample validation was carried out [23,35,36,39,40,41,46,48,50,64,67,74], all other studies performed internal validation only.
Due to the high dimensionality and complexity of MRI data using different sequences, two critical steps were used to reduce the computational power needed to carry out such complex analyses. Feature selection was one key step in the discovery of predictors from high-throughput features. The included imaging genomics studies built their classification models by selecting a set of non-redundant features. The second step consists of applying a dimensionality reduction method. Often, unimodal methods are used for each dataset separately, thus failing to properly extract important, though subtle, interactions between various sequences. The least absolute shrinkage and selection operator (Lasso) method was discussed most often to perform dimensionality reduction [27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45]. With regard to feature selection, most papers discussed the use of a support vector machine-based recursive feature elimination (SVM-RFE) method [28,32,33,34,35,36,37,42,43,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67]. The Lasso method was another often mentioned method to select features [27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45]. Other methods, which were frequently used for feature selection, concerned random forest classification algorithms [50,51,53,63,66,68,69,70,71,72,73,74], convolutional neural networks [37,46,48,56,57,65,75,76,77,78,79,80,81], and/or logistic regression models [33,57,82,83].
For the target condition, 22 studies focused on IDH mutation status, 6 on 1p/19q codeletion status, 11 on MGMT promoter methylation status, 3 on TP53 mutation status, 1 on the PTEN gene mutation, 2 on ATRX gene mutation status, 3 on the TERT promoter mutation, and 20 on the differentiation between different subtypes of glioma. Five of the included studies described multiple target conditions and/or multiple tested MLA models, each of those presented separately in Table 1. The included classification studies showed different references in classifying the presence or absence of the outcome of interest, such as standard-of-care diagnosis, (immuno)histopathology, and expert consensus. Twenty-two studies [17,18,19,23,25,28,32,33,37,39,43,45,49,51,56,61,63,66,69,70,71,74] focused on classifying the IDH mutation status in glioma. Mutations of the IDH genes serve as a diagnostic marker to diffuse WHO grade II and III gliomas as well as secondary glioblastomas. They are associated with a better prognosis in these gliomas [7]. One included study described two separate MLA methods, and therefore 23 methodologies could be analyzed. All studies used retrospectively collected data, and three [23,39,74] carried out external validation. Five studies did not report a validated AUC-value. Performance evaluation from the remaining 17 [19,25,28,32,37,43,45,49,51,56,61,63,66,69,70,71,74] studies in terms of the validated AUC ranged from 0.75 to 0.99. Additionally, 14 studies [18,19,27,32,33,37,43,45,51,56,61,63,69,71] presented the sensitivity and the specificity ranging from 54% to 98% and 67% to 99%, respectively.
Six studies [20,32,56,57,60,64] described the classification of the 1p/19q codeletion status in glioma patients, all using retrospectively collected data. The 1p/19q codeletion status is associated with better prognosis in patients with (oligodendro)glial tumors receiving adjuvant radio-chemotherapy [7]. One study [64] included an external validation. Validated AUC for classifying 1p/19q codeletion status ranged from 0.72 to 0.87 (n = 5) [32,56,57,60,64]. The range of the sensitivity and the specificity were 68% to 92% and 71% to 85%, respectively (n = 5) [32,56,57,60,64].
Regarding the classification of the MGMT promoter methylation status, all studies (n = 11) [31,35,44,51,52,53,54,55,56,65,67] used retrospectively collected data and two studies [35,67] had been externally validated. The importance of the MGMT promoter methylation status can be found in the fact that it is a predictive variable for the response of GBM to alkylating chemotherapy [7]. Seven studies [31,35,44,51,52,56,65] reported the validated AUC-value, which ranged from 0.54 to 0.90. Moreover, the sensitivity and specificity ranged from 67% to 94% and 54% to 97%, respectively (n = 7) [35,51,52,55,56,65,67].
Three studies [36,56,62] were included that described the classification of the TERT promoter mutation status in glioma. All studies used retrospectively collected data, and one study [36] was externally validated. Although still under investigation, it has been suggested that TERT promoter mutations characterize gliomas that require aggressive treatment [8]. All studies reported the validated AUC (range: 0.82–0.89). Additionally, the sensitivity and specificity ranged from 71% to 77% and 86% to 91%, respectively (n = 3).

3.2. Meta-Analysis of the Included Studies

For the classification papers, 17 out of the 60 were eligible for the meta-analysis. In total, 22 MLA methodologies, described in 17 individual studies [19,22,32,34,35,36,37,49,50,56,61,62,64,65,66,73,74] were retrieved. Meta-analysis was performed separately in subgroups for different target conditions. For inclusion in this subgroup meta-analysis, studies must focus on a specific target condition (i.e., IDH genotype, 1p/19q codeletion and MGMT- and TERT promoter status).
For the subgroup meta-analysis of the classification studies with a focus on IDH mutation status, eight MLA algorithms, originating from seven studies [19,37,49,56,61,66,74], were included. Results show an overall AUC of 0.909 (95%-CI: 0.867–0.951), as seen in Figure 3. Moreover, heterogeneity between groups, measured with Higgins I2, was estimated as 90.402% (p < 0.001).
The forest plot shows that the performance of the MLAs to classify molecular characteristics of glioma are centered around an AUC of 0.858 with a 95%-CI ranging from 0.812–0.904.
Three studies [32,56,64] were included in the subgroup meta-analysis of the 1p/19q codeletion status. Results of this subgroup analysis are displayed in Figure 4. The overall AUC is 0.748 (95%-CI: 0.699–0.797). Heterogeneity between groups was considered moderate (Higgins I2= 50.655% (p = 0.132)).
Subgroup meta-analysis of MGMT promoter methylation status included three studies [35,56,65]; see Figure 5 for an overview of the results. The overall AUC of these MLA models was estimated as 0.866 (95%-CI: 0.812–0.921). Heterogeneity between the included studies was considered very low (Higgins I2= 0% (p = 0.453)).
Three studies [36,56,62] were included in the subgroup analysis of the TERT promoter mutation status. Figure 6 displays the results of the meta-analysis with an estimated overall AUC of 0.842 (95%-CI: 0.783–0.901) and considered a low I2 heterogeneity of 0% (p = 0.582).
Classification of the 1p/19q codeletion status showed to have a significantly poorer AUC when compared to other subgroup classifications, except for the TERT mutation status classification. No significant differences in performance between the other three subgroups (i.e., IDH, MGMT, TERT) were observed due to overlap of the 95% confidence intervals.

3.3. Testing for Publication Bias

Egger’s regression test showed no significant publication bias with regard to MLAs to predict the molecular status of gliomas (p = 0.235).

4. Discussion

In this study, a number of studies that describe the classification of gliomas with the use of MLAs were reviewed and meta-analyzed. The overall performance of the classification tools as reported in AUC-values showed to be excellent. Subgroup analysis showed that the classification of 1p/19q codeletion status was significantly poorer than the classification performance of other molecular markers (i.e., IDH and MGMT). The observed heterogeneity between the included studies in the IDH-subgroup was considerably high.
The binary classification of various molecular characteristics of glioma with the use of artificial intelligence showed promising results with regard to future implementation in clinical practice. This implementation will have a significant impact on the care of glioma patients, as it could help to stratify patients for treatment options prior to undergoing surgery. However, clinically relevant studies need to be undertaken to increase the impact of these techniques in daily practice. For example, predicting 1p/19q codeletion status is more relevant in a subset of low-grade gliomas, as it enables to non-invasively distinguish IDH-mut astrocytoma (1p/19q intact) from oligodendroglioma (1p/19q codeleted) [62]. This is clinically relevant as the median survival of patients with these glioma subtypes is significantly different and can be impacted by the extent of the neurosurgical resection [86,87]. In addition, to further verify the performance of MLA methodologies, larger-scale multi-center studies using prospective data are required. Moreover, despite the growing knowledge and use of MLA methodologies, integration in widespread clinical practice still faces some challenges [12]. One major challenge for this implementation concerns the generalizability of these systems, as they are mostly trained on small datasets lacking external validation [88]. Considering external validation as an additional inclusion criterium for a sub-analysis, we found that no meta-analysis could be performed. The twelve papers included in this review which validated their results externally investigated IDH mutation status (n = 2), 1p/19q codeletion status (n = 1), MGMT promoter methylation status (n = 2), PTEN gene mutation (n = 1), ATRX gene mutation (n = 1), TERT promoter mutation (n = 1), and various predictions with regard to WHO grading (n = 4), indicating that no pooled data can be acquired from these individual studies. Although the computer-aided classification of glioma holds great potential, computer-obtained diagnosis is not likely to replace histopathologic diagnosis in the near future.

4.1. Implementation of Computer-Aided Approaches in Future Medicine

The diagnosis of different diseases by the use of MLAs is believed to hold great potential in modern medicine. The number of retrieved papers on this narrow topic is relatively limited, especially when compared to other reviews with a broader scope with regard to the use of artificial analysis in medical imaging analysis [22]. Nevertheless, conclusions and major limitations seem to be similar across fields. We can cautiously state that the accuracy of MLAs in the non-invasive classification of glioma holds great potential and is equal to or better than the predictions of healthcare professionals. On the other hand, the lack of external validation of the obtained results was recognized as the major limitation of the current scientific literature. Additionally, poor reporting is known to be prevalent in MLA studies, which limits reliable interpretation of the reported diagnostic accuracy and thereby hampers clinical implementation. Improving reporting and publication will enable greater confidence in the results of future evaluations of these promising technologies in medicine. When such improved confidence will be achieved, prospective evaluation should be carried out in the context of an intended clinical pathway. With regard to the context discussed in this paper, MLAs could be used on the preoperative imaging data, after which the predicted outcomes can be compared to the histopathological assessment after biopsy. Such implementation will help to elucidate whether important unknown covariates were present in the retrospective studies reviewed here. Thereafter, a randomized comparison could help to reveal and quantify possible clinical implications of implementing these MLAs in daily practice.
Furthermore, as recently suggested by Bhandari et al., a greater effort is needed to start translating these findings into an interpretable format for clinical radiology [89]. In addition, as several studies focused on singular molecular biomarkers in gliomas, it must be underlined several molecular alterations in astrocytic, oligodendroglial gliomas can occur in different combinations [79]. Consequently, a growing number of (and a combination of) different molecular tests are used to provide clinically relevant tissue-based biomarkers. Furthermore, the performance of MLAs to grade glioma according to the WHO grading score remains restricted by the interobserver variability of the neuropathological examination as reviewed by Van den Bent (2010) [90]. This clinically significant interobserver variation of the histological grading of glioma limits the diagnostic performance of other diagnostic tests (e.g., MLAs) as the neuropathological assessment was considered to be the ground truth.

4.2. Clinical Relevance of Computer-Aided Diagnosis

Automated diagnosis from medical imaging through artificial intelligence could help to overcome the mismatch between the increasing amount of diagnostic images and the capacity of available specialists [91]. More than 100 MLAs have now CE-marked, 57 of which can be used within neuroimaging features. Only four of these MLAs have been tailored to be used on neuro-oncology practices [91] (see https://grand-challenge.org/aiforradiology/; accessed on 5 March 2021). Only one of these software packages claims to aid in tumor differentiation (The Brain Tumours Application; Hanalytics (BioMind; https://biomind.ai/; accessed on 5 March 2021). This application focuses on the differentiation of 22 types of intracranial tumors on MRI scans, including the differentiation of astrocytoma, oligodendroglioma, and glioblastoma. Therefore, no software is commercially present to distinguish different molecular subtypes of glioma. Therefore, we conclude that the use of MLA models in daily radiological practice to non-invasively predict glioma subtype remains an important topic of future research in order to improve accuracy and commence external validation [91].

4.3. Strengths and Limitations

Meta-analysis of the aggregated MLA models showed high heterogeneity between included study groups. This heterogeneity could be expected, as multiple subgroups of target conditions are included in this analysis. Subgroup meta-analyses show significantly lower heterogeneity among included groups. However, for the IDH-subgroup, estimated heterogeneity still is remarkably high (I2 > 80%). Possible explanations for this heterogeneity could be the inclusion of multiple technically different MLA methodologies, multiple included MRI protocols with different sequences (e.g., T1-weighted, T2-weighted, diffusion-weighted), and the fact that there were no specific criteria set for the target population of glioma patients. The latter indicates that there is a good chance of variety between the included groups of patients. Although not supported by the results of the Egger’s regression test, the presence of publication bias is not unlikely, there being little interest for classification tools with poor performance. The analyzed MLA methodologies showed excellent accuracy in the classification of multiple molecular characteristics of glioma. However, some deficiencies in the methods of this study should be considered. The quality of included articles was not formally assessed because no sufficient assessment tool is currently available for prediction models using MLA-techniques. The previously announced MLA variant of the TRIPOD statement could possibly be the solution to this problem for future research [22]. Moreover, external validation of the MLA methodologies was conducted for only 12 of the 60 studies. As internal validation commonly overestimates the performance, external validation of the described system is highly preferred. Lastly, a large number of the studies found in the initial search had missing reports of the validated AUC-value. Therefore, plural studies needed to be excluded from this review. Moreover, out of 60 studies included for qualitative synthesis, 43 studies did not note the 95% confidence interval, standard error, and/or standard deviation, which led to a restriction in the number of studies eligible for meta-analysis. For this matter, we recommend a standardized way of reporting MLA findings. In addition, as MLAs are statistical model-fitting methods, the reliability and performance of each study are at least partially dependent on the study sample size. Therefore, a large dataset next to the available BraTS dataset would be indispensable for this field of research.

5. Conclusions

This systematic review and meta-analysis show good accuracy for various MLA methodologies for the classification of molecular characteristics of gliomas, which could be beneficial for treatment planning. Remarkably, various studies did not perform external validation, causing significant limitations for these study results. Quality guidelines should be used when publishing studies on MLAs, including out-of-sample external validation and standardized reporting of obtained results.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/cancers13112606/s1. Methods S1: The search strategy for (a) Medline (accessed through PubMed), (b) EMBASE, (c) The Cochrane Library is displayed here.

Author Contributions

Conceptualization, E.J.v.K. and D.J.H.A.H.; methodology, E.J.v.K., M.P., M.M., and D.J.H.A.H.; software, E.J.v.K., M.P., M.M. and D.J.H.A.H.; validation, M.M., B.K., M.t.L., F.J.A.M., and D.J.H.A.H.; formal analysis, E.J.v.K. and D.J.H.A.H.; investigation, E.J.v.K. and M.P.; resources, D.J.H.A.H.; data curation, E.J.v.K.; writing—original draft preparation, E.J.v.K.; writing—review and editing, E.J.v.K., M.P., and D.J.H.A.H.; visualization, E.J.v.K. and D.J.H.A.H.; supervision, M.M., B.K., M.t.L., F.J.A.M., and D.J.H.A.H.; project administration, E.J.v.K. and D.J.H.A.H.; funding acquisition, F.J.A.M. and D.J.H.A.H. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding. The APC was funded by The European Union INTERREG VA DEUTSCHLAND-NEDERLAND, grant number 203020/2020P18.

Conflicts of Interest

The funders had no role in the design of the study, in the collection, analyses, or interpretation of data, in the writing of the manuscript, or in the decision to publish the results.

References

  1. Koshy, M.; Villano, J.L.; Dolecek, T.A.; Howard, A.; Mahmood, U.; Chmura, S.J.; Weichselbaum, R.R.; McCarthy, B.J. Improved survival time trends for glioblastoma using the SEER 17 population-based registries. J. Neurooncol. 2012, 107, 207–212. [Google Scholar] [CrossRef] [Green Version]
  2. Ostrom, Q.T.; Gittleman, H.; Truitt, G.; Boscia, A.; Kruchko, C.; Barnholtz-Sloan, J.S. CBTRUS Statistical Report: Primary Brain and Other Central Nervous System Tumors Diagnosed in the United States in 2011–2015. Neuro Oncol. 2018, 20, iv1–iv86. [Google Scholar] [CrossRef] [Green Version]
  3. Abd-Ellah, M.K.; Awad, A.I.; Khalaf, A.A.M.; Hamed, H.F.A. A review on brain tumor diagnosis from MRI images: Practical implications, key achievements, and lessons learned. Magn. Reson. Imaging 2019, 61, 300–318. [Google Scholar] [CrossRef] [PubMed]
  4. Weller, M.; van den Bent, M.; Preusser, M.; Le Rhun, E.; Tonn, J.C.; Minniti, G.; Bendszus, M.; Balana, C.; Chinot, O.; Dirven, L.; et al. EANO guidelines on the diagnosis and treatment of diffuse gliomas of adulthood. Nat. Rev. Clin. Oncol. 2021, 18, 170–186. [Google Scholar] [CrossRef] [PubMed]
  5. Lecavalier-Barsoum, M.; Quon, H.; Abdulkarim, B. Adjuvant treatment of anaplastic oligodendrogliomas and oligoastrocytomas. Cochrane Database Syst. Rev. 2014, 2014, CD007104. [Google Scholar] [CrossRef] [PubMed]
  6. Ohgaki, H.; Kleihues, P. Genetic pathways to primary and secondary glioblastoma. Am. J. Pathol. 2007, 170, 1445–1453. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  7. Riemenschneider, M.J.; Jeuken, J.W.; Wesseling, P.; Reifenberger, G. Molecular diagnostics of gliomas: State of the art. Acta Neuropathol. 2010, 120, 567–584. [Google Scholar] [CrossRef] [Green Version]
  8. Simon, M.; Hosen, I.; Gousias, K.; Rachakonda, S.; Heidenreich, B.; Gessi, M.; Schramm, J.; Hemminki, K.; Waha, A.; Kumar, R. TERT promoter mutations: A novel independent prognostic factor in primary glioblastomas. Neuro Oncol. 2015, 17, 45–52. [Google Scholar] [CrossRef] [PubMed]
  9. Narang, J.; Jain, R.; Scarpace, L.; Saksena, S.; Schultz, L.R.; Rock, J.P.; Rosenblum, M.; Patel, S.C.; Mikkelsen, T. Tumor vascular leakiness and blood volume estimates in oligodendrogliomas using perfusion CT: An analysis of perfusion parameters helping further characterize genetic subtypes as well as differentiate from astroglial tumors. J. Neuro Oncol. 2011, 102, 287–293. [Google Scholar] [CrossRef]
  10. Saito, T.; Yamasaki, F.; Kajiwara, Y.; Abe, N.; Akiyama, Y.; Kakuda, T.; Takeshima, Y.; Sugiyama, K.; Okada, Y.; Kurisu, K. Role of perfusion-weighted imaging at 3 T in the histopathological differentiation between astrocytic and oligodendroglial tumors. Eur. J. Radiol. 2012, 81, 1863–1869. [Google Scholar] [CrossRef]
  11. Lapointe, S.; Perry, A.; Butowski, N.A. Primary brain tumours in adults. Lancet 2018, 392, 432–446. [Google Scholar] [CrossRef]
  12. Rudie, J.D.; Rauschecker, A.M.; Bryan, R.N.; Davatzikos, C.; Mohan, S. Emerging Applications of Artificial Intelligence in Neuro-Oncology. Radiology 2019, 290, 607–618. [Google Scholar] [CrossRef] [PubMed]
  13. Wang, G.; Kalra, M.; Orton, C.G. Machine learning will transform radiology significantly within the next 5 years. Med. Phys. 2017, 44, 2041–2044. [Google Scholar] [CrossRef] [Green Version]
  14. Wang, S.; Summers, R.M. Machine learning and radiology. Med. Image Anal. 2012, 16, 933–951. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  15. Six, O. The Ultimate Guide to AI in Radiology. Available online: https://www.quantib.com/the-ultimate-guide-to-ai-in-radiology (accessed on 5 July 2020).
  16. Kononenko, I. Machine learning for medical diagnosis: History, state of the art and perspective. Artif. Intell. Med. 2001, 23, 89–109. [Google Scholar] [CrossRef] [Green Version]
  17. Nguyen, A.V.; Blears, E.E.; Ross, E.; Lall, R.R.; Ortega-Barnett, J. Machine learning applications for the differentiation of primary central nervous system lymphoma from glioblastoma on imaging: A systematic review and meta-analysis. Neurosurg. Focus 2018, 45, E5. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Zhao, J.; Huang, Y.; Song, Y.; Xie, D.; Hu, M.; Qiu, H.; Chu, J. Diagnostic accuracy and potential covariates for machine learning to identify IDH mutations in glioma patients: Evidence from a meta-analysis. Eur. Radiol. 2020, 30, 4664–4674. [Google Scholar] [CrossRef] [PubMed]
  19. Liu, X.X.; Faes, L.; Kale, A.U.; Wagner, S.K.; Fu, D.J.; Bruynseels, A.; Mahendiran, T.; Moraes, G.; Shamdas, M.; Kern, C.; et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: A systematic review and meta-analysis. Lancet Digit. Health 2019, 1, E271–E297. [Google Scholar] [CrossRef]
  20. Sakai, K.; Yamada, K. Machine learning studies on major brain diseases: 5-year trends of 2014–2018. Jpn. J. Radiol. 2019, 37, 34–72. [Google Scholar] [CrossRef]
  21. Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G.; Group, P. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Int. J. Surg. 2010, 8, 336–341. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  22. Collins, G.S.; Moons, K.G.M. Reporting of artificial intelligence prediction models. Lancet 2019, 393, 1577–1579. [Google Scholar] [CrossRef]
  23. Higgins, J.P.T.; Green, S. (Eds.) Cochrane Handbook for Systematic Reviews of Interventions, version 5.1.0; The Cochrane Collaboration: London, UK, 2011. [Google Scholar]
  24. Wallace, B.C.; Dahabreh, I.J.; Trikalinos, T.A.; Lau, J.; Trow, P.; Schmid, C.H. Closing the Gap between Methodologists and End-Users: R as a Computational Back-End. J. Stat. Softw. 2012, 49, 1–15. [Google Scholar] [CrossRef] [Green Version]
  25. Viechtbauer, W. Conducting Meta-Analyses in R with the metafor Package. J. Stat. Softw. 2010, 1. [Google Scholar] [CrossRef] [Green Version]
  26. Egger, M.; Davey Smith, G.; Schneider, M.; Minder, C. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997, 315, 629–634. [Google Scholar] [CrossRef] [Green Version]
  27. Arita, H.; Kinoshita, M.; Kawaguchi, A.; Takahashi, M.; Narita, Y.; Terakawa, Y.; Tsuyuguchi, N.; Okita, Y.; Nonaka, M.; Moriuchi, S.; et al. Lesion location implemented magnetic resonance imaging radiomics for predicting IDH and TERT promoter mutations in grade II/III gliomas. Sci. Rep. 2018, 8, 11773. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  28. Cao, H.; Erson-Omay, E.Z.; Li, X.; Gunel, M.; Moliterno, J.; Fulbright, R.K. A quantitative model based on clinically relevant MRI features differentiates lower grade gliomas and glioblastoma. Eur. Radiol. 2020, 30, 3073–3082. [Google Scholar] [CrossRef] [PubMed]
  29. Carver, E.; Snyder, J.; Griffith, B.; Wen, N. Quantitative image feature analysis in diffuse glioma—A valuable MR imaging biomarker for preoperative idh mutation classification. Neuro Oncol. 2019, 21, vi61. [Google Scholar] [CrossRef]
  30. Cui, G.; Jeong, J.; Lei, Y.; Wang, T.; Dong, X.; Liu, T.; Curran, W.; Mao, H.; Yang, X. Machine-learning-based classification of low-grade and high-grade glioblastoma using radiomic features in multiparametric MRI. Med. Phys. 2018, 45, e617. [Google Scholar] [CrossRef]
  31. Li, Y.M.; Liu, X.; Qian, Z.H.; Sun, Z.Y.; Xu, K.B.; Wang, K.; Fan, X.; Zhang, Z.; Li, S.W.; Wang, Y.Y.; et al. Genotype prediction of ATRX mutation in lower-grade gliomas using an MRI radiomics signature. Eur. Radiol. 2018, 28, 2960–2968. [Google Scholar] [CrossRef]
  32. Li, Y.M.; Qian, Z.H.; Xu, K.B.; Wang, K.; Fan, X.; Li, S.W.; Jiang, T.; Liu, X.; Wang, Y.Y. MRI features predict p53 status in lower-grade gliomas via a machine-learning approach. Neuroimage Clin. 2018, 17, 306–311. [Google Scholar] [CrossRef]
  33. Hwan-Ho, C.; Hyunjin, P. Classification of low-grade and high-grade glioma using multi-modal image radiomics features. In Proceedings of the 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Jeju, Korea, 11–15 July 2017; pp. 3081–3084. [Google Scholar] [CrossRef]
  34. Jiang, C.; Kong, Z.; Liu, S.; Feng, S.; Zhang, Y.; Zhu, R.; Chen, W.; Wang, Y.; Lyu, Y.; You, H.; et al. Fusion Radiomics Features from Conventional MRI Predict MGMT Promoter Methylation Status in Lower Grade Gliomas. Eur. J. Radiol. 2019, 121, 108714. [Google Scholar] [CrossRef] [PubMed]
  35. Jiang, C.; Kong, Z.; Zhang, Y.; Liu, S.; Liu, Z.; Chen, W.; Liu, P.; Liu, D.; Wang, Y.; Lyu, Y.; et al. Conventional magnetic resonance imaging-based radiomic signature predicts telomerase reverse transcriptase promoter mutation status in grade II and III gliomas. Neuroradiology 2020, 62, 803–813. [Google Scholar] [CrossRef]
  36. Matsui, Y.; Maruyama, T.; Nitta, M.; Saito, T.; Tsuzuki, S.; Tamura, M.; Kusuda, K.; Fukuya, Y.; Asano, H.; Kawamata, T.; et al. Prediction of lower-grade glioma molecular subtypes using deep learning. J. Neurooncol. 2020, 146, 321–327. [Google Scholar] [CrossRef]
  37. Mzoughi, H.; Njeh, I.; Wali, A.; Slima, M.B.; BenHamida, A.; Mhiri, C.; Mahfoudhe, K.B. Deep Multi-Scale 3D Convolutional Neural Network (CNN) for MRI Gliomas Brain Tumor Classification. J. Digit. Imaging 2020, 33, 903–915. [Google Scholar] [CrossRef]
  38. Sasaki, T.; Kinoshita, M.; Fujita, K.; Fukai, J.; Hayashi, N.; Uematsu, Y.; Okita, Y.; Nonaka, M.; Moriuchi, S.; Uda, T.; et al. Radiomics and MGMT promoter methylation for prognostication of newly diagnosed glioblastoma. Sci. Rep. 2019, 9, 14435. [Google Scholar] [CrossRef] [PubMed]
  39. Sasaki, T.; Kinoshita, M.; Fujita, K.; Arita, H.; Uda, T.; Tsuyuguchi, H.; Hayashi, N.; Fukai, J.; Uematsu, Y.; Mori, K.; et al. Radiomics of glioblastoma for predicting mgmt promotor methylation status and prognosis. Neuro Oncol. 2018, 20, vi192. [Google Scholar] [CrossRef]
  40. Shboul, Z.A.; Chen, J.; Iftekharuddin, K.M. Prediction of Molecular Mutations in Diffuse Low-Grade Gliomas using MR Imaging Features. Sci. Rep. 2020, 10, 3711. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  41. Sun, X.; Pang, P.; Lou, L.; Feng, Q.; Ding, Z.; Zhou, J. Radiomic prediction models for the level of Ki-67 and p53 in glioma. J. Int. Med. Res. 2020, 48, 300060520914466. [Google Scholar] [CrossRef] [PubMed]
  42. Tan, Y.; Zhang, S.T.; Wei, J.W.; Dong, D.; Wang, X.C.; Yang, G.Q.; Tian, J.; Zhang, H. A radiomics nomogram may improve the prediction of IDH genotype for astrocytoma before surgery. Eur. Radiol. 2019, 29, 3325–3337. [Google Scholar] [CrossRef]
  43. Tian, H.; Wu, H.; Wu, G.; Xu, G. Noninvasive Prediction of TERT Promoter Mutations in High-Grade Glioma by Radiomics Analysis Based on Multiparameter MRI. BioMed Res. Int. 2020, 2020, 3872314. [Google Scholar] [CrossRef]
  44. Xi, Y.B.; Guo, F.; Xu, Z.L.; Li, C.; Wei, W.; Tian, P.; Liu, T.T.; Liu, L.; Chen, G.; Ye, J.; et al. Radiomics signature: A potential biomarker for the prediction of MGMT promoter methylation in glioblastoma. J. Magn. Reson. Imaging 2018, 47, 1380–1387. [Google Scholar] [CrossRef]
  45. Fan, Y.; Chen, C.; Zhao, F.; Tian, Z.; Wang, J.; Ma, X.; Xu, J. Radiomics-Based Machine Learning Technology Enables Better Differentiation Between Glioblastoma and Anaplastic Oligodendroglioma. Front. Oncol. 2019, 9, 1164. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  46. Ahammed Muneer, K.V.; Rajendran, V.R. Glioma Tumor Grade Identification Using Artificial Intelligent Techniques. J. Med. Syst. 2019, 43, 113. [Google Scholar] [CrossRef] [PubMed]
  47. Bakas, S.; Rathore, S.; Nasrallah, M.; Akbari, H.; Binder, Z.; Ha, S.M.; Mamourian, E.; Morrissette, J.; O’Rourke, D.; Davatzikos, C. Non-invasive in vivo signature of IDH1 mutational status in high grade glioma, from clinically-acquired multi-parametric magnetic resonance imaging, using multivariate machine learning. Neuro Oncol. 2018, 20, vi184–vi185. [Google Scholar] [CrossRef] [Green Version]
  48. Bangalore Yogananda, C.G.; Shah, B.R.; Vejdani-Jahromi, M.; Nalawade, S.S.; Murugesan, G.K.; Yu, F.F.; Pinho, M.C.; Wagner, B.C.; Mickey, B.; Patel, T.R.; et al. A novel fully automated MRI-based deep-learning method for classification of IDH mutation status in brain gliomas. Neuro Oncol. 2020, 22, 402–411. [Google Scholar] [CrossRef] [PubMed]
  49. Citak-Er, F.; Firat, Z.; Kovanlikaya, I.; Ture, U.; Ozturk-Isik, E. Machine-learning in grading of gliomas based on multi-parametric magnetic resonance imaging at 3T. Comput. Biol. Med. 2018, 99, 154–160. [Google Scholar] [CrossRef]
  50. De Looze, C.; Beausang, A.; Cryan, J.; Loftus, T.; Buckley, P.G.; Farrell, M.; Looby, S.; Reilly, R.; Brett, F.; Kearney, H. Machine learning: A useful radiological adjunct in determination of a newly diagnosed glioma’s grade and IDH status. J. Neurooncol. 2018, 139, 491–499. [Google Scholar] [CrossRef]
  51. Gates, E.D.H.; Lin, J.S.; Weinberg, J.S.; Prabhu, S.S.; Hamilton, J.; Hazle, J.D.; Fuller, G.N.; Baladandayuthapani, V.; Fuentes, D.T.; Schellingerhout, D. Imaging-Based Algorithm for the Local Grading of Glioma. AJNR Am. J. Neuroradiol. 2020, 41, 400–407. [Google Scholar] [CrossRef]
  52. Inano, R.; Oishi, N.; Kunieda, T.; Arakawa, Y.; Yamao, Y.; Shibata, S.; Kikuchi, T.; Fukuyama, H.; Miyamoto, S. Voxel-based clustered imaging by multiparameter diffusion tensor images for glioma grading. Neuroimage Clin. 2014, 5, 396–407. [Google Scholar] [CrossRef] [Green Version]
  53. Kim, M.; Jung, S.Y.; Park, J.E.; Jo, Y.; Park, S.Y.; Nam, S.J.; Kim, J.H.; Kim, H.S. Diffusion- and perfusion-weighted MRI radiomics model may predict isocitrate dehydrogenase (IDH) mutation and tumor aggressiveness in diffuse lower grade glioma. Eur. Radiol. 2020, 30, 2142–2151. [Google Scholar] [CrossRef]
  54. Lee, M.H.; Kim, J.; Kim, S.T.; Shin, H.M.; You, H.J.; Choi, J.W.; Seol, H.J.; Nam, D.H.; Lee, J.I.; Kong, D.S. Prediction of IDH1 Mutation Status in Glioblastoma Using Machine Learning Technique Based on Quantitative Radiomic Data. World Neurosurg. 2019, 125, e688–e696. [Google Scholar] [CrossRef]
  55. Li, Y.; Liang, Y.; Sun, Z.; Xu, K.; Fan, X.; Li, S.; Zhang, Z.; Jiang, T.; Liu, X.; Wang, Y. Radiogenomic analysis of PTEN mutation in glioblastoma using preoperative multi-parametric magnetic resonance imaging. Neuroradiology 2019, 61, 1229–1237. [Google Scholar] [CrossRef] [PubMed]
  56. Li, Z.; Wang, Y.; Yu, J.; Guo, Y.; Cao, W. Deep Learning based Radiomics (DLR) and its usage in noninvasive IDH1 prediction for low grade glioma. Sci. Rep. 2017, 7, 5467. [Google Scholar] [CrossRef] [PubMed]
  57. Lo, C.M.; Weng, R.C.; Cheng, S.J.; Wang, H.J.; Hsieh, K.L. Computer-aided diagnosis of isocitrate dehydrogenase genotypes in glioblastomas from radiomic patterns. Medicine 2020, 99, e19123. [Google Scholar] [CrossRef] [PubMed]
  58. Lu, C.F.; Hsu, F.T.; Hsieh, K.L.; Kao, Y.J.; Cheng, S.J.; Hsu, J.B.; Tsai, P.H.; Chen, R.J.; Huang, C.C.; Yen, Y.; et al. Machine Learning-Based Radiomics for Molecular Subtyping of Gliomas. Clin. Cancer Res. 2018, 24, 4429–4436. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  59. Rathore, S.; MacLean, N.; Akbari, H.; Shukla, G.; Bagley, S. TMOD-40. In Vivo Evaluation of O6-Methylguanine-DNA-Methyltransferase (MGMT) Promoter Methylation Status for De Novo Glioblastoma Patients Using Deep Learning Features. Neuro Oncol. 2019, 21, vi271–vi272. [Google Scholar] [CrossRef]
  60. Shofty, B.; Artzi, M.; Ben Bashat, D.; Liberman, G.; Haim, O.; Kashanian, A.; Bokstein, F.; Blumenthal, D.T.; Ram, Z.; Shahar, T. MRI radiomics analysis of molecular alterations in low-grade gliomas. Int. J. Comput. Assist. Radiol. Surg. 2018, 13, 563–571. [Google Scholar] [CrossRef] [PubMed]
  61. Tongtong, L.; Guoqing, W.; Jinhua, Y.; Yi, G.; Yuanyuan, W.; Zhifeng, S.; Liang, C. A mRMRMSRC feature selection method for radiomics approach. In Proceedings of the 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Jeju, Korea, 11–15 July 2017; pp. 616–619. [Google Scholar] [CrossRef]
  62. Van der Voort, S.R.; Incekara, F.; Wijnenga, M.M.J.; Kapas, G.; Gardeniers, M.; Schouten, J.W.; Starmans, M.P.A.; Nandoe Tewarie, R.; Lycklama, G.J.; French, P.J.; et al. Predicting the 1p/19q Codeletion Status of Presumed Low-Grade Glioma with an Externally Validated Machine Learning Algorithm. Clin. Cancer Res. 2019, 25, 7455–7462. [Google Scholar] [CrossRef] [Green Version]
  63. Zhang, B.; Chang, K.; Ramkissoon, S.; Tanguturi, S.; Bi, W.L.; Reardon, D.A.; Ligon, K.L.; Alexander, B.M.; Wen, P.Y.; Huang, R.Y. Multimodal MRI features predict isocitrate dehydrogenase genotype in high-grade gliomas. Neuro Oncol. 2017, 19, 109–117. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  64. Zhang, X.; Tian, Q.; Wang, L.; Liu, Y.; Li, B.; Liang, Z.; Gao, P.; Zheng, K.; Zhao, B.; Lu, H. Radiomics Strategy for Molecular Subtype Stratification of Lower-Grade Glioma: Detecting IDH and TP53 Mutations Based on Multimodal MRI. J. Magn. Reson. Imaging 2018, 48, 916–926. [Google Scholar] [CrossRef] [PubMed]
  65. Zhang, Z.; Xiao, J.; Wu, S.; Lv, F.; Gong, J.; Jiang, L.; Yu, R.; Luo, T. Deep Convolutional Radiomic Features on Diffusion Tensor Images for Classification of Glioma Grades. J. Digit. Imaging 2020, 33, 826–837. [Google Scholar] [CrossRef] [PubMed]
  66. Zhou, H.; Chang, K.; Bai, H.X.; Xiao, B.; Su, C.; Bi, W.L.; Zhang, P.J.; Senders, J.T.; Vallières, M.; Kavouridis, V.K.; et al. Machine learning reveals multimodal MRI patterns predictive of isocitrate dehydrogenase and 1p/19q status in diffuse low- and high-grade gliomas. J. Neurooncol. 2019, 142, 299–307. [Google Scholar] [CrossRef] [PubMed]
  67. Rathore, S.; Bakas, S.; Nasrallah, M.P.; Bagley, S.J.; Akbari, H.; Ha, S.M.; Mamourian, E.C.; Watt, C.D.; Davatzikos, C. Non-invasive determination of the O6-methylguanine-DNA-methyltransferase (MGMT) promoter methylation status in glioblastoma (GBM) using magnetic resonance imaging (MRI). J. Clin. Oncol. Conf. 2018, 36, 2051. [Google Scholar] [CrossRef]
  68. Bonte, S.; Goethals, I.; Van Holen, R. Automated grade prediction of glioma patients based on magnetic resonance imaging and a random forests approach. Neuro Oncol. 2016, 18, iv38. [Google Scholar] [CrossRef] [Green Version]
  69. Han, Y.; Xie, Z.; Zang, Y.; Zhang, S.; Gu, D.; Zhou, M.; Gevaert, O.; Wei, J.; Li, C.; Chen, H.; et al. Non-invasive genotype prediction of chromosome 1p/19q co-deletion by development and validation of an MRI-based radiomics signature in lower-grade gliomas. J. Neurooncol. 2018, 140, 297–306. [Google Scholar] [CrossRef]
  70. Kinoshita, M.; Arita, H.; Kawaguchi, A.; Takahashi, M.; Fukai, J.; Ishibashi, K.; Tsuyuguchi, N.; Okita, Y.; Narita, Y.; Ichimura, K.; et al. MRI based radiogenomics for glioma. Neuroradiology 2018, 60, 1118–1119. [Google Scholar] [CrossRef]
  71. Park, C.J.; Choi, Y.S.; Park, Y.W.; Ahn, S.S.; Kang, S.G.; Chang, J.H.; Kim, S.H.; Lee, S.K. Diffusion tensor imaging radiomics in lower-grade glioma: Improving subtyping of isocitrate dehydrogenase mutation status. Neuroradiology 2020, 62, 319–326. [Google Scholar] [CrossRef]
  72. Park, Y.W.; Choi, Y.S.; Ahn, S.S.; Chang, J.H.; Kim, S.H.; Lee, S.K. Radiomics MRI Phenotyping with Machine Learning to Predict the Grade of Lower-Grade Gliomas: A Study Focused on Nonenhancing Tumors. Korean J. Radiol. 2019, 20, 1381–1389. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  73. Wu, S.; Meng, J.; Yu, Q.; Li, P.; Fu, S. Radiomics-based machine learning methods for isocitrate dehydrogenase genotype prediction of diffuse gliomas. J. Cancer Res. Clin. Oncol. 2019, 145, 543–550. [Google Scholar] [CrossRef] [Green Version]
  74. Zhao, S.S.; Feng, X.L.; Hu, Y.C.; Han, Y.; Tian, Q.; Sun, Y.Z.; Zhang, J.; Ge, X.W.; Cheng, S.C.; Li, X.L.; et al. Better efficacy in differentiating WHO grade II from III oligodendrogliomas with machine-learning than radiologist’s reading from conventional T1 contrast-enhanced and fluid attenuated inversion recovery images. BMC Neurol. 2020, 20, 48. [Google Scholar] [CrossRef]
  75. Chang, K.; Bai, H.X.; Zhou, H.; Su, C.; Bi, W.L.; Agbodza, E.; Kavouridis, V.K.; Senders, J.T.; Boaro, A.; Beers, A.; et al. Residual Convolutional Neural Network for the Determination of IDH Status in Low- and High-Grade Gliomas from MR Imaging. Clin. Cancer Res. 2018, 24, 1073–1081. [Google Scholar] [CrossRef] [Green Version]
  76. Han, L.; Kamdar, M.R. MRI to MGMT: Predicting methylation status in glioblastoma patients using convolutional recurrent neural networks. Pac. Symp. Biocomput. 2018, 23, 331–342. [Google Scholar]
  77. Li, Z.C.; Bai, H.; Sun, Q.; Li, Q.; Liu, L.; Zou, Y.; Chen, Y.; Liang, C.; Zheng, H. Multiregional radiomics features from multiparametric MRI for prediction of MGMT methylation status in glioblastoma multiforme: A multicentre study. Eur. Radiol. 2018, 28, 3640–3650. [Google Scholar] [CrossRef]
  78. Li, Z.C.; Bai, H.; Sun, Q.; Zhao, Y.; Lv, Y.; Zhou, J.; Liang, C.; Chen, Y.; Liang, D.; Zheng, H. Multiregional radiomics profiling from multiparametric MRI: Identifying an imaging predictor of IDH1 mutation status in glioblastoma. Cancer Med. 2018, 7, 5999–6009. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  79. Liang, S.; Zhang, R.; Liang, D.; Song, T.; Ai, T.; Xia, C.; Xia, L.; Wang, Y. Multimodal 3D DenseNet for IDH Genotype Prediction in Gliomas. Genes 2018, 9, 382. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  80. Takahashi, S.; Tanaka, S.; Takahashi, M.; Yamazawa, E.; Hana, T.; Kitagawa, Y.; Takayanagi, S.; Takahashi, W.; Nakamoto, T.; Haga, A.; et al. Visualization of judgment basis of CNN to grading glioma. Neuro Oncol. 2019, 21, vi163. [Google Scholar] [CrossRef]
  81. Yang, Y.; Yan, L.F.; Zhang, X.; Han, Y.; Nan, H.Y.; Hu, Y.C.; Hu, B.; Yan, S.L.; Zhang, J.; Cheng, D.L.; et al. Glioma Grading on Conventional MR Images: A Deep Learning Study With Transfer Learning. Front. Neurosci. 2018, 12, 804. [Google Scholar] [CrossRef] [Green Version]
  82. Takahashi, W.; Nakamoto, T.; Akihiro, H.; Satoshi, T.; Shota, T.; Aoki, S.; Kiritoshi, T.; Ogita, M.; Yamashita, H.; Nakagawa, K. MRI-based radiogenomics analysis of 1p/19q codeletion in grade II and III gliomas. Radiother. Oncol. 2019, 133, S1057–S1058. [Google Scholar] [CrossRef]
  83. Wei, J.; Yang, G.; Hao, X.; Gu, D.; Tan, Y.; Wang, X.; Dong, D.; Zhang, S.; Wang, L.; Zhang, H.; et al. A multi-sequence and habitat-based MRI radiomics signature for preoperative prediction of MGMT promoter methylation in astrocytomas with prognostic implication. Eur. Radiol. 2019, 29, 877–888. [Google Scholar] [CrossRef] [Green Version]
  84. Batchala, P.P.; Muttikkal, T.J.E.; Donahue, J.H.; Patrie, J.T.; Schiff, D.; Fadul, C.E.; Mrachek, E.K.; Lopes, M.B.; Jain, R.; Patel, S.H. Neuroimaging-Based Classification Algorithm for Predicting 1p/19q-Codeletion Status in IDH-Mutant Lower Grade Gliomas. AJNR Am. J. Neuroradiol. 2019, 40, 426–432. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  85. Yu, J.; Shi, Z.; Lian, Y.; Li, Z.; Liu, T.; Gao, Y.; Wang, Y.; Chen, L.; Mao, Y. Noninvasive IDH1 mutation estimation based on a quantitative radiomics approach for grade II glioma. Eur. Radiol. 2017, 27, 3509–3522. [Google Scholar] [CrossRef]
  86. Clark, V.E.; Cahill, D.P. Extent of Resection Versus Molecular Classification: What Matters When? Neurosurg. Clin. N. Am. 2019, 30, 95–101. [Google Scholar] [CrossRef] [PubMed]
  87. Dubbink, H.J.; Atmodimedjo, P.N.; Kros, J.M.; French, P.J.; Sanson, M.; Idbaih, A.; Wesseling, P.; Enting, R.; Spliet, W.; Tijssen, C.; et al. Molecular classification of anaplastic oligodendroglioma using next-generation sequencing: A report of the prospective randomized EORTC Brain Tumor Group 26951 phase III trial. Neuro Oncol. 2016, 18, 388–400. [Google Scholar] [CrossRef]
  88. Sotoudeh, H.; Shafaat, O.; Bernstock, J.D.; Brooks, M.D.; Elsayed, G.A.; Chen, J.A.; Szerip, P.; Chagoya, G.; Gessler, F.; Sotoudeh, E.; et al. Artificial Intelligence in the Management of Glioma: Era of Personalized Medicine. Front. Oncol. 2019, 9, 768. [Google Scholar] [CrossRef] [PubMed]
  89. Bhandari, A.P.; Liong, R.; Koppen, J.; Murthy, S.V.; Lasocki, A. Noninvasive Determination of IDH and 1p19q Status of Lower-grade Gliomas Using MRI Radiomics: A Systematic Review. AJNR Am. J. Neuroradiol. 2021, 42, 94–101. [Google Scholar] [CrossRef] [PubMed]
  90. Van den Bent, M.J. Interobserver variation of the histopathological diagnosis in clinical trials on glioma: A clinician’s perspective. Acta Neuropathol. 2010, 120, 297–304. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  91. Van Leeuwen, K.G.; Schalekamp, S.; Rutten, M.; van Ginneken, B.; de Rooij, M. Artificial intelligence in radiology: 100 commercially available products and their scientific evidence. Eur. Radiol. 2021, 31, 3797–3804. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Classification of gliomas according to the WHO 2016 Classification Guidelines. IDH: Isocitrate dehydrogenase gene; mut: mutant; wt: wildtype.
Figure 1. Classification of gliomas according to the WHO 2016 Classification Guidelines. IDH: Isocitrate dehydrogenase gene; mut: mutant; wt: wildtype.
Cancers 13 02606 g001
Figure 2. PRISMA flowchart of systematic literature search. MLA: Machine learning algorithms; MR imaging: Magnetic resonance imaging.
Figure 2. PRISMA flowchart of systematic literature search. MLA: Machine learning algorithms; MR imaging: Magnetic resonance imaging.
Cancers 13 02606 g002
Figure 3. Forest-plot of the included studies that assessed the accuracy of predicting IDH mutation status in glioma. AUC, Area Under the receiver operator Curve; CI, Confidence Interval. Forest plot shows that the performance of the MLAs to classify IDH mutation status are centered around an AUC of 0.909 with a 95%-CI ranging from 0.867–0.951.
Figure 3. Forest-plot of the included studies that assessed the accuracy of predicting IDH mutation status in glioma. AUC, Area Under the receiver operator Curve; CI, Confidence Interval. Forest plot shows that the performance of the MLAs to classify IDH mutation status are centered around an AUC of 0.909 with a 95%-CI ranging from 0.867–0.951.
Cancers 13 02606 g003
Figure 4. Forest-plot of the included studies that assessed the accuracy of predicting 1p/19q codeletion status in glioma. AUC, Area Under the receiver operator Curve; CI, Confidence Interval. Forest plot shows that the performance of the MLAs to classify 1p/19q codeletion status are centered around an AUC of 0.748 with a 95%-CI ranging from 0.699–0.797.
Figure 4. Forest-plot of the included studies that assessed the accuracy of predicting 1p/19q codeletion status in glioma. AUC, Area Under the receiver operator Curve; CI, Confidence Interval. Forest plot shows that the performance of the MLAs to classify 1p/19q codeletion status are centered around an AUC of 0.748 with a 95%-CI ranging from 0.699–0.797.
Cancers 13 02606 g004
Figure 5. Forest-plot of the included studies that assessed the accuracy of predicting MGMT promoter methylation status in glioma. AUC, Area Under the receiver operator Curve; CI, Confidence Interval. Forest plot shows that the performance of the MLAs to classify MGMT mutation status are centered around an AUC of 0.866 with a 95%-CI ranging from 0.812–0.921.
Figure 5. Forest-plot of the included studies that assessed the accuracy of predicting MGMT promoter methylation status in glioma. AUC, Area Under the receiver operator Curve; CI, Confidence Interval. Forest plot shows that the performance of the MLAs to classify MGMT mutation status are centered around an AUC of 0.866 with a 95%-CI ranging from 0.812–0.921.
Cancers 13 02606 g005
Figure 6. Forest-plot of the included studies that assessed the accuracy of predicting TERT promoter mutation status in glioma. AUC, Area Under the receiver operator Curve; CI, Confidence Interval. Forest plot shows that the performance of the MLAs to classify TERT promoter mutation status are centered around an AUC of 0.842 with a 95%-CI ranging from 0.783–0.901.
Figure 6. Forest-plot of the included studies that assessed the accuracy of predicting TERT promoter mutation status in glioma. AUC, Area Under the receiver operator Curve; CI, Confidence Interval. Forest plot shows that the performance of the MLAs to classify TERT promoter mutation status are centered around an AUC of 0.842 with a 95%-CI ranging from 0.783–0.901.
Cancers 13 02606 g006
Table 1. Participant demographics, study characteristics and outcomes of the included studies.
Table 1. Participant demographics, study characteristics and outcomes of the included studies.
Training SetValidation Set Performance *
First Author
(Year of Publication) (Reference)
NAge
(Mean ± SD)
Gender
(Male-Female)
NValidated on
an Independent External Dataset?
Input
Imaging Data
MLA MethodTarget
Condition
SensitivitySpecificity AUC (±SD)Accuracy
Ahammed Muneer (2019) [46]389NRNR168NoT2w images; tumor segmentationDeep CNN Glioma grade92.7298.13NR94.64
Arita (2018) [27]111NRNR58NoT2w-based VOI segmentation and T1w, T2w, FLAIR, and T1w +c imagesLasso and Elastic-Net Regularized Generalized Linear ModelIDH genotype NRNRNR87
Bakas (2018) [47]86NRNRNRNoT1w, T2w, FLAIR, T1w +c images; DTI series and DSC-PWI seriesMultivariate machine learning model with a Random Forests algorithmIDH genotype66.792.9NR88.4
Bangalore Yogananda (2020) [48]214NRNR214NoT2w, FLAIR, and T1w +c images3D Dense-UNet: T2-NetIDH genotype97 98 0.98 ± 0.14697.14
3D Dense-UNet: TS-NetIDH genotype98 97 0.99 ± 0.14697.12
Batchala (2019) [84]102NR50–52106NoT1w, T2w, FLAIR, and T1w +c images; DSC-PWI seriesMultivariate model1p/19q integrityNRNRNR81.1
Bonte (2016) [68]274NRNRNRNoBraTS-data (T1w, T2w, FLAIR, and T1w +c images)Random Forests algorithmLGG/HGG95.579.5NR92.3
Cao (2020) [28]141NR74–6788NoT1w, T2w, FLAIR, and T1w +c imagesLasso and Elastic-Net Regularized Generalized Linear Model with Support vector machine classifierLGG/HGGNRNR0.915 ± 0.356NR
Carver (2019) [29]78NRNR50YesT1w, T2w, FLAIR, and T1w +c imagesLasso and Elastic-Net Regularized Generalized Linear ModelIDH genotypeNRNRNR74
Chang (2018) [77]1188NRNR153NoT1w, T2w, FLAIR, and T1w +c imagesResidual CNN modelIDH genotypeNRNR0.9383.0
Citak-Er (2018) [49]4349.5 ± 12.825–18NRNoT1w, T2w, DW images; DTI series, DSC-PWI series, and MRSSupport vector machine classifier with linear kernel and logistic regression with a Random Forests algorithmLGG/HGG86.796.4NR93.0
Cui (2018) [30]40NRNRNRNoT1w, T2w, FLAIR, and T1w +c images; tumor segmentationLasso and Elastic-Net Regularized Generalized Linear ModelLGG/HGGNRNR0.84NR
De Looze (2018) [50]381NR251–130NRNoThree VASARI criteria as assessed on T1w, T2w, FLAIR, and DW imagesRandom Forests modelIDH genotype81770.88NR
Glioma grade II/III82940.98NR
Glioma grade II/IV1001001.0NR
Glioma grade III/IV83970.97NR
Fan (2019) [45]12646.8NRNRNoT1w +c imagesLasso and Elastic-Net Regularized Generalized Linear Model adopted into linear discriminant analysis and Support vector machine classifierglioblastoma/anaplastic oligodendro-glioma100.091.00.92393.8
Gates (2020) [51]23NRNRNRNoT2, ADC, CBV, and KtransRandom Forests algorithmGlioma gradeNRNRNR96
Han (2018) [76]117NRNR21NoT1w, T2w, and FLAIR imagesRecurrent CNN modelMGMT promoter methylation statusNRNR0.5453
Han (2018) [71] 18441.67120–6493NoT2w images and
T2w-based segmentation
Random Forests algorithm1p/19q integrity68.371.20.760 ± 0.47770.0
Hwan-Ho (2017) [33]108NRNRNRNoBraTS-data (T1w, T2w, FLAIR, and T1w +c images) and BraTS-segmentationLasso and Elastic-Net Regularized Generalized Linear Model and logistic regressionGlioma grade88.8990.740.887089.81
Inano (2014) [52]33NR22–1133NoDW images, FA-maps, first eigenvalue, second eigenvalue, third eigenvalue, MD-maps, and raw T2 signal
with no diffusion-weighting
Support vector machine classifiersGlioma grade84.8 74.5 0.912 ± 0.02880.4
Jiang (2019) [34]8745.4 ± 13.143–4435YesT2w and T1w +c imagesLasso regression model with fusion Radiomics model and Support vector machine classifierMGMT promoter methylation status82.185.70.898 ± 0.32388.6
Jiang (2020) [35]8345.5 ± 12.350–3333YesT2w and T1w +c imagesLasso regression model with radiomics signature model and Support vector machine classifierTERT promoter mutation status71.489.50.827 ± 0.47084.8
Kim (2020) [53]127NR68–5928NoT1w, T2w, FLAIR, T1w +c, DW images; DSC-PWI seriesRecursive feature elimination with Support vector machine, completed with a Random Forests algorithm and a logistic regression classifierIDH genotype53.686.70.747 ± 0.228NR
Kinoshita (2018) [70]199NRNRNRNoConventional MR sequences (NOS)Random Forests algorithmGlioma gradeNRNR0.71164.5
Lee (2019) [54]88NR47–4135YesT1w, T2w, FLAIR, DW images; DSC-PWI seriesEight machine learning classifiers:
K-Nearest Neighbors, Support vector classification, Decision Tree, Random Forest, AdaBoost, Naive Bayes, Linear Discriminant Analysis, and Gradient Boosting
IDH genotypeNRNRNR83.4
Li (2019) [55]6960.037–3240YesT2w and T1w +c imagesSupport vector machine classifier with Support vector machine classifierPTEN genotype86.770.00.78782.5
Li (2018) [32]6343.625–3891YesT2w imagesLasso regression model with Support vector machine classifierATRX genotype57.185.70.72576.9
Li (2018) [33]18039.2111–6992NoT2w imagesLasso regression model with Support vector machine classifierP53 status62.285.10.76370.7
Li (2017) [56]15140.7 ± 10.881–70151NoT1w and FLAIR imagesCNN for segmentation followed by DLR model with Support vector machine classifierIDH genotype94.3886.670.952192.44
Li (2018) [77]13354.279–5460NoT1w, T2w, FLAIR, and T1w +c imagesMultiregional Radiomics modelMGMT-methylationNRNR0.8880
Li (2018) [78]11853.6 70–48107NoT1w, T2w, FLAIR, and T1w +c imagesMultiregional Radiomics modelsIDH genotype80990.9697
Liang (2018) [79]16752.4 ± 15.5NRNRNoBraTS-data (T1w, T2w, FLAIR, and T1w +c images)Multimodal Three-Dimensional DenseNetIDH genotype78.588.00.85784.6
Lo (2020) [57]39NR28–11NRNoT1w +c images; processed by transformed ranklet images.Logistic regression
classifier
IDH genotype5797NR90
Lu (2018) [58]214NRNR70YesT1w, T2w, FLAIR, T1w +c, and DW images
(T2w and DW images were optional)
Three-level machine learning modelLGG/HGG82.590.5NR87.7
Matsui (2020) [36]21742131–86NRNoT1w, T2w, and FLAIR imagesLasso regression model with DLR modelGrading LGGNRNRNR58.5
Mzoughi (2020) [37]284NRNR67YesT1w +c imagesLasso regression model with 3D CNN model with Support vector machine classifierGlioma gradeNRNRNR96.4
Park (2020) [71]168NRNR168NoT2w, FLAIR, and T1w +c imagesRandom Forests algorithmIDH genotypeNRNR0.900 ± 0.298NR
Park (2019) [72]13644.99 ± 12.9465–7199YesT2w, FLAIR, and T1w +c images; DTI seriesRandom Forests algorithmGlioma grade72.660.40.72 ± 0.5166.7
Rathore (2019) [59]202NRNRNRNoT1w, T2w, FLAIR, and T1w +c images.
Data were sometimes complemented with DTI and DSC-PWI series
CNN adjusted with a Support vector machine classifierIDH genotype83860.8585
MGMT83850.8483
Rathore (2018) [67]111NRNRNRNoT1w, T2w, FLAIR, and T1w +c imagesSupport Vector Machine model with a Random Forests algorithmMGMT-methylation75.097.00.8088.28
Rathore (2019) [59]270NRNRNRNoT1w, T2w, FLAIR, and T1w +c images; DTI and DSC-PWI seriesCross-validated sequential feature selectionMGMT-methylationNRNRNR86.95
Sasaki (2018) [39]207NRNRNR
NoT1w, T2w, FLAIR, and T1w +c imagesLasso regression model with supervised component principal analysisMGMT-methylationNRNRNR68
Sasaki (2019) [38]201NRNRNRNoT1w, T2w, and T1w +c imagesLasso regression model with supervised component principal analysisMGMT-methylation6766NR67
Shboul (2020) [40]81NRNR27NoT1w, T2w, FLAIR, and T1w +c imagesLasso regression model with supervised component principal analysis and multi-resolution fractal modelingIDH genotype90 79 0.84 ± 0.156NR
1p/19q integrity75 85 0.80 ± 0.208NR
MGMT-methylation93 73 0.83 ± 0.208NR
ATRX genotype69 83 0.70 ± 0.468NR
TERT promoter mutation status77 86 0.82 ± 0.208NR
Shofty (2018) [60]4737.7 ± 10.627–20NRNoT2w, FLAIR, and T1w +c imagesEnsemble Radiomic Classifier model with a Support vector machine classifier1p/19q integrity92830.8787
Sun (2020) [41]92NRNRNRNoT1w, T2w imagesLasso regression model with logistic regression modelsP53 status100400.70981.3
Takahashi (2019) [80]44NRNR11NoDW (b1000 and b2000) images, ADC-maps, FA-maps, and MK-mapsDeep CNN modelGlioma gradeNRNRNR82
Takahashi (2019) [82]38NRNRNRNoT2w-based VOI segmentationLogistic regression models1p/19q integrity69.773.30.73671.1
Tan (2019) [42]7447.93 ± 13.2845–2931NoFLAIR and T1w +c images; ADC-mapsRadiomics Nomogram modelIDH genotype86.787.50.900 ± 0.11687.1
Tian (2020) [43]88NR53–3538NoT1w, T2w, FLAIR, and T1w +c images; MRSLasso regression model with Radiomics Nomogram modelTERT promoter mutation status75.090.90.889 ± 0.33584.2
Tongtong (2017) [61]110NRNRNRNo3D FLAIR imagesSupport vector machine classifier with minimum redundancy, maximum relevance, and maximum sparse representation coefficientIDH genotype88790.9085
van der Voort (2019) [62]284NR161–123129YesT2w and T1w +c images. Data were sometimes complemented with FLAIR imagesSupport vector machine classifier1p/19q integrity73.2 61.7 0.723 ± 0.08469.3
Wei (2019) [83]74NR42–3231NoFLAIR and T1w +c images; ADC-mapsFusion Radiomics model by logistic regression modellingMGMT promoter methylation94.453.90.902 ± 0.30577.4
Wu (2019) [73]8453.5 ± 15.067–5942NoT1w, T2w, FLAIR, and T1w +c imagesRandom Forests algorithmIDH genotypeNRNR0.931 ± 0.23389.5
Xi (2018) [44]98NR55–4320YesT1w, T2w, and T1w +c imagesLasso regression model with Support vector machine modelMGMT promoter methylation87.575.0NR80.0
Yang (2018) [81]113NRNRNRNoT1w, T2w, FLAIR, and T1w +c imagesCNN modelLGG/HGGNRNRNR86.7
Yu (2017) [85]11040.3 ± 11.354–5630NoFLAIR imagesRadiomics modelIDH genotype88670.7983
Zhang (2017) [63]9051.452–3830NoT1w, T2w, FLAIR, T1w +c, and DW imagesRandom Forests algorithmIDH genotypeNRNR0.923189
Zhang (2018) [64]73NRNR30NoT1w, T2w, FLAIR, and T1w +c imagesSupport vector machine-based recursive feature eliminationIDH genotype 85.070.00.79280.0
P53 status84.685.70.86985.0
Zhang (2020) [65]108NR61–47NRNoDTI seriesCNN model with a Support vector machine classifierLGG/HGG98860.9394
Glioma grade III/IV981000.9998
Zhao (2020) [74]3645.0 ± 14.419–1736NoFLAIR and T1w +c imagesRandom Forests algorithmGlioma grade II/III77.878.30.861 ± 0.24078.1
Zhou (2019) [66]538NR303–235206YesFLAIR and T1w +c imagesRandom Forests algorithm with a Support vector machine classifierIDH genotypeNRNR0.919 ± 0.286NR
Legend: ADC: Apparent diffusion coefficient; ARTX: Alpha thalassemia/mental retardation syndrome X linked gene; BraTS: Brain Tumor Segmentation Challenge; CNN: Convolutional neural network; DW: Diffusion-weighted images; DTI: Diffusion tensor imaging; DSC-PWI: Dynamic susceptibility contrast perfusions weighted imaging; FA: Fractional anisotropy imaging; FLAIR: fluid attenuated inversion recovery; HGG: High grade glioma; IDH: Isocitrate dehydrogenase gene; LGG: Low-grade glioma; MGMT: O6-Methylguanine-DNA Methyltransferase; MRS: Magnetic resonance spectroscopy imaging; TERT: Telomerase reverse transcriptase gene; T1w: T1-weighted images; T1w +c: T1-weighted post-contrast images; T2w: T2-weighted images; VASARI: Visually AcceSAble Rembrandt Images. * If cross-validation was used, the Performance values of the cross-validation set were provided here. When the dataset was split into training/validation/test sets, the Performance evaluation values with regard to the investigated outcome (e.g., IDH genotype) of the Validation set were provided here.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

van Kempen, E.J.; Post, M.; Mannil, M.; Kusters, B.; ter Laan, M.; Meijer, F.J.A.; Henssen, D.J.H.A. Accuracy of Machine Learning Algorithms for the Classification of Molecular Features of Gliomas on MRI: A Systematic Literature Review and Meta-Analysis. Cancers 2021, 13, 2606. https://doi.org/10.3390/cancers13112606

AMA Style

van Kempen EJ, Post M, Mannil M, Kusters B, ter Laan M, Meijer FJA, Henssen DJHA. Accuracy of Machine Learning Algorithms for the Classification of Molecular Features of Gliomas on MRI: A Systematic Literature Review and Meta-Analysis. Cancers. 2021; 13(11):2606. https://doi.org/10.3390/cancers13112606

Chicago/Turabian Style

van Kempen, Evi J., Max Post, Manoj Mannil, Benno Kusters, Mark ter Laan, Frederick J. A. Meijer, and Dylan J. H. A. Henssen. 2021. "Accuracy of Machine Learning Algorithms for the Classification of Molecular Features of Gliomas on MRI: A Systematic Literature Review and Meta-Analysis" Cancers 13, no. 11: 2606. https://doi.org/10.3390/cancers13112606

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop