Next Article in Journal
Identification of Functional Cortical Plasticity in Children with Cerebral Palsy Associated to Robotic-Assisted Gait Training: An fNIRS Study
Next Article in Special Issue
Overview of Artificial Intelligence in Breast Cancer Medical Imaging
Previous Article in Journal
Validation of the Trifecta Scoring Metric in Vacuum-Assisted Mini-Percutaneous Nephrolithotomy: A Single-Center Experience
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
JCM
Volume 11
Issue 22
10.3390/jcm11226789
jcm-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Radiomics Combined with Multiple Machine Learning Algorithms in Differentiating Pancreatic Ductal Adenocarcinoma from Pancreatic Neuroendocrine Tumor: More Hands Produce a Stronger Flame

by
Tao Zhang
1,†,
Yu Xiang
2,†,
Hang Wang
3,
Hong Yun
2,
Yichun Liu
4,
Xing Wang
5,* and
Hao Zhang
5,*
1
State Key Laboratory of Biotherapy, Sichuan University, Chengdu 610041, China
2
Department of Biotherapy, West China Hospital, Sichuan University, Chengdu 610041, China
3
West China School of Medicine, Sichuan University, Chengdu 610041, China
4
Sichuan Center for Disease Control and Prevention, Chengdu 610041, China
5
Department of Pancreatic Surgery, West China Hospital, Sichuan University, Chengdu 610041, China
*
Authors to whom correspondence should be addressed.
†
These authors contributed equally to this work.
J. Clin. Med. 2022, 11(22), 6789; https://doi.org/10.3390/jcm11226789
Submission received: 9 October 2022 / Revised: 4 November 2022 / Accepted: 15 November 2022 / Published: 16 November 2022
(This article belongs to the Special Issue Clinical Application of Artificial Intelligence in Cancer Research, Diagnosis and Therapy)
Browse Figures
Review Reports Versions Notes

Abstract

:
The aim of this study was to assess the diagnostic ability of radiomics combined with multiple machine learning algorithms to differentiate pancreatic ductal adenocarcinoma (PDAC) from pancreatic neuroendocrine tumor (pNET). This retrospective study included a total of 238 patients diagnosed with PDAC or pNET. Using specialized software, radiologists manually mapped regions of interest (ROIs) from computed tomography images and automatically extracted radiomics features. A total of 45 discriminative models were built by five selection algorithms and nine classification algorithms. The performances of the discriminative models were assessed by sensitivity, specificity and the area under receiver operating characteristic curve (AUC) in the training and validation datasets. Using the combination of Gradient Boosting Decision Tree (GBDT) as the selection algorithm and Random Forest (RF) as the classification algorithm, the optimal diagnostic ability with the highest AUC was presented in the training and validation datasets. The sensitivity, specificity and AUC of the model were 0.804, 0.973 and 0.971 in the training dataset and 0.742, 0.934 and 0.930 in the validation dataset, respectively. The combination of radiomics and multiple machine learning algorithms showed the potential ability to discriminate PDAC from pNET. We suggest that multi-algorithm modeling should be considered for similar studies in the future rather than using a single algorithm empirically.

1. Introduction

Pancreatic cancer is one of the most common malignancies with an increasing incidence and is the seventh leading cause of cancer death due to poor prognosis [1,2,3]. Based on its origins and pathology, pancreatic cancer has been classified into several subtypes, including pancreatic ductal adenocarcinoma (PDAC) and pancreatic neuroendocrine tumor (pNET). Among them, PDAC, accounting for 90% of all primary pancreatic malignancies, shows aggressive behavior and poor prognosis, which leads to a five-year survival rate of less than 5% [4,5]. It can only be cured by surgical resection, which is available merely to a small number of patients whose tumors can be surgically removed before they progress to the advanced stage [2]. Unfortunately, most patients tend to be diagnosed with metastasis, in which case chemotherapy and radiotherapy do not significantly improve survival rates [6]. Even immunotherapy, which has shown success in other tumors in recent years, is ineffective in treating PDAC [7]. In contrast, pNET is not as common as PDAC and carries a significantly better prognosis with a five-year survival rate of 51.3% [8]. Pancreatic neuroendocrine tumors are classified into functioning or non-functioning according to clinical symptoms. The tumor cells in functioning pNET can secrete a variety of hormones, such as glucagon, insulin and gastrin, which can lead to symptoms of hormone hypersecretion [9]. Non-functioning pNET does not produce excess hormones but has a wide range of clinical behaviors [10]. The treatment strategy for pNET includes surgical resection and chemotherapy, while hormone therapy is controversial. Contrary to PDAC, pNET patients can benefit from early treatment and long-term survival can be expected even with metastases [11]. Therefore, the importance of an early diagnosis of PDAC and pNET cannot be overstated due to their different prognosis and therapeutic strategies.
Magnetic resonance imaging (MRI) and computed tomography (CT) are widely used in detecting abdominal lesions and lymph node metastases. However, MRI has some limitations compared with CT, such as it being time-consuming and costly, and patients experience noise and isolation during the scan. CT is considered as an economical radiological exam to detect pancreatic tumors [12]. Nevertheless, the CT characteristics of PDAC and pNET are quite similar, which makes it easy to lead to misdiagnosis by visual assessment [13,14]. Consequently, it is necessary to develop more feasible methods to assist in diagnosis and improve accuracy. The emergence of radiomics has shown promising prospects in the domain of radiological evaluation. Radiomics refers to the high-throughput extraction of quantitative image features from medical imaging followed by data analysis to support clinical decision-making [15,16,17]. At the same time, new methods for analyzing mineable radiomics feature are required. Machine learning is a major branch of artificial intelligence and is defined as the ability of a machine to learn and predict future events and outcomes based on large datasets [18]. The combination of radiomics and machine learning has presented promising performance in previous research, including predicting histological subtypes of lung cancer, the grade of meningioma and survival outcomes in non-small-cell lung cancer [19,20,21]. Moreover, it has also been prominent in the diagnosis of pancreatic lesions such as pseudocysts, serous cystadenomas, autoimmune pancreatitis and PDAC [22,23,24]. Previous studies have explored the ability to differentiate PDAC from pNET by purely textural features based on CT or MRI in relatively small sample sizes [25,26].
Although radiomics and machine learning have been widely used in disease diagnosis, the application of radiomics and multiple machine learning algorithms combined in discriminating PDAC and pNET has not been reported. The aim of this study was to evaluate the diagnostic ability of radiomics features combined with multiple machine learning algorithms in differentiating PDAC from pNET.

2. Materials and Methods

2.1. Patient Selection

We retrospectively viewed our database to search for patients diagnosed with PDAC or pNET in West China Hospital with detailed medical records from August 2013 to May 2019. The eligibility criteria for the inclusion of selected patients were: (1) with complete medical records; (2) with a pathological diagnosis confirmation of PDAC or pNET; (3) with high-quality CT images before surgery. Exclusion criteria were: (1) history of treatment before CT scan, such as chemotherapy and radiotherapy; (2) history of other cancers. A total of 238 patients with PDAC or pNET met the inclusion criteria in the initial selection. Then, we collected the information of these patients, including gender, age, body mass index (BMI), glucose, calcium, procalcitonin, α-fetoprotein (AFP), carcinoembryonic antigen (CEA), glucoprotein antigen 199 (CA19-9), glucoprotein antigen 125 (CA12-5), total bilirubin (TBIL), amylase and lipase. This study was approved by the Ethics Committee of West China Hospital, Sichuan University, and all patients’ informed consents were waived.

2.2. Image Acquisition and Texture Feature Extraction

All selected patients underwent CT examinations, the operating parameters of which were reported in previous studies [27]. The procedures of scanning are summarized in the Supplementary Materials. The CT diagnostic criteria for pancreatic cancer are divided into three parts: morphologic evaluation, vascular evaluation and extrapancreatic evaluation [28]. Morphologic evaluation includes appearance (confined masses of the pancreas are less dense than normal pancreas), size, location (most commonly PDAC in the head of the pancreas and neuroendocrine elsewhere), pancreatic duct narrowing/abrupt cut-off with or without upstream dilatation, and biliary tree abrupt cut-off with or without upstream dilatation. Vascular evaluation can be divided into arterial evaluation and venous evaluation. Arterial evaluation includes the evaluation of the celiac axis, superior mesenteric artery (SMA), common hepatic artery (CHA) and arterial variants. Venous evaluation includes the evaluation of the main portal vein (MPV), superior mesenteric vein (SMV), venous collaterals and intravenous thrombosis. Extrapancreatic evaluation includes the evaluation of liver metastases, peritoneal nodes, suspicious lymph nodes in the abdomen and ascites. PDAC is considered if the above criteria for pancreatic cancer are met, otherwise pNET is suspected, but the final diagnosis is based on pathological results. Recognizable CT images with clear boundaries were acquired from the picture archiving and communication systems in the radiology department. Two radiologists with pancreatic expertise extracted the texture features from CT images using the Local Image Features Extraction (LIFEx) software (v3.74, CEA-SHFJ, Orsay, France) [29]. In the guidelines of the software, ROI was manually drawn along the border of tumor issue slice by slice to gain the 3D features (Figure 1). Any differences in ROIs were recorded and discussed. The preferred ROI of each patient was selected by the third senior radiologist and was included in our study. All ROI data from the enhanced CT were labeled as PDAC or pNET according to the pathological results. Finally, a total of 48 radiomics features were automatically exported and saved through the LIFEx software.

2.3. Discriminative Model Establishment and Data Analysis

The selection algorithm is considered as an important data preprocessing step in machine learning tasks, due to its promising ability of reducing the dimensionality of the data. Moreover, it contributes to better machine learning models with a higher prediction accuracy and less training time [30]. We employed 5 selection algorithms in our study, which were Distance Correlation (DC), Random Forest (RF), Least Absolute Shrinkage and Selection Operator (LASSO), eXtreme Gradient Boosting (Xgboost) and Gradient Boosting Decision Tree (GBDT). In addition, 9 classification algorithms were adopted on the basis of the selection methods. The 9 classification algorithms were Linear Discriminant Analysis (LDA), Support Vector Machines (SVM), Random Forest (RF), Adaptive Boosting (AdaBoost), K-nearest neighborhood (KNN), Gaussian Naive Bayes (GaussianNB), Logistic Regression (LR), GBDT and Decision Tree (DT). In this way, 45 diagnostic models were built with cross combinations of 5 different feature-selection algorithms and 9 classification algorithms [31]. The included patients were randomly assigned into training and validation datasets at a ratio of 3:1. The models were initially trained in the training dataset and then were validated in the validation dataset. In addition, these two processes were repeated 10-fold to ensure the accuracy and robustness. Sensitivity, specificity and AUC were all calculated in the training and validation datasets. All machine learning processes were programmed in the python programming language (sklearn package). As for clinical data analysis, the chi-square test, independent-sample t-test, Mann–Whitney U-test and Pearson’s correlation coefficient were conducted in the statistical package IBM SPSS Statistics software (Version 26.0, IBM, Armonk, NY, USA).

3. Results

3.1. Patient Characters

A total of 238 patients were included in this study. Among these patients, 156 patients were diagnosed as PDAC and 82 patients were diagnosed as pNET. The gender ratios (male: female) for each subtype were 97:59 and 48: 34, respectively. The mean ages of patients were 59.69 and 53.39 (p < 0.001), respectively. The laboratory examination showed increased glucose levels of PDAC patients (6.83 mmol/L vs. 5.45 mmol/L, p < 0.001). PDAC patients had higher levels of CA19-9 than pNET patients (380.10 U/mL vs. 137.91 U/mL, p < 0.001). The level of TBIL in PDAC patients was significantly higher than that of pNET patients (115.31 umol/L vs. 26.07 umol/L, p < 0.001). There were no significant differences between gender, BMI, calcium, procalcitonin, AFP, CEA, CA12-5, amylase and lipase (p > 0.05). The clinical characters and laboratory indexes of patients are shown in Table 1.

3.2. Radiomics Features

A total of forty-eight features were extracted from six matrixes. The correlation of these features was tested by Pearson’s correlation coefficient. The results suggested that most of the features were independent. Some features were shown to have a positive correlation, such as GLCM_Entropy and HISTO_Entropy (Pearson’s correlation = 0.971), GLZLM_GLNU and GLRLM_GLNU (Pearson’s correlation = 0.923) and GLZLM_LZHGE and GLZLM_LZE (Pearson’s correlation = 0.998). The heat map of correlation among the radiomics features is shown in Figure 2.

3.3. Model Assessment

A total of forty-five models were developed with a combination of five selection algorithms and nine classification algorithms. Radiomics features were introduced into the models. All the discriminative models exhibited a feasible diagnostic ability in the validation dataset. The optimal model in this study was established by the selection algorithm (GBDT) and classification algorithm (RF), which presented the best diagnostic performance with the highest AUC in the training dataset as well as in the validation dataset. This model demonstrated sensitivity, specificity and AUC values in the training dataset of 0.804, 0.973 and 0.971, respectively; in the validation dataset they were 0.742, 0.934 and 0.93, respectively. In the training dataset, the positive and negative prediction values were 0.950 and 0.886, respectively; in the validation dataset, the positive and negative prediction values were 0.898 and 0.822, respectively. The sensitivity, specificity and AUC of the 45 models in the training and validation datasets are summarized in Table 2.
In addition, the combination of RF + SVM and GBDT + SVM showed over-fitting, so we finally excluded them from the model performance comparison. To assess the robustness of the discriminative models, the processes of training and validating were repeated 10 times. The receiver operating characteristic curves (ROCs) of the optimal discriminative model in the cross validation of all folds are shown in Figure 3.

4. Discussion

The incidence and mortality of pancreatic cancer varies in different regions of the world. Globally, pancreatic cancer is the seventh leading cause of cancer-related death and the fourth leading cause of cancer-related death in western countries [3,32]. The possible reasons for this difference we believe include dietary habits, obesity rates, level of economic development and ability to treat cancer. Despite having regional differences in incidence and mortality, pancreatic cancer remains one of the major challenges in the diagnosis and treatment of cancer worldwide. As a major subtype of pancreatic cancer, PDAC has unparalleled research value.
The accurate diagnosis of PDAC and pNET is important because of their different treatments and prognoses. CT is widely used as a noninvasive examination approach of abdominal tumors. However, accurate diagnosis by naked eye assessment is challenging due to the similar radiological features shared by PDAC and pNET. In this study, we extracted quantitative features from CT images and established diagnosis models, which combined radiomics with multiple machine learning algorithms to differentiate PDAC from pNET. In addition, the diagnostic ability of 45 models was investigated, demonstrating stable and outstanding discriminative performances. It is worth noting that the combination of GBDT and RF was preferred for the statistical analysis and presented an optimal discriminative ability.
In the regular statistical analysis of baseline information, differences between some parameters were significant. Our study found that the CA19-9 level of PDAC patients was significantly higher than that of pNET patients, which was consistent with previous studies reporting a significant difference in CA19-9 between PDAC patients and pNET patients [33]. The results of reduced blood glucose levels in pNET could be explained by hyperinsulinemic hypoglycemia [9]. In addition, our results suggested that the differences in bilirubin levels were significant in PADC patients and pNET patients. Other researchers have also mentioned hyperbilirubinaemia due to cholestasis in PDAC, and elevated TBIL was associated with a poorer prognosis in patients with pancreatic cancer [34,35]. Except for biochemical indexes, clinical parameters were associated with cancer type. Our study found a significant difference in the mean age between the two types of tumors, which reached a consensus with previous studies [36]. The results demonstrated that the mean age of PDAC patients was higher than that of pNET patients, suggesting that older patients are more likely to suffer from malignant tumors. In addition, it is generally accepted that PDAC usually occurs in older patients, with some studies suggesting that the median age of PDAC patients at diagnosis is 71 [2]. Comparatively, the mean age of the PDAC patients in this study was extremely younger (59.69 years). The possible reason for this discrepancy may be the trend towards a younger incidence of PDAC. It may also be that advances in diagnostic tools have led to an increase in early PDAC detection rates. Similar results have been reported in recent studies, where the mean age of PDAC patients was also significantly lower [37,38]. However, this finding needs to be treated with caution, as it is only the conclusion of a single-center retrospective study conducted in China from 2013 to 2019.
The current diagnostic methods of pancreatic cancer are biopsy, biomarkers and imaging tools [39]. Cytological examinations of clinically unresectable tumors are crucial for determining prognosis and treatment. However, the procedure of biopsy is associated with complications such as infection and hemorrhage and has a risk of peritoneal seeding [12,40]. Additionally, the pathological diagnosis between PDAC and pNET is sometimes mistaken because both of them demonstrate nuclear atypia, necrosis and a high mitotic rate [41]. Biomarkers have a limited role in differentiating PDAC from pNET [41]. Serum chromogranin A (CgA) is widely used as a biomarker for pNET, but it presents a sensitivity and specificity of 60% and 75%, respectively [42]. Moreover, the majority of PDAC patients have also been found with elevated CgA levels [41]. In terms of glucose, it varies in different subtypes of pNET. Insulinoma, the most common pNET, presents a low blood glucose level, while glucagonoma is characterized by diabetes mellitus [9]. However, it has been reported that 85% of PDAC patients are diagnosed with hyperglycemia due to dysregulation of the glucose metabolism [43]. Therefore, blood glucose level is not sufficient for predicting subtypes of pancreatic cancer. In regard to radiological evaluation, the diagnosis of PDAC and pNET by conventional CT depends on the differences in radiological features, which are induced by tumor growth. Originating from endocrine cells or the pluripotent duct cells of the pancreas, pNET usually appears as a solid avidly enhancing mass [14]. However, in some cases, ductal obstruction, ductal dilatation and up-stream pancreatic atrophy could be caused by related metabolites of pNET [25]. Thus, these atypical characteristics and insufficient radiological features make accurate diagnosis difficult by visual assessment. Diagnosis may be delayed due to non-specific clinical symptoms and the lack of effective diagnostic methods, leading to larger primary tumors or even metastases at the time of initial diagnosis [44].
Radiomics provides more quantitative information beyond the information provided by naked eye assessment, including lesion shape, volume and texture [45]. The processed data are too numerous for radiologists to perform a complete visual assessment. However, it can be further analyzed by machine learning, which has the potential to deal with complex tasks, thus improving diagnostic accuracy, optimizing clinical workflow and decreasing costs and workload [46]. The combination of radiomics and machine learning provides promising radiological evaluation methods. In previous studies, researchers investigated various algorithms and radiomics features of pancreatic lesions. Some researchers applied a few volumetric CT texture features to distinguish PADC from pNET, reporting that the combination of fifth percentile + skewness generates the highest AUC of 0.887, and the corresponding sensitivity and specificity were 0.9 and 0.8, respectively [47]. However, the limited number of texture parameters could not reflect the heterogeneity of tumors. Other researchers developed three models to differentiate nonfunctional neuroendocrine tumor (NF-pNET) from PDAC, including a model based on radiomics signatures alone, one based on clinical parameters alone and another model that integrated both [25]. It impressed us that not only radiomics features but clinical parameters were integrated into the models. The results showed that the AUC of the integrated model hit 0.884, improving on the discriminative ability based on radiomics features and clinical parameters alone. Although their model only used a single algorithm, it showed great discriminative ability. However, the small sample size of this study reduced the reliability of its conclusions. In the diagnosis of PDAC and pNET, our study included more algorithms and showed better a discriminative ability with a higher AUC. This might provide a noninvasive method to distinguish those subtypes of pancreatic cancer with a high specificity and sensitivity. Compared with similar studies, our sample size was relatively larger, which greatly increased the reliability of our research conclusions. In addition to CT, some studies have built differentiation models of PNET and PDAC based on MRI or PET-CT, and these models also have excellent differentiation abilities [26,48,49]. In addition to the small sample size, most of these studies only adopted a single algorithm and perhaps did not select the most suitable algorithm, which made their conclusions limited.
Most of the studies about machine leaning in assisting diagnosis showed the results of single machine leaning method or different combinations of algorithms. For example, a study conducted to distinguish axillary lymph node status reported that LDA classifier presented the highest AUC of 0.81 in a 2D analysis [50]. Another researcher reported that RF showed the highest AUC of 0.968 in distinguishing benign ovarian tumors from epithelial ovarian cancer [51]. A study about the diagnosis of glioblastoma and metastatic brain tumors noted that two models combining DC and LDA and combining DC and LR achieved the highest AUC of 0.80 in the validating dataset [52].
However, for the best discriminative models obtained in different studies, the respective algorithms used are not exactly the same. There is no best answer to the choice of algorithms for studies related to radiomics combined with machine learning.
Though there are numerous algorithms for researchers to choose from, previous studies have employed only a few classifiers for discrimination. Therefore, we included more algorithms to evaluate their discriminative performance and tried to select the optimal algorithm. In our study, five selection algorithms and nine classification algorithms were adopted to determine the suitable discriminative models. Our research found that the combination of GBDT and RF presented the best performance. GBDT, a widely used algorithm in machine learning, receives promising predictability when coping with numerous factors and complicated relations [53]. Other researchers suggested that GBDT showed the highest performance in predicting colorectal cancer compared with RF and SVM, indicating that implementing a GBDT model is time-saving and cost-effective [54]. So, GBDT was applicable for our study because of its excellent prediction and ability to deal with numerical values. RF, a collection of decision trees, can balance data and be employed in classification tasks [55,56]. In a previous study, researchers adopted RF for assisting with Parkinson’s disease, suggesting that RF achieved better classification compared with LR and SVM [57]. Considering the various choices of algorithms and different types of cancer, more attention needs to be paid in feature-selection algorithms and machine-learning classifiers in future investigations to build more reliable models. Though a finite number of open-source algorithms were employed in our research, the results of our study could provide a reference for algorithm selection when predicting subtypes of cancer with machine learning.
There were several limitations in our study. First, this was a retrospective study and therefore bias was inevitable. Additionally, we established the model and tested the performance in the database of our institution, and an external validation set from another institution was required.

5. Conclusions

In conclusion, methods based on enhanced CT radiomics features combined with multiple machine learning algorithms have a promising ability to differentiate PDAC from pNET. In addition, this assistance method is expected to facilitate clinical decision-making and reduce the invasive injury caused by pathological examinations. We also suggest that when similar research is conducted in the future multiple algorithms should be considered in the model rather than empirically using only a single algorithm.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/jcm11226789/s1. Document S1: The details and parameters of CT scanning procedures. All the detailed information about CT equipment and contrast agent and the specific operation details of the scanning process are described in the Supplementary Document S1.

Author Contributions

Conception and design: T.Z., X.W. and H.Z. Administrative support: X.W. and H.Z. Collection and assembly of data: Y.X., H.W. and H.Y. and T.Z. Data analysis and interpretation: T.Z., Y.X., H.W. and Y.L. Manuscript writing: T.Z. Final approval of manuscript: All authors. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and was approved by the Ethics Committee of West China Hospital, Sichuan University (Approval Number 2019-903, 6 November 2019).

Informed Consent Statement

As this study was a retrospective study and did not disclose any personal information, informed consent was waived with the approval of the Institutional Ethics Committee.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Acknowledgments

Thanks to Ting Zhang and Jian Wang.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Bray, F.; Ferlay, J.; Soerjomataram, I.; Siegel, R.L.; Torre, L.A.; Jemal, A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA A Cancer J. Clin. 2018, 68, 394–424. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. Mizrahi, J.D.; Surana, R.; Valle, J.W.; Shroff, R.T. Pancreatic cancer. Lancet 2020, 395, 2008–2020. [Google Scholar] [CrossRef]
  3. Sung, H.; Ferlay, J.; Siegel, R.L.; Laversanne, M.; Soerjomataram, I.; Jemal, A.; Bray, F. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA A Cancer J. Clin. 2021, 71, 209–249. [Google Scholar] [CrossRef] [PubMed]
  4. Luo, G.; Fan, Z.; Gong, Y.; Jin, K.; Yang, C.; Cheng, H.; Huang, D.; Ni, Q.; Liu, C.; Yu, X. Characteristics and Outcomes of Pancreatic Cancer by Histological Subtypes. Pancreas 2019, 48, 817–822. [Google Scholar] [CrossRef] [PubMed]
  5. Bengtsson, A.; Andersson, R.; Ansari, D. The actual 5-year survivors of pancreatic ductal adenocarcinoma based on real-world data. Sci. Rep. 2020, 10, 16425. [Google Scholar] [CrossRef]
  6. Sarantis, P.; Koustas, E.; Papadimitropoulou, A.; Papavassiliou, A.G.; Karamouzis, M.V. Pancreatic ductal adenocarcinoma: Treatment hurdles, tumor microenvironment and immunotherapy. World J. Gastrointest. Oncol. 2020, 12, 173–181. [Google Scholar] [CrossRef]
  7. Somani, V.K.; Zhang, D.; Dodhiawala, P.B.; Lander, V.E.; Liu, X.; Kang, L.I.; Chen, H.P.; Knolhoff, B.L.; Li, L.; Grierson, P.M.; et al. IRAK4 Signaling Drives Resistance to Checkpoint Immunotherapy in Pancreatic Ductal Adenocarcinoma. Gastroenterology 2022, 162, 2047–2062. [Google Scholar] [CrossRef]
  8. Yadav, S.; Sharma, P.; Zakalik, D. Comparison of Demographics, Tumor Characteristics, and Survival Between Pancreatic Adenocarcinomas and Pancreatic Neuroendocrine Tumors: A Population-based Study. Am. J. Clin. Oncol. 2018, 41, 485–491. [Google Scholar] [CrossRef]
  9. Ro, C.; Chai, W.; Yu, V.E.; Yu, R. Pancreatic neuroendocrine tumors: Biology, diagnosis, and treatment. Chin. J. Cancer 2013, 32, 312–324. [Google Scholar] [CrossRef] [Green Version]
  10. Cejas, P.; Drier, Y.; Dreijerink, K.M.A.; Brosens, L.A.A.; Deshpande, V.; Epstein, C.B.; Conemans, E.B.; Morsink, F.H.M.; Graham, M.K.; Valk, G.D.; et al. Enhancer signatures stratify and predict outcomes of non-functional pancreatic neuroendocrine tumors. Nat. Med. 2019, 25, 1260–1265. [Google Scholar] [CrossRef]
  11. O’Grady, H.L.; Conlon, K.C. Pancreatic neuroendocrine tumours. Eur. J. Surg. Oncol. 2008, 34, 324–332. [Google Scholar] [CrossRef] [PubMed]
  12. Tempero, M.A.; Malafa, M.P.; Al-Hawary, M.; Asbun, H.; Bain, A.; Behrman, S.W.; Benson, A.B., 3rd; Binder, E.; Cardin, D.B.; Cha, C.; et al. Pancreatic Adenocarcinoma, Version 2.2017, NCCN Clinical Practice Guidelines in Oncology. J. Natl. Compr. Cancer Netw. JNCCN 2017, 15, 1028–1061. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Kimura, T.; Miyamoto, H.; Fukuya, A.; Kitamura, S.; Okamoto, K.; Kimura, M.; Muguruma, N.; Ikemoto, T.; Shimada, M.; Yoneda, A.; et al. Neuroendocrine carcinoma of the pancreas with similar genetic alterations to invasive ductal adenocarcinoma. Clin. J. Gastroenterol. 2016, 9, 261–265. [Google Scholar] [CrossRef] [PubMed]
  14. Raman, S.P.; Hruban, R.H.; Cameron, J.L.; Wolfgang, C.L.; Fishman, E.K. Pancreatic imaging mimics: Part 2, pancreatic neuroendocrine tumors and their mimics. AJR Am. J. Roentgenol. 2012, 199, 309–318. [Google Scholar] [CrossRef]
  15. Kumar, V.; Gu, Y.; Basu, S.; Berglund, A.; Eschrich, S.A.; Schabath, M.B.; Forster, K.; Aerts, H.J.; Dekker, A.; Fenstermacher, D.; et al. Radiomics: The process and the challenges. Magn. Reson. Imaging 2012, 30, 1234–1248. [Google Scholar] [CrossRef] [Green Version]
  16. Lambin, P.; Leijenaar, R.T.H.; Deist, T.M.; Peerlings, J.; de Jong, E.E.C.; van Timmeren, J.; Sanduleanu, S.; Larue, R.; Even, A.J.G.; Jochems, A.; et al. Radiomics: The bridge between medical imaging and personalized medicine. Nat. Rev. Clin. Oncol. 2017, 14, 749–762. [Google Scholar] [CrossRef] [Green Version]
  17. Rogers, W.; Thulasi Seetha, S.; Refaee, T.A.G.; Lieverse, R.I.Y.; Granzier, R.W.Y.; Ibrahim, A.; Keek, S.A.; Sanduleanu, S.; Primakov, S.P.; Beuque, M.P.L.; et al. Radiomics: From qualitative to quantitative imaging. Br. J. Radiol. 2020, 93, 20190948. [Google Scholar] [CrossRef]
  18. Machine learning in cancer diagnostics. EBioMedicine 2019, 45, 1–2. [CrossRef] [Green Version]
  19. Hyun, S.H.; Ahn, M.S.; Koh, Y.W.; Lee, S.J. A Machine-Learning Approach Using PET-Based Radiomics to Predict the Histological Subtypes of Lung Cancer. Clin. Nucl. Med. 2019, 44, 956–960. [Google Scholar] [CrossRef]
  20. Park, Y.W.; Oh, J.; You, S.C.; Han, K.; Ahn, S.S.; Choi, Y.S.; Chang, J.H.; Kim, S.H.; Lee, S.K. Radiomics and machine learning may accurately predict the grade and histological subtype in meningiomas using conventional and diffusion tensor imaging. Eur. Radiol. 2019, 29, 4068–4076. [Google Scholar] [CrossRef]
  21. Sun, W.; Jiang, M.; Dang, J.; Chang, P.; Yin, F.F. Effect of machine learning methods on predicting NSCLC overall survival time based on Radiomics analysis. Radiat. Oncol. 2018, 13, 197. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  22. Park, S.; Chu, L.C.; Hruban, R.H.; Vogelstein, B.; Kinzler, K.W.; Yuille, A.L.; Fouladi, D.F.; Shayesteh, S.; Ghandili, S.; Wolfgang, C.L.; et al. Differentiating autoimmune pancreatitis from pancreatic ductal adenocarcinoma with CT radiomics features. Diagn. Interv. Imaging 2020, 101, 555–564. [Google Scholar] [CrossRef] [PubMed]
  23. Dalal, V.; Carmicheal, J.; Dhaliwal, A.; Jain, M.; Kaur, S.; Batra, S.K. Radiomics in stratification of pancreatic cystic lesions: Machine learning in action. Cancer Lett. 2020, 469, 228–237. [Google Scholar] [CrossRef] [PubMed]
  24. Yang, J.; Guo, X.; Ou, X.; Zhang, W.; Ma, X. Discrimination of Pancreatic Serous Cystadenomas From Mucinous Cystadenomas With CT Textural Features: Based on Machine Learning. Front. Oncol. 2019, 9, 494. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  25. He, M.; Liu, Z.; Lin, Y.; Wan, J.; Li, J.; Xu, K.; Wang, Y.; Jin, Z.; Tian, J.; Xue, H. Differentiation of atypical non-functional pancreatic neuroendocrine tumor and pancreatic ductal adenocarcinoma using CT based radiomics. Eur. J. Radiol. 2019, 117, 102–111. [Google Scholar] [CrossRef] [PubMed]
  26. Xu, J.; Yang, J.; Feng, Y.; Zhang, J.; Zhang, Y.; Chang, S.; Jin, J.; Du, X. MRI Feature-Based Nomogram Model for Discrimination Between Non-Hypervascular Pancreatic Neuroendocrine Tumors and Pancreatic Ductal Adenocarcinomas. Front. Oncol. 2022, 12, 856306. [Google Scholar] [CrossRef]
  27. Yu, H.; Huang, Z.; Li, M.; Wei, Y.; Zhang, L.; Yang, C.; Zhang, Y.; Song, B. Differential Diagnosis of Nonhypervascular Pancreatic Neuroendocrine Neoplasms From Pancreatic Ductal Adenocarcinomas, Based on Computed Tomography Radiological Features and Texture Analysis. Acad. Radiol. 2020, 27, 332–341. [Google Scholar] [CrossRef]
  28. Al-Hawary, M.M.; Francis, I.R.; Chari, S.T.; Fishman, E.K.; Hough, D.M.; Lu, D.S.; Macari, M.; Megibow, A.J.; Miller, F.H.; Mortele, K.J.; et al. Pancreatic ductal adenocarcinoma radiology reporting template: Consensus statement of the society of abdominal radiology and the american pancreatic association. Gastroenterology 2014, 146, 291–304.e291. [Google Scholar] [CrossRef]
  29. Nioche, C.; Orlhac, F.; Boughdad, S.; Reuzé, S.; Goya-Outi, J.; Robert, C.; Pellot-Barakat, C.; Soussan, M.; Frouin, F.; Buvat, I. LIFEx: A Freeware for Radiomic Feature Calculation in Multimodality Imaging to Accelerate Advances in the Characterization of Tumor Heterogeneity. Cancer Res. 2018, 78, 4786–4789. [Google Scholar] [CrossRef] [Green Version]
  30. Savic, M.; Kurbalija, V.; Ivanovic, M.; Bosnic, Z. A Feature Selection Method Based on Feature Correlation Networks. In Proceedings of the 7th International Conference on Model and Data Engineering (MEDI), Barcelona, Spain, 4–6 October 2017; pp. 248–261. [Google Scholar]
  31. Zhang, Y.; Chen, C.; Cheng, Y.; Teng, Y.; Guo, W.; Xu, H.; Ou, X.; Wang, J.; Li, H.; Ma, X.; et al. Ability of Radiomics in Differentiation of Anaplastic Oligodendroglioma from Atypical Low-Grade Oligodendroglioma Using Machine-Learning Approach. Front. Oncol. 2019, 9, 1371. [Google Scholar] [CrossRef]
  32. Siegel, R.L.; Miller, K.D.; Fuchs, H.E.; Jemal, A. Cancer Statistics, 2021. CA A Cancer J. Clin. 2021, 71, 7–33. [Google Scholar] [CrossRef] [PubMed]
  33. Zhuge, X.; Guo, C.; Chen, Y.; Feng, L.; Jia, R.; Zhao, Y.; Sun, K.; Wang, Z.; Chen, X. The Levels of Tumor Markers in Pancreatic Neuroendocrine Carcinoma and Their Values in Differentiation Between Pancreatic Neuroendocrine Carcinoma and Pancreatic Ductal Adenocarcinoma. Pancreas 2018, 47, 1290–1295. [Google Scholar] [CrossRef] [PubMed]
  34. Álvarez, R.; Carrato, A.; Adeva, J.; Alés, I.; Prados, S.; Valladares, M.; Macarulla, T.; Muñoz, A.; Hidalgo, M. Management of hyperbilirubinaemia in pancreatic cancer patients. Eur. J. Cancer 2018, 94, 26–36. [Google Scholar] [CrossRef] [PubMed]
  35. Zhang, S.; Huang, X.; Tian, Y.; Aimaiti, S.; Zhang, J.; Zhao, J.; Chen, Y.; Wang, C. Clinicopathologic characteristics, laboratory parameters, treatment protocols, and outcomes of pancreatic cancer: A retrospective cohort study of 1433 patients in China. PeerJ 2018, 6, e4893. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  36. Guo, C.; Zhuge, X.; Wang, Q.; Xiao, W.; Wang, Z.; Wang, Z.; Feng, Z.; Chen, X. The differentiation of pancreatic neuroendocrine carcinoma from pancreatic ductal adenocarcinoma: The values of CT imaging features and texture analysis. Cancer Imaging 2018, 18, 37. [Google Scholar] [CrossRef] [Green Version]
  37. Park, S.; Sham, J.G.; Kawamoto, S.; Blair, A.B.; Rozich, N.; Fouladi, D.F.; Shayesteh, S.; Hruban, R.H.; He, J.; Wolfgang, C.L.; et al. CT Radiomics-Based Preoperative Survival Prediction in Patients With Pancreatic Ductal Adenocarcinoma. AJR Am. J. Roentgenol. 2021, 217, 1104–1112. [Google Scholar] [CrossRef]
  38. Meng, Y.; Zhang, H.; Li, Q.; Liu, F.; Fang, X.; Li, J.; Yu, J.; Feng, X.; Zhu, M.; Li, N.; et al. CT Radiomics and Machine-Learning Models for Predicting Tumor-Stroma Ratio in Patients With Pancreatic Ductal Adenocarcinoma. Front. Oncol. 2021, 11, 707288. [Google Scholar] [CrossRef]
  39. Chu, L.C.; Goggins, M.G.; Fishman, E.K. Diagnosis and Detection of Pancreatic Cancer. Cancer J. 2017, 23, 333–342. [Google Scholar] [CrossRef]
  40. Ryozawa, S.; Kitoh, H.; Gondo, T.; Urayama, N.; Yamashita, H.; Ozawa, H.; Yanai, H.; Okita, K. Usefulness of endoscopic ultrasound-guided fine-needle aspiration biopsy for the diagnosis of pancreatic cancer. J. Gastroenterol. 2005, 40, 907–911. [Google Scholar] [CrossRef]
  41. Gao, H.L.; Wang, W.Q.; Yu, X.J.; Liu, L. Molecular drivers and cells of origin in pancreatic ductal adenocarcinoma and pancreatic neuroendocrine carcinoma. Exp. Hematol. Oncol. 2020, 9, 28. [Google Scholar] [CrossRef]
  42. Pulvirenti, A.; Rao, D.; McIntyre, C.A.; Gonen, M.; Tang, L.H.; Klimstra, D.S.; Fleisher, M.; Ramanathan, L.V.; Reidy-Lagunes, D.; Allen, P.J. Limited role of Chromogranin A as clinical biomarker for pancreatic neuroendocrine tumors. HPB 2019, 21, 612–618. [Google Scholar] [CrossRef] [PubMed]
  43. Goldberg, R.; Meirovitz, A.; Abecassis, A.; Hermano, E.; Rubinstein, A.M.; Nahmias, D.; Grinshpun, A.; Peretz, T.; Elkin, M. Regulation of Heparanase in Diabetes-Associated Pancreatic Carcinoma. Front. Oncol. 2019, 9, 1405. [Google Scholar] [CrossRef] [PubMed]
  44. Wang, W.Z.; Xu, L.J.; Hu, D.M.; Tang, W. Case of an abnormal procalcitonin-producing metastatic pancreatic neuroendocrine tumor. Clin. Case Rep. 2020, 8, 2269–2272. [Google Scholar] [CrossRef] [PubMed]
  45. Hassani, C.; Varghese, B.A.; Nieva, J.; Duddalwar, V. Radiomics in Pulmonary Lesion Imaging. AJR Am. J. Roentgenol. 2019, 212, 497–504. [Google Scholar] [CrossRef]
  46. Goldenberg, S.L.; Nir, G.; Salcudean, S.E. A new era: Artificial intelligence and machine learning in prostate cancer. Nat. Rev. Urol. 2019, 16, 391–403. [Google Scholar] [CrossRef]
  47. Li, J.; Lu, J.; Liang, P.; Li, A.; Hu, Y.; Shen, Y.; Hu, D.; Li, Z. Differentiation of atypical pancreatic neuroendocrine tumors from pancreatic ductal adenocarcinomas: Using whole-tumor CT texture analysis as quantitative biomarkers. Cancer Med. 2018, 7, 4924–4931. [Google Scholar] [CrossRef]
  48. Yoo, J.; Kim, S.H.; Jeon, S.K.; Bae, J.S.; Han, J.K. Added value of [(68)Ga]Ga-DOTA-TOC PET/CT for characterizing pancreatic neuroendocrine neoplasms: A comparison with contrast-enhanced CT and/or MRI in a large study cohort. Eur. Radiol. 2021, 31, 7734–7745. [Google Scholar] [CrossRef]
  49. Li, J.; Liang, L.; Yu, H.; Shen, Y.; Hu, Y.; Hu, D.; Tang, H.; Li, Z. Whole-tumor histogram analysis of non-Gaussian distribution DWI parameters to differentiation of pancreatic neuroendocrine tumors from pancreatic ductal adenocarcinomas. Magn. Reson. Imaging 2019, 55, 52–59. [Google Scholar] [CrossRef]
  50. Arefan, D.; Chai, R.; Sun, M.; Zuley, M.L.; Wu, S. Machine learning prediction of axillary lymph node metastasis in breast cancer: 2D versus 3D radiomic features. Med. Phys. 2020, 47, 6334–6342. [Google Scholar] [CrossRef]
  51. Kawakami, E.; Tabata, J.; Yanaihara, N.; Ishikawa, T.; Koseki, K.; Iida, Y.; Saito, M.; Komazaki, H.; Shapiro, J.S.; Goto, C.; et al. Application of Artificial Intelligence for Preoperative Diagnostic and Prognostic Prediction in Epithelial Ovarian Cancer Based on Blood Biomarkers. Clin. Cancer Res. 2019, 25, 3006–3015. [Google Scholar] [CrossRef]
  52. Chen, C.; Ou, X.; Wang, J.; Guo, W.; Ma, X. Radiomics-Based Machine Learning in Differentiation Between Glioblastoma and Metastatic Brain Tumors. Front. Oncol. 2019, 9, 806. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  53. Zhu, S.-L.; Dong, J.; Zhang, C.; Huang, Y.-B.; Pan, W. Application of machine learning in the diagnosis of gastric cancer based on noninvasive characteristics. PLoS ONE 2020, 15, e0244869. [Google Scholar] [CrossRef] [PubMed]
  54. Gao, J.; Zhang, L.; Yu, G.; Qu, G.; Li, Y.; Yang, X. Model with the GBDT for Colorectal Adenoma Risk Diagnosis. Curr. Bioinform. 2020, 15, 971–979. [Google Scholar] [CrossRef]
  55. Al-Quraishi, T.; Abawajy, J.H.; Chowdhury, M.U.; Rajasegarar, S.; Abdalrada, A.S. Breast Cancer Recurrence Prediction Using Random Forest Model. In Proceedings of the 3rd International Conference on Soft Computing and Data Mining (SCDM), Johor, Malaysia, 6–7 February 2018; pp. 318–329. [Google Scholar]
  56. Okun, O.; Priisalu, H. Random forest for gene expression based cancer classification: Overlooked issues. In Proceedings of the 3rd Iberian Conference on Pattern Recognition and Image Analysis, Girona, Spain, 6–8 June 2007; pp. 483–490. [Google Scholar]
  57. Xu, S.; Pan, Z. A novel ensemble of random forest for assisting diagnosis of Parkinson’s disease on small handwritten dynamics dataset. Int. J. Med. Inform. 2020, 144, 104283. [Google Scholar] [CrossRef] [PubMed]
Jcm 11 06789 g001 550
Figure 1. ROI manually drawn from two cases in CT images. (A) Example of a 51 year old male with PDAC with lesion in pancreatic head. (B) Example of a 60 year old female with pNET with lesion in pancreatic body. The pink areas in the figure are the delineated ROIs.
Figure 1. ROI manually drawn from two cases in CT images. (A) Example of a 51 year old male with PDAC with lesion in pancreatic head. (B) Example of a 60 year old female with pNET with lesion in pancreatic body. The pink areas in the figure are the delineated ROIs.
Jcm 11 06789 g001
Jcm 11 06789 g002 550
Figure 2. The heat map of Pearson’s correlation coefficients among radiomics features.
Figure 2. The heat map of Pearson’s correlation coefficients among radiomics features.
Jcm 11 06789 g002
Jcm 11 06789 g003 550
Figure 3. The AUC of optimal discriminative model.
Figure 3. The AUC of optimal discriminative model.
Jcm 11 06789 g003
Table
Table 1. Characteristics of patients.
Table 1. Characteristics of patients.
CharacteristicsPDAC (n = 156)pNET (n = 82)p Value
Male gender, n (%)97 (62.2%)48 (58.5%)0.307
Mean age (year)59.69 ± 11.5753.39 ± 13.01<0.001 *
BMI (kg/m2)25.26 ± 5.6721.64 ± 4.250.701
Glucose (mmol/L)6.83 ± 3.47 (n = 137)5.45 ± 3.39 (n = 76)<0.001 *
Procalcitonin (ng/mL)1.06 ± 2.22 (n = 115)0.92 ± 1.18 (n = 58)0.812
AFP (ng/mL)3.90 ± 8.58 (n = 151)3.114 ± 1.70 (n = 68)0.434
CEA (U/mL)6.24 ± 10.01 (n = 153)3.78 ± 6.68 (n = 77)0.092
CA19-9 (U/mL)380.10 ± 380.60 (n = 155)137.91 ± 282.90 (n = 79)<0.001 *
CA125 (U/mL)48.96 ± 49.41 (n = 134)20.54 ± 15.04 (n = 66)0.184
TBIL (umol/L)115.31 ± 144.51 (n = 156)26.07 ± 51.74 (n = 82)<0.001*
Amylase (IU/L)111.64 ± 107.14 (n = 140)225.13 ± 651.94 (n = 70)0.208
Lipase (IU/L)183.38 ± 270.75 (n = 127)309.11 ± 1058.07 (n = 65)0.312
* p < 0.05; PDAC, pancreatic ductal adenocarcinoma; pNET, pancreatic neuroendocrine tumor; BMI, body mass index; AFP, α-fetoprotein; CEA, carcinoembryonic antigen; CA19-9, glucoprotein antigen 199; CA125, glucoprotein antigen 125; TBIL, total bilirubin.
Table
Table 2. Discriminative performance of models.
Table 2. Discriminative performance of models.
Training DatasetValidation Dataset
SensitivitySpecificityAUCSensitivitySpecificityAUC
DC + LDA0.8390.9030.9530.7750.8840.938
DC + SVM0.9720.5630.9310.9870.5280.910
DC + RF0.7300.9630.9350.6530.9290.879
DC + AdaBoost1.0001.0001.0000.7530.9050.919
DC + KNN0.8340.9240.9610.7690.8720.884
DC + GaussianNB0.8100.8510.9220.7810.7960.896
DC + LR0.7590.9030.9260.7030.8630.901
DC + GBDT1.0001.0001.0000.7580.8790.918
DC + DT1.0001.0001.0000.7930.8410.817
RF + LDA0.7970.950.9540.7750.9290.927
RF + SVM1.001.00-0.001.00-
RF + RF0.7450.9740.9570.6920.9310.902
RF + AdaBoost1.0001.0001.0000.7590.9120.921
RF + KNN0.6080.9170.8860.3950.8300.707
RF + GaussianNB0.6320.9560.9170.6040.9210.883
RF + LR0.7960.8970.9480.8160.8780.944
RF + GBDT1.0001.0001.0000.7460.8840.927
RF + DT1.0001.0001.0000.7320.8320.782
LASSO + LDA0.6720.9160.8780.6330.8830.811
LASSO + SVM0.7180.8460.8140.7120.8030.784
LASSO + RF0.7090.9290.9170.6260.8590.847
LASSO + AdaBoost1.0001.0001.0000.6600.8520.828
LASSO + KNN0.6590.9550.9150.6010.8940.796
LASSO + GaussianNB0.3280.9610.8200.3570.9290.766
LASSO + LR0.3640.9860.8190.3400.9630.788
LASSO + GBDT1.0001.0001.0000.6770.8330.849
LASSO + DT1.0001.0001.0000.6630.8420.753
Xgboost + LDA0.8080.8810.9450.8170.8760.943
Xgboost + SVM0.9140.7610.9390.9360.7130.951
Xgboost + RF0.7500.9800.9600.7110.9680.925
Xgboost + AdaBoost1.0001.0001.0000.8210.9160.932
Xgboost + KNN0.8040.9410.9670.7770.9280.940
Xgboost + GaussianNB0.8480.8010.9180.8400.7390.896
Xgboost + LR0.8150.8660.9380.8510.8550.948
Xgboost + GBDT1.0001.0001.0000.7580.9250.931
Xgboost + DT1.0001.0001.0000.7830.8680.825
GBDT + LDA0.8610.9140.9660.8270.8860.945
GBDT + SVM1.0001.000-0.0001.000-
GBDT + RF0.8040.9730.9710.7420.9340.930
GBDT + AdaBoost1.0001.0001.0000.7990.8900.929
GBDT + KNN0.6200.9420.8810.4290.8660.730
GBDT + GaussianNB0.3290.9540.9070.3730.9280.884
GBDT + LR0.8090.9200.9530.7820.8820.927
GBDT + GBDT1.0001.0001.0000.7460.8930.927
GBDT + DT1.0001.0001.0000.7350.8280.781
Abbreviations: AUC, area under the receiver operating characteristic curve; DC, Distance Correlation; RF, Random Forest; LASSO, Least Absolute Shrinkage and Selection Operator; Xgboost, eXtreme Gradient Boosting; GBDT, Gradient Boosting Decision Tree; LDA, Linear Discriminant Analysis; SVM, Support Vector Machines; AdaBoost, Adaptive Boosting; KNN, K-nearest neighborhood; GaussianNB, Gaussian Naive Bayes; LR, Logistic Regression; DT, Decision Tree—means that the model is over-fitting, so the results are not displayed.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Zhang, T.; Xiang, Y.; Wang, H.; Yun, H.; Liu, Y.; Wang, X.; Zhang, H. Radiomics Combined with Multiple Machine Learning Algorithms in Differentiating Pancreatic Ductal Adenocarcinoma from Pancreatic Neuroendocrine Tumor: More Hands Produce a Stronger Flame. J. Clin. Med. 2022, 11, 6789. https://doi.org/10.3390/jcm11226789

AMA Style

Zhang T, Xiang Y, Wang H, Yun H, Liu Y, Wang X, Zhang H. Radiomics Combined with Multiple Machine Learning Algorithms in Differentiating Pancreatic Ductal Adenocarcinoma from Pancreatic Neuroendocrine Tumor: More Hands Produce a Stronger Flame. Journal of Clinical Medicine. 2022; 11(22):6789. https://doi.org/10.3390/jcm11226789

Chicago/Turabian Style

Zhang, Tao, Yu Xiang, Hang Wang, Hong Yun, Yichun Liu, Xing Wang, and Hao Zhang. 2022. "Radiomics Combined with Multiple Machine Learning Algorithms in Differentiating Pancreatic Ductal Adenocarcinoma from Pancreatic Neuroendocrine Tumor: More Hands Produce a Stronger Flame" Journal of Clinical Medicine 11, no. 22: 6789. https://doi.org/10.3390/jcm11226789

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop