Radiomics Combined with Multiple Machine Learning Algorithms in Differentiating Pancreatic Ductal Adenocarcinoma from Pancreatic Neuroendocrine Tumor: More Hands Produce a Stronger Flame

The aim of this study was to assess the diagnostic ability of radiomics combined with multiple machine learning algorithms to differentiate pancreatic ductal adenocarcinoma (PDAC) from pancreatic neuroendocrine tumor (pNET). This retrospective study included a total of 238 patients diagnosed with PDAC or pNET. Using specialized software, radiologists manually mapped regions of interest (ROIs) from computed tomography images and automatically extracted radiomics features. A total of 45 discriminative models were built by five selection algorithms and nine classification algorithms. The performances of the discriminative models were assessed by sensitivity, specificity and the area under receiver operating characteristic curve (AUC) in the training and validation datasets. Using the combination of Gradient Boosting Decision Tree (GBDT) as the selection algorithm and Random Forest (RF) as the classification algorithm, the optimal diagnostic ability with the highest AUC was presented in the training and validation datasets. The sensitivity, specificity and AUC of the model were 0.804, 0.973 and 0.971 in the training dataset and 0.742, 0.934 and 0.930 in the validation dataset, respectively. The combination of radiomics and multiple machine learning algorithms showed the potential ability to discriminate PDAC from pNET. We suggest that multi-algorithm modeling should be considered for similar studies in the future rather than using a single algorithm empirically.


Introduction
Pancreatic cancer is one of the most common malignancies with an increasing incidence and is the seventh leading cause of cancer death due to poor prognosis [1][2][3]. Based on its origins and pathology, pancreatic cancer has been classified into several subtypes, including pancreatic ductal adenocarcinoma (PDAC) and pancreatic neuroendocrine tumor (pNET). Among them, PDAC, accounting for 90% of all primary pancreatic malignancies, shows aggressive behavior and poor prognosis, which leads to a five-year survival rate of less than 5% [4,5]. It can only be cured by surgical resection, which is available merely to a small number of patients whose tumors can be surgically removed before they progress to the advanced stage [2]. Unfortunately, most patients tend to be diagnosed with metastasis, in which case chemotherapy and radiotherapy do not significantly improve survival rates [6]. Even immunotherapy, which has shown success in other tumors in recent years, is ineffective in treating PDAC [7]. In contrast, pNET is not as common as PDAC and J. Clin. Med. 2022, 11, 6789 2 of 12 carries a significantly better prognosis with a five-year survival rate of 51.3% [8]. Pancreatic neuroendocrine tumors are classified into functioning or non-functioning according to clinical symptoms. The tumor cells in functioning pNET can secrete a variety of hormones, such as glucagon, insulin and gastrin, which can lead to symptoms of hormone hypersecretion [9]. Non-functioning pNET does not produce excess hormones but has a wide range of clinical behaviors [10]. The treatment strategy for pNET includes surgical resection and chemotherapy, while hormone therapy is controversial. Contrary to PDAC, pNET patients can benefit from early treatment and long-term survival can be expected even with metastases [11]. Therefore, the importance of an early diagnosis of PDAC and pNET cannot be overstated due to their different prognosis and therapeutic strategies.
Magnetic resonance imaging (MRI) and computed tomography (CT) are widely used in detecting abdominal lesions and lymph node metastases. However, MRI has some limitations compared with CT, such as it being time-consuming and costly, and patients experience noise and isolation during the scan. CT is considered as an economical radiological exam to detect pancreatic tumors [12]. Nevertheless, the CT characteristics of PDAC and pNET are quite similar, which makes it easy to lead to misdiagnosis by visual assessment [13,14]. Consequently, it is necessary to develop more feasible methods to assist in diagnosis and improve accuracy. The emergence of radiomics has shown promising prospects in the domain of radiological evaluation. Radiomics refers to the high-throughput extraction of quantitative image features from medical imaging followed by data analysis to support clinical decision-making [15][16][17]. At the same time, new methods for analyzing mineable radiomics feature are required. Machine learning is a major branch of artificial intelligence and is defined as the ability of a machine to learn and predict future events and outcomes based on large datasets [18]. The combination of radiomics and machine learning has presented promising performance in previous research, including predicting histological subtypes of lung cancer, the grade of meningioma and survival outcomes in non-small-cell lung cancer [19][20][21]. Moreover, it has also been prominent in the diagnosis of pancreatic lesions such as pseudocysts, serous cystadenomas, autoimmune pancreatitis and PDAC [22][23][24]. Previous studies have explored the ability to differentiate PDAC from pNET by purely textural features based on CT or MRI in relatively small sample sizes [25,26].
Although radiomics and machine learning have been widely used in disease diagnosis, the application of radiomics and multiple machine learning algorithms combined in discriminating PDAC and pNET has not been reported. The aim of this study was to evaluate the diagnostic ability of radiomics features combined with multiple machine learning algorithms in differentiating PDAC from pNET.

Patient Selection
We retrospectively viewed our database to search for patients diagnosed with PDAC or pNET in West China Hospital with detailed medical records from August 2013 to May 2019. The eligibility criteria for the inclusion of selected patients were: (1) with complete medical records; (2) with a pathological diagnosis confirmation of PDAC or pNET; (3) with high-quality CT images before surgery. Exclusion criteria were: (1) history of treatment before CT scan, such as chemotherapy and radiotherapy; (2) history of other cancers. A total of 238 patients with PDAC or pNET met the inclusion criteria in the initial selection. Then, we collected the information of these patients, including gender, age, body mass index (BMI), glucose, calcium, procalcitonin, α-fetoprotein (AFP), carcinoembryonic antigen (CEA), glucoprotein antigen 199 (CA19-9), glucoprotein antigen 125 (CA12-5), total bilirubin (TBIL), amylase and lipase. This study was approved by the Ethics Committee of West China Hospital, Sichuan University, and all patients' informed consents were waived.

Image Acquisition and Texture Feature Extraction
All selected patients underwent CT examinations, the operating parameters of which were reported in previous studies [27]. The procedures of scanning are summarized in the Supplementary Materials. The CT diagnostic criteria for pancreatic cancer are divided into three parts: morphologic evaluation, vascular evaluation and extrapancreatic evaluation [28]. Morphologic evaluation includes appearance (confined masses of the pancreas are less dense than normal pancreas), size, location (most commonly PDAC in the head of the pancreas and neuroendocrine elsewhere), pancreatic duct narrowing/abrupt cut-off with or without upstream dilatation, and biliary tree abrupt cut-off with or without upstream dilatation. Vascular evaluation can be divided into arterial evaluation and venous evaluation. Arterial evaluation includes the evaluation of the celiac axis, superior mesenteric artery (SMA), common hepatic artery (CHA) and arterial variants. Venous evaluation includes the evaluation of the main portal vein (MPV), superior mesenteric vein (SMV), venous collaterals and intravenous thrombosis. Extrapancreatic evaluation includes the evaluation of liver metastases, peritoneal nodes, suspicious lymph nodes in the abdomen and ascites. PDAC is considered if the above criteria for pancreatic cancer are met, otherwise pNET is suspected, but the final diagnosis is based on pathological results. Recognizable CT images with clear boundaries were acquired from the picture archiving and communication systems in the radiology department. Two radiologists with pancreatic expertise extracted the texture features from CT images using the Local Image Features Extraction (LIFEx) software (v3.74, CEA-SHFJ, Orsay, France) [29]. In the guidelines of the software, ROI was manually drawn along the border of tumor issue slice by slice to gain the 3D features ( Figure 1). Any differences in ROIs were recorded and discussed. The preferred ROI of each patient was selected by the third senior radiologist and was included in our study. All ROI data from the enhanced CT were labeled as PDAC or pNET according to the pathological results. Finally, a total of 48 radiomics features were automatically exported and saved through the LIFEx software.
the Ethics Committee of West China Hospital, Sichuan University, and all patients' informed consents were waived.

Image Acquisition and Texture Feature Extraction
All selected patients underwent CT examinations, the operating parameters of which were reported in previous studies [27]. The procedures of scanning are summarized in the Supplementary Materials. The CT diagnostic criteria for pancreatic cancer are divided into three parts: morphologic evaluation, vascular evaluation and extrapancreatic evaluation [28]. Morphologic evaluation includes appearance (confined masses of the pancreas are less dense than normal pancreas), size, location (most commonly PDAC in the head of the pancreas and neuroendocrine elsewhere), pancreatic duct narrowing/abrupt cut-off with or without upstream dilatation, and biliary tree abrupt cut-off with or without upstream dilatation. Vascular evaluation can be divided into arterial evaluation and venous evaluation. Arterial evaluation includes the evaluation of the celiac axis, superior mesenteric artery (SMA), common hepatic artery (CHA) and arterial variants. Venous evaluation includes the evaluation of the main portal vein (MPV), superior mesenteric vein (SMV), venous collaterals and intravenous thrombosis. Extrapancreatic evaluation includes the evaluation of liver metastases, peritoneal nodes, suspicious lymph nodes in the abdomen and ascites. PDAC is considered if the above criteria for pancreatic cancer are met, otherwise pNET is suspected, but the final diagnosis is based on pathological results. Recognizable CT images with clear boundaries were acquired from the picture archiving and communication systems in the radiology department. Two radiologists with pancreatic expertise extracted the texture features from CT images using the Local Image Features Extraction (LIFEx) software (v3.74, CEA-SHFJ, Orsay, France) [29]. In the guidelines of the software, ROI was manually drawn along the border of tumor issue slice by slice to gain the 3D features ( Figure 1). Any differences in ROIs were recorded and discussed. The preferred ROI of each patient was selected by the third senior radiologist and was included in our study. All ROI data from the enhanced CT were labeled as PDAC or pNET according to the pathological results. Finally, a total of 48 radiomics features were automatically exported and saved through the LIFEx software.

Discriminative Model Establishment and Data Analysis
The selection algorithm is considered as an important data preprocessing step in machine learning tasks, due to its promising ability of reducing the dimensionality of the data. Moreover, it contributes to better machine learning models with a higher prediction accuracy and less training time [30]. We employed 5 selection algorithms in our

Discriminative Model Establishment and Data Analysis
The selection algorithm is considered as an important data preprocessing step in machine learning tasks, due to its promising ability of reducing the dimensionality of the data. Moreover, it contributes to better machine learning models with a higher prediction accuracy and less training time [30]. We employed 5 selection algorithms in our study, which were Distance Correlation (DC), Random Forest (RF), Least Absolute Shrinkage and Selection Operator (LASSO), eXtreme Gradient Boosting (Xgboost) and Gradient Boosting Decision Tree (GBDT). In addition, 9 classification algorithms were adopted on the basis of the selection methods. The 9 classification algorithms were Linear Discriminant Analysis (LDA), Support Vector Machines (SVM), Random Forest (RF), Adaptive Boosting (AdaBoost), K-nearest neighborhood (KNN), Gaussian Naive Bayes (GaussianNB), Logistic Regression (LR), GBDT and Decision Tree (DT). In this way, 45 diagnostic models were built with cross combinations of 5 different feature-selection algorithms and 9 classification algorithms [31]. The included patients were randomly assigned into training and validation datasets at a ratio of 3:1. The models were initially trained in the training dataset and then were validated in the validation dataset. In addition, these two processes were repeated 10-fold to ensure the accuracy and robustness. Sensitivity, specificity and AUC were all calculated in the training and validation datasets. All machine learning processes were programmed in the python programming language (sklearn package). As for clinical data analysis, the chi-square test, independent-sample t-test, Mann-Whitney U-test and Pearson's correlation coefficient were conducted in the statistical package IBM SPSS Statistics software (Version 26.0, IBM, Armonk, NY, USA).

Patient Characters
A total of 238 patients were included in this study. Among these patients, 156 patients were diagnosed as PDAC and 82 patients were diagnosed as pNET. The gender ratios (male: female) for each subtype were 97:59 and 48: 34, respectively. The mean ages of patients were 59.69 and 53.39 (p < 0.001), respectively. The laboratory examination showed increased glucose levels of PDAC patients (6.83 mmol/L vs. 5.45 mmol/L, p < 0.001). PDAC patients had higher levels of CA19-9 than pNET patients (380.10 U/mL vs. 137.91 U/mL, p < 0.001). The level of TBIL in PDAC patients was significantly higher than that of pNET patients (115.31 umol/L vs. 26.07 umol/L, p < 0.001). There were no significant differences between gender, BMI, calcium, procalcitonin, AFP, CEA, CA12-5, amylase and lipase (p > 0.05). The clinical characters and laboratory indexes of patients are shown in Table 1.

Radiomics Features
A total of forty-eight features were extracted from six matrixes. The correlation of these features was tested by Pearson's correlation coefficient. The results suggested that most of the features were independent. Some features were shown to have a positive correlation, such as GLCM_Entropy and HISTO_Entropy (Pearson's correlation = 0.971), GLZLM_GLNU and GLRLM_GLNU (Pearson's correlation = 0.923) and GLZLM_LZHGE and GLZLM_LZE (Pearson's correlation = 0.998). The heat map of correlation among the radiomics features is shown in Figure 2. these features was tested by Pearson's correlation coefficient. The results suggested tha most of the features were independent. Some features were shown to have a positive correlation, such as GLCM_Entropy and HISTO_Entropy (Pearson's correlation = 0.971) GLZLM_GLNU and GLRLM_GLNU (Pearson's correlation = 0.923) and GLZLM_LZHGE and GLZLM_LZE (Pearson's correlation = 0.998). The heat map of cor relation among the radiomics features is shown in Figure 2.

Model Assessment
A total of forty-five models were developed with a combination of five selection al gorithms and nine classification algorithms. Radiomics features were introduced into the models. All the discriminative models exhibited a feasible diagnostic ability in the vali dation dataset. The optimal model in this study was established by the selection algo rithm (GBDT) and classification algorithm (RF), which presented the best diagnostic per formance with the highest AUC in the training dataset as well as in the validation da taset. This model demonstrated sensitivity, specificity and AUC values in the training dataset of 0.804, 0.973 and 0.971, respectively; in the validation dataset they were 0.742 0.934 and 0.93, respectively. In the training dataset, the positive and negative prediction values were 0.950 and 0.886, respectively; in the validation dataset, the positive and neg ative prediction values were 0.898 and 0.822, respectively. The sensitivity, specificity and AUC of the 45 models in the training and validation datasets are summarized in Table 2

Model Assessment
A total of forty-five models were developed with a combination of five selection algorithms and nine classification algorithms. Radiomics features were introduced into the models. All the discriminative models exhibited a feasible diagnostic ability in the validation dataset. The optimal model in this study was established by the selection algorithm (GBDT) and classification algorithm (RF), which presented the best diagnostic performance with the highest AUC in the training dataset as well as in the validation dataset. This model demonstrated sensitivity, specificity and AUC values in the training dataset of 0.804, 0.973 and 0.971, respectively; in the validation dataset they were 0.742, 0.934 and 0.93, respectively. In the training dataset, the positive and negative prediction values were 0.950 and 0.886, respectively; in the validation dataset, the positive and negative prediction values were 0.898 and 0.822, respectively. The sensitivity, specificity and AUC of the 45 models in the training and validation datasets are summarized in Table 2. In addition, the combination of RF + SVM and GBDT + SVM showed over-fitting, so we finally excluded them from the model performance comparison. To assess the robustness of the discriminative models, the processes of training and validating were repeated 10 times. The receiver operating characteristic curves (ROCs) of the optimal discriminative model in the cross validation of all folds are shown in Figure 3.
In addition, the combination of RF + SVM and GBDT + SVM showed over-fitting, so we finally excluded them from the model performance comparison. To assess the robustness of the discriminative models, the processes of training and validating were repeated 10 times. The receiver operating characteristic curves (ROCs) of the optimal discriminative model in the cross validation of all folds are shown in Figure 3.

Discussion
The incidence and mortality of pancreatic cancer varies in different regions of the world. Globally, pancreatic cancer is the seventh leading cause of cancer-related death and the fourth leading cause of cancer-related death in western countries [3,32]. The possible reasons for this difference we believe include dietary habits, obesity rates, level of economic development and ability to treat cancer. Despite having regional differences in incidence and mortality, pancreatic cancer remains one of the major challenges in the diagnosis and treatment of cancer worldwide. As a major subtype of pancreatic cancer, PDAC has unparalleled research value.
The accurate diagnosis of PDAC and pNET is important because of their different treatments and prognoses. CT is widely used as a noninvasive examination approach of abdominal tumors. However, accurate diagnosis by naked eye assessment is challenging due to the similar radiological features shared by PDAC and pNET. In this study, we extracted quantitative features from CT images and established diagnosis models, which combined radiomics with multiple machine learning algorithms to differentiate PDAC from pNET. In addition, the diagnostic ability of 45 models was investigated, demonstrating stable and outstanding discriminative performances. It is worth noting that the combination of GBDT and RF was preferred for the statistical analysis and presented an optimal discriminative ability.
In the regular statistical analysis of baseline information, differences between some parameters were significant. Our study found that the CA19-9 level of PDAC patients was significantly higher than that of pNET patients, which was consistent with previous studies reporting a significant difference in CA19-9 between PDAC patients and pNET patients [33]. The results of reduced blood glucose levels in pNET could be explained by hyperinsulinemic hypoglycemia [9]. In addition, our results suggested that the differences in bilirubin levels were significant in PADC patients and pNET patients. Other researchers have also mentioned hyperbilirubinaemia due to cholestasis in PDAC, and el-

Discussion
The incidence and mortality of pancreatic cancer varies in different regions of the world. Globally, pancreatic cancer is the seventh leading cause of cancer-related death and the fourth leading cause of cancer-related death in western countries [3,32]. The possible reasons for this difference we believe include dietary habits, obesity rates, level of economic development and ability to treat cancer. Despite having regional differences in incidence and mortality, pancreatic cancer remains one of the major challenges in the diagnosis and treatment of cancer worldwide. As a major subtype of pancreatic cancer, PDAC has unparalleled research value.
The accurate diagnosis of PDAC and pNET is important because of their different treatments and prognoses. CT is widely used as a noninvasive examination approach of abdominal tumors. However, accurate diagnosis by naked eye assessment is challenging due to the similar radiological features shared by PDAC and pNET. In this study, we extracted quantitative features from CT images and established diagnosis models, which combined radiomics with multiple machine learning algorithms to differentiate PDAC from pNET. In addition, the diagnostic ability of 45 models was investigated, demonstrating stable and outstanding discriminative performances. It is worth noting that the combination of GBDT and RF was preferred for the statistical analysis and presented an optimal discriminative ability.
In the regular statistical analysis of baseline information, differences between some parameters were significant. Our study found that the CA19-9 level of PDAC patients was significantly higher than that of pNET patients, which was consistent with previous studies reporting a significant difference in CA19-9 between PDAC patients and pNET patients [33]. The results of reduced blood glucose levels in pNET could be explained by hyperinsulinemic hypoglycemia [9]. In addition, our results suggested that the differences in bilirubin levels were significant in PADC patients and pNET patients. Other researchers have also mentioned hyperbilirubinaemia due to cholestasis in PDAC, and elevated TBIL was associated with a poorer prognosis in patients with pancreatic cancer [34,35]. Except for biochemical indexes, clinical parameters were associated with cancer type. Our study found a significant difference in the mean age between the two types of tumors, which reached a consensus with previous studies [36]. The results demonstrated that the mean age of PDAC patients was higher than that of pNET patients, suggesting that older patients are more likely to suffer from malignant tumors. In addition, it is generally accepted that PDAC usually occurs in older patients, with some studies suggesting that the median age of PDAC patients at diagnosis is 71 [2]. Comparatively, the mean age of the PDAC patients in this study was extremely younger (59.69 years). The possible reason for this discrepancy may be the trend towards a younger incidence of PDAC. It may also be that advances in diagnostic tools have led to an increase in early PDAC detection rates. Similar results have been reported in recent studies, where the mean age of PDAC patients was also significantly lower [37,38]. However, this finding needs to be treated with caution, as it is only the conclusion of a single-center retrospective study conducted in China from 2013 to 2019.
The current diagnostic methods of pancreatic cancer are biopsy, biomarkers and imaging tools [39]. Cytological examinations of clinically unresectable tumors are crucial for determining prognosis and treatment. However, the procedure of biopsy is associated with complications such as infection and hemorrhage and has a risk of peritoneal seeding [12,40]. Additionally, the pathological diagnosis between PDAC and pNET is sometimes mistaken because both of them demonstrate nuclear atypia, necrosis and a high mitotic rate [41]. Biomarkers have a limited role in differentiating PDAC from pNET [41]. Serum chromogranin A (CgA) is widely used as a biomarker for pNET, but it presents a sensitivity and specificity of 60% and 75%, respectively [42]. Moreover, the majority of PDAC patients have also been found with elevated CgA levels [41]. In terms of glucose, it varies in different subtypes of pNET. Insulinoma, the most common pNET, presents a low blood glucose level, while glucagonoma is characterized by diabetes mellitus [9]. However, it has been reported that 85% of PDAC patients are diagnosed with hyperglycemia due to dysregulation of the glucose metabolism [43]. Therefore, blood glucose level is not sufficient for predicting subtypes of pancreatic cancer. In regard to radiological evaluation, the diagnosis of PDAC and pNET by conventional CT depends on the differences in radiological features, which are induced by tumor growth. Originating from endocrine cells or the pluripotent duct cells of the pancreas, pNET usually appears as a solid avidly enhancing mass [14]. However, in some cases, ductal obstruction, ductal dilatation and up-stream pancreatic atrophy could be caused by related metabolites of pNET [25]. Thus, these atypical characteristics and insufficient radiological features make accurate diagnosis difficult by visual assessment. Diagnosis may be delayed due to non-specific clinical symptoms and the lack of effective diagnostic methods, leading to larger primary tumors or even metastases at the time of initial diagnosis [44].
Radiomics provides more quantitative information beyond the information provided by naked eye assessment, including lesion shape, volume and texture [45]. The processed data are too numerous for radiologists to perform a complete visual assessment. However, it can be further analyzed by machine learning, which has the potential to deal with complex tasks, thus improving diagnostic accuracy, optimizing clinical workflow and decreasing costs and workload [46]. The combination of radiomics and machine learning provides promising radiological evaluation methods. In previous studies, researchers investigated various algorithms and radiomics features of pancreatic lesions. Some researchers applied a few volumetric CT texture features to distinguish PADC from pNET, reporting that the combination of fifth percentile + skewness generates the highest AUC of 0.887, and the corresponding sensitivity and specificity were 0.9 and 0.8, respectively [47]. However, the limited number of texture parameters could not reflect the heterogeneity of tumors. Other researchers developed three models to differentiate nonfunctional neuroendocrine tumor (NF-pNET) from PDAC, including a model based on radiomics signatures alone, one based on clinical parameters alone and another model that integrated both [25]. It impressed us that not only radiomics features but clinical parameters were integrated into the models. The results showed that the AUC of the integrated model hit 0.884, improving on the discriminative ability based on radiomics features and clinical parameters alone. Although their model only used a single algorithm, it showed great discriminative ability. However, the small sample size of this study reduced the reliability of its conclusions. In the diagnosis of PDAC and pNET, our study included more algorithms and showed better a discriminative ability with a higher AUC. This might provide a noninvasive method to distinguish those subtypes of pancreatic cancer with a high specificity and sensitivity. Compared with similar studies, our sample size was relatively larger, which greatly increased the reliability of our research conclusions. In addition to CT, some studies have built differentiation models of PNET and PDAC based on MRI or PET-CT, and these models also have excellent differentiation abilities [26,48,49]. In addition to the small sample size, most of these studies only adopted a single algorithm and perhaps did not select the most suitable algorithm, which made their conclusions limited.
Most of the studies about machine leaning in assisting diagnosis showed the results of single machine leaning method or different combinations of algorithms. For example, a study conducted to distinguish axillary lymph node status reported that LDA classifier presented the highest AUC of 0.81 in a 2D analysis [50]. Another researcher reported that RF showed the highest AUC of 0.968 in distinguishing benign ovarian tumors from epithelial ovarian cancer [51]. A study about the diagnosis of glioblastoma and metastatic brain tumors noted that two models combining DC and LDA and combining DC and LR achieved the highest AUC of 0.80 in the validating dataset [52].
However, for the best discriminative models obtained in different studies, the respective algorithms used are not exactly the same. There is no best answer to the choice of algorithms for studies related to radiomics combined with machine learning.
Though there are numerous algorithms for researchers to choose from, previous studies have employed only a few classifiers for discrimination. Therefore, we included more algorithms to evaluate their discriminative performance and tried to select the optimal algorithm. In our study, five selection algorithms and nine classification algorithms were adopted to determine the suitable discriminative models. Our research found that the combination of GBDT and RF presented the best performance. GBDT, a widely used algorithm in machine learning, receives promising predictability when coping with numerous factors and complicated relations [53]. Other researchers suggested that GBDT showed the highest performance in predicting colorectal cancer compared with RF and SVM, indicating that implementing a GBDT model is time-saving and cost-effective [54]. So, GBDT was applicable for our study because of its excellent prediction and ability to deal with numerical values. RF, a collection of decision trees, can balance data and be employed in classification tasks [55,56]. In a previous study, researchers adopted RF for assisting with Parkinson's disease, suggesting that RF achieved better classification compared with LR and SVM [57]. Considering the various choices of algorithms and different types of cancer, more attention needs to be paid in feature-selection algorithms and machine-learning classifiers in future investigations to build more reliable models. Though a finite number of open-source algorithms were employed in our research, the results of our study could provide a reference for algorithm selection when predicting subtypes of cancer with machine learning.
There were several limitations in our study. First, this was a retrospective study and therefore bias was inevitable. Additionally, we established the model and tested the performance in the database of our institution, and an external validation set from another institution was required.

Conclusions
In conclusion, methods based on enhanced CT radiomics features combined with multiple machine learning algorithms have a promising ability to differentiate PDAC from pNET. In addition, this assistance method is expected to facilitate clinical decision-making and reduce the invasive injury caused by pathological examinations. We also suggest that when similar research is conducted in the future multiple algorithms should be considered in the model rather than empirically using only a single algorithm.
Supplementary Materials: The following supporting information can be downloaded at: https: //www.mdpi.com/article/10.3390/jcm11226789/s1. Document S1: The details and parameters of CT scanning procedures. All the detailed information about CT equipment and contrast agent and the specific operation details of the scanning process are described in the Supplementary Document S1.
Author Contributions: Conception and design: T.Z., X.W. and H.Z. Administrative support: X.W. and H.Z. Collection and assembly of data: Y.X., H.W. and H.Y. and T.Z. Data analysis and interpretation: T.Z., Y.X., H.W. and Y.L. Manuscript writing: T.Z. Final approval of manuscript: All authors. All authors have read and agreed to the published version of the manuscript.
Funding: This research received no external funding.

Institutional Review Board Statement:
The study was conducted in accordance with the Declaration of Helsinki and was approved by the Ethics Committee of West China Hospital, Sichuan University (Approval Number 2019-903, 6 November 2019).

Informed Consent Statement:
As this study was a retrospective study and did not disclose any personal information, informed consent was waived with the approval of the Institutional Ethics Committee.

Data Availability Statement:
The data presented in this study are available on request from the corresponding author.