Development of a Deep Learning Model for Malignant Small Bowel Tumors Survival: A SEER-Based Study
by
, ,
Minyue Yin
1,2,†
,
Jiaxi Lin
1,2,†,
Lu Liu
1,2,
Jingwen Gao
1,2,
Wei Xu
1,2,
Chenyan Yu
1,2,
Shuting Qu
1,2,
Xiaolin Liu
1,2,
Lijuan Qian
1,2,
Chunfang Xu
1,2,* and
Jinzhou Zhu
1,2,* 1
Department of Gastroenterology, The First Affiliated Hospital of Soochow University, Suzhou 215006, China
2
Suzhou Clinical Center of Digestive Diseases, Suzhou 215006, China
*
Authors to whom correspondence should be addressed.
†
These authors contributed equally to this work.
Diagnostics 2022, 12(5), 1247; https://doi.org/10.3390/diagnostics12051247
Submission received: 15 April 2022
/
Revised: 11 May 2022
/
Accepted: 12 May 2022
/
Published: 17 May 2022
(This article belongs to the Special Issue Modern Imaging and Computer-Aided Diagnosis in Gastroenterology)
Abstract
:Background This study aims to explore a deep learning (DL) algorithm for developing a prognostic model and perform survival analyses in SBT patients. Methods The demographic and clinical features of patients with SBTs were extracted from the Surveillance, Epidemiology and End Results (SEER) database. We randomly split the samples into the training set and the validation set at 7:3. Cox proportional hazards (Cox-PH) analysis and the DeepSurv algorithm were used to develop models. The performance of the Cox-PH and DeepSurv models was evaluated using receiver operating characteristic curves, calibration curves, C-statistics and decision-curve analysis (DCA). A Kaplan–Meier (K–M) survival analysis was performed for further explanation on prognostic effect of the Cox-PH model. Results The multivariate analysis demonstrated that seven variables were associated with cancer-specific survival (CSS) (all p < 0.05). The DeepSurv model showed better performance than the Cox-PH model (C-index: 0.871 vs. 0.866). The calibration curves and DCA revealed that the two models had good discrimination and calibration. Moreover, patients with ileac malignancy and N2 stage disease were not responding to surgery according to the K–M analysis. Conclusions This study reported a DeepSurv model that performed well in CSS in SBT patients. It might offer insights into future research to explore more DL algorithms in cohort studies.
1. Background
Previous studies indicate that small bowel tumors (SBTs) are rare, with an estimated incidence rate of 2.6 per 100,000 for males and 2.0 per 100,000 for females in 2019 [1]. Despite the fact that the small intestine takes up the majority of the whole gastrointestinal tract, SBTs only account for 1–2% of all gastrointestinal tumors [2]. Nevertheless, SBTs have been on the rise for decades, with an average annual growth rate of 1.8%, owing to the remarkable progress of endoscopy and imaging [1]. Malignancy is more frequent in SBTs, with the most common histological diagnoses being adenocarcinoma, neuroendocrine tumors, gastrointestinal stromal tumors (GISTs) and lymphoma [3]. SBTs are characterized by a lengthy silence-period, nonspecific symptoms and a high degree of malignancy [1,4]. Therefore, studies that focus on the prognosis of SBTs are largely restricted.
Artificial intelligence (AI), consisting of machine learning (ML) and deep learning (DL), has lately been applied to the field of medicine [5,6,7,8,9]. Using an ensemble of ML models, Levi et al. [8] developed a prediction model for detecting patients with gastrointestinal bleeding who required transfusion. Skrede et al. [5] trained a DL model, based on ten-layer convolutional neural networks (CNNs), for risk stratification to help decision making in the therapy of colorectal cancer. Therefore, AI-aided systems are useful in various clinical settings, e.g., risk stratification, personalized treatments and disease classification based on endoscopic images [10,11,12]. As for AI application in survival analysis, Katzman et al. [13] first proposed a novel technology, based on deep neural networks, for survival analysis in 2018. They proved that the use of DeepSurv allowed for higher performance of prognostic prediction and effective treatment recommendations, which was superior to the state-of-the-art survival methods such as Cox-PH and random survival forests. Since then, DeepSurv has been used in a variety of fields [14,15,16], e.g., the prognosis of lung cancer [17], the therapeutic efficacy of chemoradiation in head and neck cancer [18] and the detection of cognitive decline [19], which all showed better performance than traditional Cox-PH models.
To date, several survival models have been proposed for predicting overall survival (OS) and cancer-specific survival (CSS) in SBTs [20,21,22,23,24]. However, these models all used traditional Cox proportional hazards methods (Cox-PH), which were time-consuming due to manual feature selection. Furthermore, the Cox-PH is a semiparametric model that explores the linear correlation between covariates, even though relations among covariates and survival outcomes are usually complex and nonlinear. Therefore, our study aims to evaluate the feasibility of DL models for survival analysis in malignant SBTs, based on the Surveillance, Epidemiology and End Results (SEER) dataset [13].
2. Materials and Methods
2.1. Cohorts Dataset
This study was based on the SEER database that provides information on cancer statistics, such as demographic and clinical data, in an effort to reduce the cancer burden [10,11,13,14]. SBTs were diagnosed using the third edition of the International Classification of Disease for Oncology (ICD-O-3), with site codes of C17.0–17.9. For cases diagnosed in 2018 and later, the SEER program applied Extent of Disease (EOD), a set of three items including primary tumor, lymph nodes and metastasis. According to the aforementioned criteria, cases diagnosed with SBTs in 2018 were included in this study. All patients were selected from the “SEER Incidence Dataset”, which collected cancer incidence data from population-based cancer registries covering nearly 47.9% of the U.S. population, using SEER*Stat software (version 3.8.9.2). Moreover, the following are the exclusion criteria: (1) patients under the age of 18; (2) patients whose survival duration was unclear; (3) patients with incomplete demographic data. We collected variables from the SEER database, including age, sex, race, income, region, T stage, N stage, M stage, tumor size, primary tumor site, histological type and the number of lymph nodes removed. Our primary endpoint was CSS, the time interval between the first diagnosis and death from SBTs. Figure 1 shows the flowchart of this study.
2.2. Data Preprocessing
A total of 1987 patients were included in our study and were randomly split into the training set (n = 1398) and validation set (n = 589) at 7:3. Missing data, assumed to be missing at random, were multiple imputed using a random forest algorithm from the ‘mice’ package in R software [25]. The imputed variables were tumor size (21.67% in the training set and 21.05% in the validation set) and the number of lymph nodes removed (4.94% in the training set and 2.89% in the validation set).
2.3. Development of the Cox-PH Model
Univariate and multivariate Cox regression analyses were used to select important features and develop a prognostic model. The variables with a two-sided p-value of less than 0.05, determined in the univariate analysis, were integrated into the multivariate analysis. The performances of the 3-, 6- and 9-month survival models were evaluated with time-dependent receiver operating characteristic (ROC) curve, calibration curve, Harrell C-statistics and decision-curve analysis (DCA) based on the training and validation sets.
2.4. Development of the DeepSurv Model
DeepSurv is a feed-forward neural-network-based extension of the Cox proportional hazards model, consisting of an input layer, an output layer and multiple hidden layers, which can predict the survival outcomes. Patients’ baseline data are fed into the network as an input layer. A fully connected layer of nodes is followed by a dropout layer in the hidden layer. The output layer generates the probability of each patient’s survival event estimated using the DeepSurv model. To begin, we used the training cohort, which was randomly split into the internal training set and internal validation set at 8:2, to develop the DL model of a four-layer neural network, and the validation cohort to evaluate the efficacy of this model. The process of training involved making real-time adjustments to the hyperparameters, which included the input standardization, the Rectified Linear Units (ReLU) activation function, the Adaptive Moment Estimate (Adam) optimizer and the learning rate scheduling [13]. Early stopping regularization was used to prevent model overtraining and avoid overfitting. The performance of the model was evaluated using the Harrell C-index, ROC curve, calibration curve and DCA based on the validation set.
2.5. Statistical Analysis
Continuous variables were presented as the mean ± standard deviation (SD) if normally distributed or as the median if not. Categorical variables were described as frequencies. Baseline characteristics were compared using Student’s t-test or nonparametric tests for continuous variables and Chi-square tests for categorical variables. Furthermore, Kaplan–Meier (K–M) survival analysis was applied to evaluate the prognostic effect of those independent factors determined with multivariate Cox regression analysis using the log-rank test. Statistical analysis was performed using the ‘MASS’ package of R software (version 4.0) and the ‘pycox’ package of Python (version 3.9). A two-side p < 0.05 was considered statistically significant.
3. Results
3.1. Patient Characteristics
Our study involved a total of 1987 patients with malignant SBTs, with more males (53.6%) than females (46.4%). A majority of patients with SBTs were between the age of 40 and 80, were white (74.1%), and lived in the metropolis (89.1%) with a comfortable household income. Carcinoid (50.4%) was the most common histological type, followed by adenocarcinoma (30.9%) and stromal sarcoma (11.0%). Primary tumor sites were located in the duodenum (35.1%), ileum (33.3%), jejunum (11.0%) and overlapping sites (0.8%). Based on the updated American Joint Committee on Cancer (AJCC) 8th edition staging criteria, patients were classified as stage I (24.8%), II (15.5%), III (25.2%) and IV (24.7%). More details are listed in Table 1.
3.2. Performance of the Cox-PH Model
The dataset was randomly split into the training set (n = 1398) and the validation set (n = 589) at 7:3. Multivariate Cox regression analysis was performed to determine independent prognostic factors for CSS, including age, race, primary tumor site, histological type, N stage, surgery and the number of retrieval lymph nodes, as shown in Figure 2. The results showed that patients who were Asian, had tumors located in the ileum and had N2 stage disease had poor prognosis (p < 0.05). Additionally, histological types had the greatest impact on prognosis, of which adenocarcinoma and others (neuroendocrine carcinoma, squamous cell carcinoma, signet ring carcinoma, etc.) accounted for a large proportion (p < 0.001).
The C-indices of the Cox-PH model were 0.878 and 0.866, respectively, based on the training and validation set, as shown in Table S1. Furthermore, ROC curves (Figure 3A) were constructed to evaluate the discrimination capability of this model based on the validation set, with the areas under the ROC curve (AUCs) of 0.874, 0.922 and 0.908 for 3-, 6- and 9-month survival probability, respectively. Figure 4A–C shows that our model had minor deviations from the reference line in the calibration curve for 3-, 6- and 9-month CSS in the validation set, with the Brier scores of 0.282, 0.265 and 0.265, respectively. Furthermore, the C-index, AUCs and Brier scores based on the training set are demonstrated in Table 2. Additionally, DCA curves for this Cox-PH model are presented in Figure 5A–C for 3-, 6- and 9-month CSS, demonstrating that our model could help patients obtain a net benefit of 8–10%.
3.3. Performance of the DeepSurv Model
The aforementioned training set (n = 1398) was randomly split into the “internal training” set (n = 1129) and the “internal validation” set (n = 269) at 8:2 for the best set of hyperparameters. The validation set (n = 589) was regarded as the “testing” set for evaluating the performance of the final DeepSurv model. We tuned DeepSurv’s hyperparameters by referring to previous studies [7,13], as described in Table S1. With the best performing hyperparameters, four layers (nodes: 32, 64, 128, 256), a dropout rate of 0.2 and a learning rate of 0.020092, the performance of the final model was evaluated with the validation set (n = 589). The C-index was 0.871, while the AUCs were 0.878, 0.891 and 0.891 for 3-, 6- and 9-month CSS, respectively, which were inferior to those of the Cox-PH model, as shown in Table 2 and Figure 3B. Furthermore, the calibration and prediction curves of the DeepSurv model for the 3-, 6- and 9-month CSS showed great correlations in the validation cohort, with Brier scores of 0.058, 0.070 and 0.080, respectively, indicating a high degree of reliability (Figure 4D–F). DCA indicated that this model had similar net benefits as the Cox-PH model, demonstrating both of them possess good clinical practice in predicting CSS in patients with SBTs (Figure 5D–F).
3.4. K–M Survival Analysis Based on the COX-PH Model
Survival analysis showed that patients who were old and had adenocarcinoma had poorer prognosis (p < 0.0001), while patients who had a tumor located in the ileum had relatively good prognosis (p < 0.001), as shown in Figure S1A–C. Furthermore, we conducted a sub-group analysis in patients who received surgery. It demonstrated that surgery could significantly improve the CSS whatever the histological type was (Figure S1D, p < 0.0001). For patients who had a tumor located in the ileum and had N2 stage disease, surgery showed no significant efficacy in improving prognosis (Figure S1E,F, p > 0.05).
4. Discussion
Due to the relatively low prevalence, studies concerned with the management of SBTs are lacking. Therefore, effective methods for the risk stratification and personalized therapy of SBTs are essential for improving their survival rates.
Currently, it is well known that the AJCC stage system is applied to predict the survival of patients with SBTs. However, this system only takes TNM stage into consideration for predicting cancer risk, neglecting other significant clinical features and demographic information. To improve predictive accuracy, recent studies have constructed various prognostic models for small bowel adenocarcinoma and neuroendocrine carcinoma, using Cox regression analysis [20,21,22,24,26]. Wang et al. [20] built and verified nomograms to predict OS and CSS for small intestine adenocarcinoma (SIA), with higher sensitivity and specificity than the AJCC stage system. That study highlighted ten variables, including demographic data (age and marital status) and therapeutic regimen (surgery and chemotherapy). Zheng et al. [21] also fitted a Cox-PH model for predicting the CSS of SIA, with the C-index of 0.728 based on the internal validation. Cox regression analysis achieves high accuracy and has become a popular method for prognostic predictions. Nevertheless, it is impractical to fit survival models based on linear relation, which requires more complex training methods, e.g., ML algorithms or even deep neural networks.
The remarkable progress in AI facilitates the development of complex nonlinear models based on big clinical data in survival analysis. Adeoye et al. [7] discovered that the DeepSurv model had a higher C-index (0.95 vs. 0.83) for assessing the malignant-transformation risk in patients with oral leukoplakia and lichenoid lesions than traditional Cox regression analysis. As mentioned in the ‘Introduction’ section, it is indeed practicable to apply the DeepSurv algorithm to fitting prognostic models. However, to our knowledge, relative research using DL algorithms for prognostic prediction in SBTs are limited. Therefore, this study presented the development of a DeepSurv model for predicting the CSS of patients with SBTs and compared the performance of the Cox-PH and DeepSurv models.
Our study concluded that the DeepSurv model performed better in predicting survival for patients with SBTs than the Cox-PH model, based on their C-indices. Additionally, calibration curves demonstrated that the estimated risk when using DeepSurv was closer to the observed risk than when using Cox-PH analysis, with lower Brier scores. DCA showed that the two models obtained similar performance in clinical practice, with 8–10% net benefits. As a result, the predictions estimated using this DeepSurv model were proven to have good discrimination and calibration. The advantages of the DeepSurv model, we believe, attribute to the automated features extraction of inputs and the analytic ability of complex big data [17,27,28].
In terms of K–M analysis, our study demonstrated that surgery was a good choice for improving prognosis in patients with SBTs. However, in the sub-group analysis, there was some controversy over surgery. Firstly, patients with ileac tumors were proven to have better CSS compared to those with duodenal or jejunal cancer in our study. Similarly, Nicholl et al. [29] drew the same conclusion from analyzing 1444 patients with small intestine adenocarcinoma, whereas for further analysis, we found that patients with ileac malignancy were not responding to surgery according to the K–M survival analysis. Secondly, surgery had an adverse effect on patients with N2 stage disease, who were classified into stage IIIB-IV, in our study. Many research studies have revealed that palliative chemotherapy could prolong life for stage IV patients, no matter whether surgery was performed or not [30,31]. Therefore, chemotherapy might become another alternative for patients with an advanced stage of SBT. More research is needed to further explore more effective and precise treatment for different sub-group SBT patients.
There are some limitations in our study. To begin with, this was a retrospective study based on the SEER dataset, with potential heterogeneity of the enrolled patients. More external validations are needed in future research. Secondly, more risk factors could be included in future research to better guide the therapeutic regimen for SBT patients.
5. Conclusions
In conclusion, our study found that the DeepSurv model performed better than the traditional Cox-PH model in predicting the survival of patients with SBTs. The DL model exhibited feasibility and promise in clinical practices, which might offer insights for future research to explore more DL algorithms in prognostic analysis.
Supplementary Materials
The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/diagnostics12051247/s1, Figure S1: K–M analysis of independent risk factors for cancer-specific survival. (A). Age; (B). primary tumor site (ileum); (C). histological type (adenocarcinoma); (D–F). the efficacy of surgery on prognosis in the sub-groups of patients who had adenocarcinoma (D), had tumor located in ileum (E) and had N2 stage disease (F). (A) Age = 0 means age < 30, age = 1 means 30 < age ≤ 40, age = 2 means 40 < age ≤ 50, age = 3 means 50 < age ≤ 60, age = 4 means 60 < age ≤ 70, age = 5 means 70 < age ≤ 80, age = 6 means age ≥ 80. (D–F) Strata = 0 means no surgery, strata = 1 means surgery, n:1 means Nx, n:2 means N0, n:3 means N1, n:4 means N2. Table S1: C-indices of Cox-PH and DeepSurv models based on different hyperparameters.
Author Contributions
Conceptualization, M.Y. and J.L.; methodology, M.Y.; software, M.Y. and J.L.; validation, L.L., J.G. and W.X.; formal analysis, C.Y.; data curation, J.L.; writing—original draft preparation, M.Y.; writing—review and editing, J.Z.; visualization, S.Q.; supervision, X.L. and L.Q.; project administration, C.X. and J.Z.; funding acquisition, C.X. and J.Z. All authors have read and agreed to the published version of the manuscript.
Funding
This research was funded by Science and Technology Plan of Suzhou City; (SKY2021038) sponsored by Chunfang Xu and Youth Program of Suzhou Health Committee; (KJXW2019001) sponsored by Jinzhou Zhu.
Institutional Review Board Statement
The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board of the first Affiliated Hospital of Soochow University. (Protocol code 098 and date of approval: 31 March 2022).
Informed Consent Statement
Not applicable.
Data Availability Statement
The data presented in this study are openly available in https://seer.cancer.gov/ (accessed on 20 March 2022).
Conflicts of Interest
The authors declare no conflict of interest.
References
- Benson, A.B.; Venook, A.P.; Al-Hawary, M.M.; Arain, M.A.; Chen, Y.J.; Ciombor, K.K.; Cohen, S.A.; Cooper, H.S.; Deming, D.A.; Garrido-Laguna, I.; et al. Small Bowel Adenocarcinoma, Version 1.2020, NCCN Clinical Practice Guidelines in Oncology. J. Natl. Compr. Cancer Netw. 2019, 17, 1109–1133. [Google Scholar] [CrossRef] [PubMed]
- Kang, B.; Zhang, Q.; Shang, D.; Ni, Q.; Muhammad, F.; Hou, L.; Cui, W. Resolution of intussusception after spontaneous expulsion of an ileal lipoma per rectum: A case report and literature review. World J. Surg. Oncol. 2014, 12, 143. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Wang, N.; Yang, J.; Lyu, J.; Liu, Q.; He, H.; Liu, J.; Li, L.; Ren, X.; Li, Z. A convenient clinical nomogram for predicting the cancer-specific survival of individual patients with small-intestine adenocarcinoma. BMC Cancer 2020, 20, 505. [Google Scholar] [CrossRef]
- Min, M.; Noujaim, M.G.; Green, J.; Schlieve, C.R.; Vaze, A.; Cahan, M.A.; Cave, D.R. Role of Mucosal Protrusion Angle in Discriminating between True and False Masses of the Small Bowel on Video Capsule Endoscopy. J. Clin. Med. 2019, 8, 418. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Skrede, O.-J.; De Raedt, S.; Kleppe, A.; Hveem, T.S.; Liestøl, K.; Maddison, J.; A Askautrud, H.; Pradhan, M.; Nesheim, J.A.; Albregtsen, F.; et al. Deep learning for prediction of colorectal cancer outcome: A discovery and validation study. Lancet 2020, 395, 350–360. [Google Scholar] [CrossRef]
- Foersch, S.; Eckstein, M.; Wagner, D.-C.; Gach, F.; Woerl, A.-C.; Geiger, J.; Glasner, C.; Schelbert, S.; Schulz, S.; Porubsky, S.; et al. Deep learning for diagnosis and survival prediction in soft tissue sarcoma. Ann. Oncol. 2021, 32, 1178–1187. [Google Scholar] [CrossRef]
- Adeoye, J.; Koohi-Moghadam, M.; Lo, A.W.I.; Tsang, R.K.-Y.; Chow, V.L.Y.; Zheng, L.-W.; Choi, S.-W.; Thomson, P.; Su, Y.-X. Deep Learning Predicts the Malignant-Transformation-Free Survival of Oral Potentially Malignant Disorders. Cancers 2021, 13, 6054. [Google Scholar] [CrossRef]
- Levi, R.; Carli, F.; Arévalo, A.R.; Altinel, Y.; Stein, D.J.; Naldini, M.M.; Grassi, F.; Zanoni, A.; Finkelstein, S.; Vieira, S.M.; et al. Artificial intelligence-based prediction of transfusion in the intensive care unit in patients with gastrointestinal bleeding. BMJ Health Care Inform. 2021, 28, e100245. [Google Scholar] [CrossRef]
- Huang, L.; Lu, X.; Huang, X.; Zou, X.; Wu, L.; Zhou, Z.; Wu, D.; Tang, D.; Chen, D.; Wan, X.; et al. Intelligent difficulty scoring and assistance system for endoscopic extraction of common bile duct stones based on deep learning: Multicenter study. Laryngo-Rhino-Otologie 2020, 53, 491–498. [Google Scholar] [CrossRef]
- Itoh, T.; Kawahira, H.; Nakashima, H.; Yata, N. Deep learning analyzes Helicobacter pylori infection by upper gastrointestinal endoscopy images. Endosc. Int. Open 2018, 6, E139–E144. [Google Scholar] [CrossRef] [Green Version]
- Hirasawa, T.; Aoyama, K.; Tanimoto, T.; Ishihara, S.; Shichijo, S.; Ozawa, T.; Ohnishi, T.; Fujishiro, M.; Matsuo, K.; Fujisaki, J.; et al. Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images. Gastric Cancer 2018, 21, 653–660. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Chuang, W.-Y.; Chen, C.-C.; Yu, W.-H.; Yeh, C.-J.; Chang, S.-H.; Ueng, S.-H.; Wang, T.-H.; Hsueh, C.; Kuo, C.-F.; Yeh, C.-Y. Identification of nodal micrometastasis in colorectal cancer using deep learning on annotation-free whole-slide images. Mod. Pathol. 2021, 34, 1901–1911. [Google Scholar] [CrossRef] [PubMed]
- Katzman, J.L.; Shaham, U.; Cloninger, A.; Bates, J.; Jiang, T.; Kluger, Y. DeepSurv: Personalized treatment recommender system using a Cox proportional hazards deep neural network. BMC Med. Res. Methodol. 2018, 18, 24. [Google Scholar] [CrossRef]
- Kather, J.N.; Pearson, A.T.; Halama, N.; Jäger, D.; Krause, J.; Loosen, S.H.; Marx, A.; Boor, P.; Tacke, F.; Neumann, U.P.; et al. Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nat. Med. 2019, 25, 1054–1056. [Google Scholar] [CrossRef] [PubMed]
- Liu, Z.; Liu, Y.; Zhang, W.; Hong, Y.; Meng, J.; Wang, J.; Zheng, S.; Xu, X. Deep learning for prediction of hepatocellular carcinoma recurrence after resection or liver transplantation: A discovery and validation study. Hepatol. Int. 2022, 1–13. [Google Scholar] [CrossRef]
- Hosny, A.; Parmar, C.; Coroller, T.P.; Grossmann, P.; Zeleznik, R.; Kumar, A.; Bussink, J.; Gillies, R.J.; Mak, R.H.; Aerts, H.J.W.L. Deep learning for lung cancer prognostication: A retrospective multi-cohort radiomics study. PLoS Med. 2018, 15, e1002711. [Google Scholar] [CrossRef] [Green Version]
- She, Y.; Jin, Z.; Wu, J.; Deng, J.; Zhang, L.; Su, H.; Jiang, G.; Liu, H.; Xie, D.; Cao, N.; et al. Development and Validation of a Deep Learning Model for Non–Small Cell Lung Cancer Survival. JAMA Netw. Open 2020, 3, e205842. [Google Scholar] [CrossRef] [PubMed]
- Howard, F.M.; Kochanny, S.; Koshy, M.; Spiotto, M.; Pearson, A.T. Machine Learning–Guided Adjuvant Treatment of Head and Neck Cancer. JAMA Netw. Open 2020, 3, e2025881. [Google Scholar] [CrossRef]
- Wang, L.; Laurentiev, J.; Yang, J.; Lo, Y.-C.; Amariglio, R.E.; Blacker, D.; Sperling, R.A.; Marshall, G.A.; Zhou, L. Development and Validation of a Deep Learning Model for Earlier Detection of Cognitive Decline From Clinical Notes in Electronic Health Records. JAMA Netw. Open 2021, 4, e2135174. [Google Scholar] [CrossRef]
- Wang, D.; Li, C.; Li, Y.; Liu, W.; Zhao, L.; Güngör, C.; Tan, F.; Zhou, Y. Specific survival nomograms based on SEER database for small intestine adenocarcinoma. Ann. Palliat. Med. 2021, 10, 7440–7457. [Google Scholar] [CrossRef]
- Zheng, Z.; Zhou, X.; Zhang, J.; Zhao, B.; Chen, C.; Liu, X.; Cao, H.; Li, T.; Geng, R.; Wang, W.; et al. Nomograms predict survival of patients with small bowel adenocarcinoma: A SEER-based study. Int. J. Clin. Oncol. 2020, 26, 387–398. [Google Scholar] [CrossRef]
- Jiang, S.; Zhao, L.; Xie, C.; Su, H.; Yan, Y. Prognostic Performance of Different Lymph Node Staging Systems in Patients with Small Bowel Neuroendocrine Tumors. Front. Endocrinol. 2020, 11, 402. [Google Scholar] [CrossRef] [PubMed]
- Dong, D.; Fang, M.-J.; Tang, L.; Shan, X.-H.; Gao, J.-B.; Giganti, F.; Wang, R.-P.; Chen, X.; Wang, X.-X.; Palumbo, D.; et al. Deep learning radiomic nomogram can predict the number of lymph node metastasis in locally advanced gastric cancer: An international multicenter study. Ann. Oncol. 2020, 31, 912–920. [Google Scholar] [CrossRef] [PubMed]
- Gu, Y.; Deng, H.; Wang, D.; Li, Y. Metastasis Pattern and Survival Analysis in Primary Small Bowel Adenocarcinoma: A SEER-Based Study. Front. Surg. 2021, 8, 759162. [Google Scholar] [CrossRef] [PubMed]
- Blazek, K.; van Zwieten, A.; Saglimbene, V.; Teixeira-Pinto, A. A practical guide to multiple imputation of missing data in nephrology. Kidney Int. 2020, 99, 68–74. [Google Scholar] [CrossRef]
- Xie, J.-B.; Pang, Y.-S.; Li, X.; Wu, X.-T. Role of Removed Lymph Nodes on the Prognosis of M0 Small-Bowel Neuroendocrine Tumors: A Propensity Score Matching Analysis from SEER Database. J. Gastrointest. Surg. 2021, 25, 3188–3197. [Google Scholar] [CrossRef]
- Shaffie, A.; Soliman, A.; Fraiwan, L.; Ghazal, M.; Taher, F.; Dunlap, N.; Wang, B.; Van Berkel, V.; Keynton, R.; Elmaghraby, A.; et al. A Generalized Deep Learning-Based Diagnostic System for Early Diagnosis of Various Types of Pulmonary Nodules. Technol. Cancer Res. Treat. 2018, 17, 1533033818798800. [Google Scholar] [CrossRef]
- Hathaway, Q.A.; Yanamala, N.; Budoff, M.J.; Sengupta, P.P.; Zeb, I. Deep neural survival networks for cardiovascular risk prediction: The Multi-Ethnic Study of Atherosclerosis (MESA). Comput. Biol. Med. 2021, 139, 104983. [Google Scholar] [CrossRef]
- Nicholl, M.B.; Ahuja, V.; Conway, W.C.; Vu, V.D.; Sim, M.-S.; Singh, G. Small Bowel Adenocarcinoma: Understaged and Undertreated? Ann. Surg. Oncol. 2010, 17, 2728–2732. [Google Scholar] [CrossRef]
- Liu, T.; Wu, Y.; Jiang, T. Efficacy of surgery and chemotherapy for stage IV small bowel adenocarcinoma: A population-based analysis using Surveillance, Epidemiology, and End Result Program database. Cancer Med. 2020, 9, 6638–6645. [Google Scholar] [CrossRef]
- Aparicio, T.; Henriques, J.; Manfredi, S.; Tougeron, D.; Bouché, O.; Pezet, D.; Piessen, G.; Coriat, R.; Zaanan, A.; Legoux, J.-L.; et al. Small bowel adenocarcinoma: Results from a nationwide prospective ARCAD-NADEGE cohort study of 347 patients. Int. J. Cancer 2020, 147, 967–977. [Google Scholar] [CrossRef] [PubMed]
Figure 1.
Flowchart of this study.
Figure 2.
Multivariate Cox regression analysis. * p < 0.05 was considered statistically significant.
Figure 2.
Multivariate Cox regression analysis. * p < 0.05 was considered statistically significant.
Figure 3.
ROC curves of the Cox-PH (A) and DeepSurv (B) models.
Figure 4.
Calibration curves of the Cox-PH (A–C) and DeepSurv (D–F) models for 3-, 6- and 9-month cancer-specific survival.
Figure 4.
Calibration curves of the Cox-PH (A–C) and DeepSurv (D–F) models for 3-, 6- and 9-month cancer-specific survival.
Figure 5.
Decision-curve analysis of the Cox-PH (A–C) and DeepSurv (D–F) models for 3-, 6- and 9-month cancer-specific survival.
Figure 5.
Decision-curve analysis of the Cox-PH (A–C) and DeepSurv (D–F) models for 3-, 6- and 9-month cancer-specific survival.
Table 1.
Characteristics of patients included in our study.
| The Training Set (n = 1398) | The Validation Set (n = 589) | |||||
|---|---|---|---|---|---|---|
| Characteristics | Non-CSD (n = 1238) | CSD (n = 160) | p | Non-CSD (n = 534) | CSD (n = 55) | p |
| Sex (%) | 0.404 | 0.320 | ||||
| Female | 566 (45.7) | 67 (41.9) | 258 (48.3) | 31 (56.4) | ||
| Male | 672 (54.3) | 93 (58.1) | 276 (51.7) | 24 (43.6) | ||
| Age (years, %) | <0.001 | <0.001 | ||||
| <30 | 7 (0.6) | 2 (1.2) | 4 (0.7) | 0 (0.0) | ||
| 30–39 | 40 (3.2) | 1 (0.6) | 20 (3.7) | 1 (1.8) | ||
| 40–49 | 100 (8.1) | 2 (1.2) | 42 (7.9) | 1 (1.8) | ||
| 50–59 | 249 (20.1) | 16 (10.0) | 119 (22.3) | 10 (18.2) | ||
| 60–69 | 382 (30.9) | 39 (24.4) | 142 (26.6) | 8 (14.5) | ||
| 70–79 | 308 (24.9) | 53 (33.1) | 143 (26.8) | 14 (25.5) | ||
| ≥80 | 152 (12.3) | 47 (29.4) | 64 (12.0) | 21 (38.2) | ||
| Race (%) | 0.056 | 0.199 | ||||
| Black | 246 (19.9) | 29 (18.1) | 89 (16.7) | 11 (20.0) | ||
| White | 914 (73.8) | 112 (70.0) | 408 (76.4) | 39 (70.9) | ||
| Asian | 70 (5.7) | 18 (11.2) | 33 (6.2) | 3 (5.5) | ||
| Indian | 8 (0.6) | 1 (0.6) | 4 (0.7) | 2 (3.6) | ||
| Median household Income ($, %) | 0.732 | 0.984 | ||||
| (−∞, 35,000) | 19 (1.5) | 1 (0.6) | 10 (1.9) | 1 (1.8) | ||
| [35,000, 55,000) | 271 (21.9) | 38 (23.8) | 106 (19.9) | 12 (21.8) | ||
| [55,000, 75,000) | 539 (43.5) | 66 (41.2) | 222 (41.6) | 23 (41.8) | ||
| [75,000, +∞) | 409 (33.0) | 55 (34.4) | 196 (36.7) | 19 (34.5) | ||
| Region | 0.386 | 0.743 | ||||
| non-metropolis | 141 (11.4) | 14 (8.8) | 55 (10.3) | 7 (12.7) | ||
| metropolis | 1097 (88.6) | 146 (91.2) | 479 (89.7) | 48 (87.3) | ||
| Primary tumor site (%) | <0.001 | <0.001 | ||||
| NOS | 131 (10.6) | 18 (11.2) | 51 (9.6) | 5 (9.1) | ||
| Duodenum | 537 (43.4) | 102 (63.7) | 210 (39.3) | 38 (69.1) | ||
| Jejunum | 145 (11.7) | 15 (9.4) | 57 (10.7) | 2 (3.6) | ||
| Ileum | 419 (33.8) | 23 (14.4) | 210 (39.3) | 9 (16.4) | ||
| Overlapping | 6 (0.5) | 2 (1.2) | 6 (1.1) | 1 (1.8) | ||
| Histological type (%) | <0.001 | <0.001 | ||||
| others | 72 (5.8) | 31 (19.4) | 40 (7.5) | 10 (18.2) | ||
| Adenocarcinoma | 320 (25.8) | 111 (69.4) | 143 (26.8) | 39 (70.9) | ||
| Carcinoid | 685 (55.3) | 16 (10.0) | 295 (55.2) | 6 (10.9) | ||
| Stromal sarcoma | 161 (13.0) | 2 (1.2) | 56 (10.5) | 0 (0.0) | ||
| T stage (%) | <0.001 | 0.001 | ||||
| Tx | 164 (13.2) | 68 (42.5) | 77 (14.4) | 23 (41.8) | ||
| T0 | 3 (0.2) | 2 (1.2) | 0 | 0 | ||
| Tis | 0 (0.0) | 1 (0.6) | 0 | 0 | ||
| T1 | 298 (24.1) | 22 (13.8) | 112 (21.0) | 9 (16.4) | ||
| T2 | 176 (14.2) | 5 (3.1) | 84 (15.7) | 1 (1.8) | ||
| T3 | 340 (27.5) | 17 (10.6) | 146 (27.3) | 5 (9.1) | ||
| T4 | 257 (20.8) | 45 (28.1) | 115 (21.5) | 17 (30.9) | ||
| N stage (%) | 0.731 | 0.556 | ||||
| Nx | 109 (8.8) | 37 (23.1) | 53 (9.9) | 12 (21.8) | ||
| N0 | 687 (55.5) | 79 (49.4) | 260 (48.7) | 27 (49.1) | ||
| N1 | 303 (24.5) | 29 (18.1) | 150 (28.1) | 11 (20.0) | ||
| N2 | 139 (11.2) | 15 (9.4) | 71 (13.3) | 5 (9.1) | ||
| M stage (%) | <0.001 | <0.001 | ||||
| M1 | 983 (79.4) | 78 (48.8) | 408 (76.4) | 27 (49.1) | ||
| M2 | 255 (20.6) | 82 (51.2) | 126 (23.6) | 28 (50.9) | ||
| Stage (%) | <0.001 | <0.001 | ||||
| NA | 117 (9.5) | 27 (16.9) | 42 (7.9) | 9 (16.4) | ||
| I | 342 (27.6) | 16 (10.0) | 132 (24.7) | 3 (5.5) | ||
| II | 207 (16.7) | 17 (10.6) | 77 (14.4) | 6 (10.9) | ||
| III | 317 (25.6) | 18 (11.2) | 157 (29.4) | 9 (16.4) | ||
| IV | 255 (20.6) | 82 (51.2) | 126 (23.6) | 28 (50.9) | ||
| Surgery (%) | <0.001 | <0.001 | ||||
| No | 288 (23.3) | 114 (71.2) | 123 (23.0) | 40 (72.7) | ||
| Yes | 950 (76.7) | 46 (28.7) | 411 (77.0) | 15 (27.3) | ||
| Tumor size (mm, median [IQR]) | 21.00 [12.00, 42.00] | 32.00 [20.00, 55.00] | <0.001 | 22.00 [12.00, 41.00] | 40.00 [25.50, 53.00] | <0.001 |
| Removed nodes (median [IQR]) | 3.00 [0.00, 15.00] | 0.00 [0.00, 1.00] | <0.001 | 3.00 [0.00, 15.00] | 0.00 [0.00, 0.00] | <0.001 |
| Lymph node examine (%) | <0.001 | <0.001 | ||||
| No | 522 (42.2) | 115 (71.9) | 222 (41.6) | 41 (74.5) | ||
| Yes | 716 (57.8) | 45 (28.1) | 312 (58.4) | 14 (25.5) | ||
CSD = cancer-specific death; NOS = no other specific; NA = not available. The p-values for T-stage and N stage were calculated excluding Tx and Nx.
Table 2.
Performance of the COX-PH and DeepSurv models.
| Model | Group | C-index | AUCs | Brier Scores | ||||
|---|---|---|---|---|---|---|---|---|
| 3-Month | 6-Month | 9-Month | 3-Month | 6-Month | 9-Month | |||
| Cox-PH | The training set (n = 1398) | 0.878 | 0.898 | 0.897 | 0.894 | 0.262 | 0.252 | 0.246 |
| The validation set (n = 589) | 0.866 | 0.874 | 0.922 | 0.908 | 0.282 | 0.265 | 0.265 | |
| DeepSurv | The internal validation set (n = 269) | 0.869 | 0.874 | 0.871 | 0.871 | 0.068 | 0.079 | 0.085 |
| The validation set (n = 589) | 0.871 | 0.878 | 0.891 | 0.891 | 0.058 | 0.070 | 0.080 | |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
MDPI and ACS Style
Yin, M.; Lin, J.; Liu, L.; Gao, J.; Xu, W.; Yu, C.; Qu, S.; Liu, X.; Qian, L.; Xu, C.; et al. Development of a Deep Learning Model for Malignant Small Bowel Tumors Survival: A SEER-Based Study. Diagnostics 2022, 12, 1247. https://doi.org/10.3390/diagnostics12051247
AMA Style
Yin M, Lin J, Liu L, Gao J, Xu W, Yu C, Qu S, Liu X, Qian L, Xu C, et al. Development of a Deep Learning Model for Malignant Small Bowel Tumors Survival: A SEER-Based Study. Diagnostics. 2022; 12(5):1247. https://doi.org/10.3390/diagnostics12051247
Chicago/Turabian StyleYin, Minyue, Jiaxi Lin, Lu Liu, Jingwen Gao, Wei Xu, Chenyan Yu, Shuting Qu, Xiaolin Liu, Lijuan Qian, Chunfang Xu, and et al. 2022. "Development of a Deep Learning Model for Malignant Small Bowel Tumors Survival: A SEER-Based Study" Diagnostics 12, no. 5: 1247. https://doi.org/10.3390/diagnostics12051247
Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.
