Using Machine Learning to Revise the AJCC Staging System for Neuroendocrine Tumors of the Pancreas
Simple Summary
Abstract
1. Introduction
2. Materials and Methods
2.1. Data
2.2. EACCD
2.3. EACCD Prognostic System
2.4. Software
3. Results
3.1. Survival Curves of AJCC TNM Staging on SEER Data
3.2. EACCD Based Prognostic System in T, N, M
3.3. EACCD Based Prognostic System in T, N, M, A
4. Discussion
4.1. Number of Prognostic Groups
4.2. Partitioning Patients by 5-Year Overall Survival Using EACCD
4.3. Comparison of EACCD TNM with AJCC Staging
4.4. Comparison of EACCD TNMA with EACCD TNM
4.5. Clinical Applications of EACCD TNM and TNMA Systems
4.6. Effect of Variable Inclusion on EACCD Performance
4.7. Integrating EACCD into Clinical Decision Support
4.8. Limitations
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
| AJCC | American Joint Committee on Cancer |
| C-index | Concordance Index |
| TNM | Tumor, Lymph Node, and Metastasis |
| EACCD | Ensemble Algorithm for Clustering Cancer Data |
| SEER | Surveillance, Epidemiology, and End Results |
| pNETs | pancreatic neuroendocrine tumors |
| ENETS | European Neuroendocrine Tumor Society |
| AI | Artificial Intelligence |
References
- Siegel, R.L.; Kratzer, T.B.; Giaquinto, A.N.; Sung, H.; Jemal, A. Cancer statistics, 2025. CA A Cancer J. Clin. 2025, 75, 10–45. [Google Scholar] [CrossRef]
- Teo, R.Y.A.; Teo, T.Z.; Tai, D.W.M.; Tan, D.M.; Ong, S.; Goh, B.K.P. Systematic review of current prognostication systems for pancreatic neuroendocrine neoplasms. Surgery 2019, 165, 672–685. [Google Scholar] [CrossRef] [PubMed]
- Sun, J. Pancreatic Neuroendocrine Tumors. Intractable Rare Dis. Res. 2017, 6, 21–28. [Google Scholar] [CrossRef] [PubMed]
- Ro, C.; Chai, W.; Yu, V.E.; Yu, R. Pancreatic neuroendocrine tumors: Biology, diagnosis, and treatment. Chin. J. Cancer 2013, 32, 312–324. [Google Scholar] [CrossRef]
- Akirov, A.; Larouche, V.; Alshehri, S.; Asa, S.L.; Ezzat, S. Treatment Options for Pancreatic Neuroendocrine Tumors. Cancers 2019, 11, 828. [Google Scholar] [CrossRef] [PubMed]
- Yang, M.; Zeng, L.; Zhang, Y.; Wang, W.-G.; Wang, L.; Ke, N.-W.; Liu, X.-B.; Tian, B.-L. TNM Staging of Pancreatic Neuroendocrine Tumors. Medicine 2016, 95, e660. [Google Scholar] [CrossRef]
- American Joint Committee on Cancer. AJCC Cancer Staging Manual, 8th ed.; Corrected at 3rd printing; AJCC; American Joint Committee on Cancer: Chicago, IL, USA, 2017. [Google Scholar]
- Li, X.; Gou, S.; Liu, Z.; Ye, Z.; Wang, C. Assessment of the American Joint Commission on Cancer 8th Edition Staging System for Patients with Pancreatic Neuroendocrine Tumors: A Surveillance, Epidemiology, and End Results analysis. Cancer Med. 2018, 7, 626–634. [Google Scholar] [CrossRef]
- Rindi, G.; Dasari, A.; Hope, T.A.; Mittra, E.; Thomas, S.A.; Wei, A.C. AJCC Cancer Staging System Neuroendocrine Tumors of the Pancreas, 9th ed.; American College of Surgeons: Chicago, IL, USA, 2024. [Google Scholar]
- Heng, X.; Chen, B.; Zhao, K.; Li, J.; Wu, W.; Peng, Y.; Zhong, R. Comparison of nomogram for Primary Nonfunctional Pancreatic Neuroendocrine Tumors based on the 7th vs 8th edition of the AJCC cancer staging manual. PLoS ONE 2023, 18, e0284930. [Google Scholar] [CrossRef] [PubMed]
- Chen, D.; Xing, K.; Henson, D.; Sheng, L.; Schwartz, A.M.; Cheng, X. Developing Prognostic Systems of Cancer Patients by Ensemble Clustering. BioMed Res. Int. 2009, 2009, 632786. [Google Scholar] [CrossRef]
- Qi, R.; Wu, D.; Sheng, L.; Henson, D.; Schwartz, A.; Xu, E.; Xing, K.; Chen, D. On an ensemble algorithm for clustering cancer patient data. BMC Syst. Biol. 2013, 7, S9. [Google Scholar] [CrossRef]
- Wang, H.; Chen, D.; Pan, Q.; Hueman, M.T. Using Weighted Differences in Hazards as Effect Sizes for Survival Data. J. Stat. Theory Pract. 2022, 16, 12. [Google Scholar] [CrossRef]
- Hueman, M.T.; Wang, H.; Yang, C.Q.; Sheng, L.; Henson, D.E.; Schwartz, A.M.; Chen, D. Creating prognostic systems for cancer patients: A demonstration using breast cancer. Cancer Med. 2018, 7, 3611–3621. [Google Scholar] [CrossRef]
- Yang, C.Q.; Gardiner, L.; Wang, H.; Hueman, M.T.; Chen, D. Creating Prognostic Systems for Well-Differentiated Thyroid Cancer Using Machine Learning. Front. Endocrinol. 2019, 10, 288. [Google Scholar] [CrossRef]
- Henson, D.E.; Hueman, M.T.; Chen, D.; Patel, J.A.; Wang, H.; Schwartz, A.M. The anatomy of the TNM for colon cancer. J. Gastrointest. Oncol. 2017, 8, 12–19. [Google Scholar] [CrossRef]
- Hueman, M.; Wang, H.; Liu, Z.; Henson, D.; Nguyen, C.; Park, D.; Sheng, L.; Chen, D. Expanding TNM for lung cancer through machine learning. Thorac. Cancer 2021, 12, 1423–1430. [Google Scholar] [CrossRef]
- Grimley, P.M.; Liu, Z.; Darcy, K.M.; Hueman, M.T.; Wang, H.; Sheng, L.; Henson, D.E.; Chen, D. A prognostic system for epithelial ovarian carcinomas using machine learning. Acta Obs. Gynecol. Scand. 2021, 100, 1511–1519. [Google Scholar] [CrossRef] [PubMed]
- Praiss, A.M.; Huang, Y.; Clair, C.M.S.; Tergas, A.I.; Melamed, A.; Khoury-Collado, F.; Hou, J.Y.; Hu, J.; Hur, C.; Hershman, D.L.; et al. Using machine learning to create prognostic systems for endometrial cancer. Gynecol. Oncol. 2020, 159, 744–750. [Google Scholar] [CrossRef] [PubMed]
- Wang, H.; Liu, Z.; Yang, J.; Sheng, L.; Chen, D. Using Machine Learning to Expand the Ann Arbor Staging System for Hodgkin and Non-Hodgkin Lymphoma. BioMedInformatics 2023, 3, 514–525. [Google Scholar] [CrossRef]
- Yang, C.Q.; Wang, H.; Liu, Z.; Hueman, M.T.; Bhaskaran, A.; Henson, D.E.; Sheng, L.; Chen, D. Integrating additional factors into the TNM staging for cutaneous melanoma by machine learning. PLoS ONE 2021, 16, e0257949. [Google Scholar] [CrossRef] [PubMed]
- National Cancer Institute, DCCPS, Surveillance Research Program. Surveillance, Epidemiology, and End Results (SEER) Program Research Data (2000–2019). National Cancer Institute 2024. Available online: https://seer.cancer.gov/ (accessed on 2 June 2025).
- Liu, M.; Sun, X.; Zhang, Z.; Xu, X.; Yu, X.; Zhuo, Q.; Ji, S. The clinical characteristics and survival associations of pancreatic neuroendocrine tumors: Does age matter? Gland. Surg 2021, 10, 574–583. [Google Scholar] [CrossRef]
- Kaplan, E.L.; Meier, P. Nonparametric Estimation from Incomplete Observations. J. Am. Stat. Assoc. 1958, 53, 457–481. [Google Scholar] [CrossRef]
- Huan, W.; Li, S.; Dechang, C. Perspective Chapter: Using Effect Sizes to Study the Survival Difference between Two Groups. In Recent Advances in Biostatistics; IntechOpen: London, UK, 2023. [Google Scholar]
- Bien, J.; Tibshirani, R. Hierarchical Clustering With Prototypes via Minimax Linkage. J. Am. Stat. Assoc. 2011, 106, 1075–1084. [Google Scholar] [CrossRef]
- Harrell, F.E.; Lee, K.L.; Mark, D.B. Multivariable prognostic models: Issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat. Med. 1996, 15, 361–387. [Google Scholar] [CrossRef]
- R Core Team. R: A Language and Environment for Statistical Computing; R Core Team: Vienna, Austria, 2023. [Google Scholar]
- Kang, L.; Chen, W.; Petrick, N.A.; Gallas, B.D. Comparing two correlated C indices with right-censored survival outcome: A one-shot nonparametric approach. Stat. Med. 2015, 34, 685–703. [Google Scholar] [CrossRef]
- Erkaya, M.; Inal, E.; Benlice, C.; Kuzu, M.; Emre, G. A Methodological and Survival Comparison of NCDB and SEER Database for Colon Cancer Research. J. Surg. Oncol. 2025, 132, 114–134. [Google Scholar] [CrossRef] [PubMed]
- Liu, P.-H.; Pruitt, S.L.; Singal, A.G.; Murphy, C.C. Comparing SEER and NCDB: A case study using colorectal cancer. Cancer Causes Control. 2024, 35, 1477–1485. [Google Scholar] [CrossRef] [PubMed]
- Kos-Kudła, B.; Castaño, J.P.; Denecke, T.; Grande, E.; Kjaer, A.; Koumarianou, A.; de Mestier, L.; Partelli, S.; Perren, A.; Stättner, S.; et al. European Neuroendocrine Tumour Society (ENETS) 2023 guidance paper for nonfunctioning pancreatic neuroendocrine tumours. J. Neuroendocrinol. 2023, 35, e13343. [Google Scholar] [CrossRef] [PubMed]









| Factors | Levels | Definitions |
|---|---|---|
| Primary Tumor (T) | T1 | Tumor limited to the pancreas, ≤2 cm in greatest dimension |
| T2 | Tumor limited to the pancreas, >2 cm but ≤4 cm in greatest dimension | |
| T3 | Tumor limited to the pancreas, >4 cm in greatest dimension; or tumor invading the duodenum, ampulla of Vater, or common bile duct | |
| T4 | Tumor invading adjacent organs (stomach, spleen, colon, adrenal gland) or the wall of large vessels (celiac axis, superior mesenteric artery/vein, splenic artery/vein, gastroduodenal artery/vein, portal vein) | |
| Regional Nodes Positive (N) | N0 | No regional lymph node involvement |
| N1 | Regional lymph node involvement | |
| Metastasis (M) | M0 | No distant metastasis |
| M1 | Distant metastases | |
| Age (A) | A1 | 0 ≤ Age < 50 |
| A2 | 50 ≤ Age |
| T | N | M | A | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| T1 | T2 | T3 | T4 | N0 | N1 | M0 | M1 | A1 | A2 | ||
| Dataset 1 | 41% | 34% | 22% | 3% | 78% | 22% | 83% | 17% | |||
| Dataset 2 | 41% | 34% | 22% | 3% | 79% | 21% | 84% | 16% | 20% | 80% | |
| Stage 1 | Stage 2 | Stage 3 | Stage 4 |
|---|---|---|---|
| T1N0M0 (1254) | T2N0M0 (745) | T4N0M0 (25) | T1N0M1 (22) |
| T3N0M0 (252) | T1N1M0 (56) | T2N0M1 (150) | |
| T2N1M0 (142) | T3N0M1 (86) | ||
| T3N1M0 (233) | T4N0M1 (35) | ||
| T4N1M0 (21) | T1N1M1 (15) | ||
| T2N1M1 (81) | |||
| T3N1M1 (131) | |||
| T4N1M1 (30) |
| Group 1 | Group 2 | Group 3 | Group 4 |
|---|---|---|---|
| T1N0M0 | T1N1M0 | T3N1M1 | T4N1M1 |
| T2N0M0 | T2N1M0 | T4N1M0 | T4N0M1 |
| T3N0M0 | T1N1M1 | T4N0M0 | T2N0M1 |
| T3N1M0 | T1N0M1 | T2N1M1 | |
| T3N0M1 |
| Group 1 | Group 2 | Group 3 | Group 4 | Group 5 |
|---|---|---|---|---|
| T1N0M0A1 | T3N1M0A1 | T3N1M0A2 | T3N1M1A2 | T2N1M1A1 |
| T2N0M0A1 | T3N0M0A2 | T3N0M1A1 | T3N0M1A2 | T3N1M1A1 |
| T3N0M0A1 | T1N0M0A2 | T1N1M0A2 | T4N0M0A2 | T2N0M1A2 |
| T2N1M0A1 | T2N0M0A2 | T2N1M0A2 | T4N1M0A2 | T2N1M1A2 |
| T1N0M1A2 | T4N0M1A2 | |||
| T2N0M1A1 | T4N1M1A2 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Hillman, J.; Clark, Q.; Rehm, L.; Ahmed, A.E.; Chen, D. Using Machine Learning to Revise the AJCC Staging System for Neuroendocrine Tumors of the Pancreas. Cancers 2025, 17, 3658. https://doi.org/10.3390/cancers17223658
Hillman J, Clark Q, Rehm L, Ahmed AE, Chen D. Using Machine Learning to Revise the AJCC Staging System for Neuroendocrine Tumors of the Pancreas. Cancers. 2025; 17(22):3658. https://doi.org/10.3390/cancers17223658
Chicago/Turabian StyleHillman, Jacob, Quinn Clark, Liam Rehm, Anwar E. Ahmed, and Dechang Chen. 2025. "Using Machine Learning to Revise the AJCC Staging System for Neuroendocrine Tumors of the Pancreas" Cancers 17, no. 22: 3658. https://doi.org/10.3390/cancers17223658
APA StyleHillman, J., Clark, Q., Rehm, L., Ahmed, A. E., & Chen, D. (2025). Using Machine Learning to Revise the AJCC Staging System for Neuroendocrine Tumors of the Pancreas. Cancers, 17(22), 3658. https://doi.org/10.3390/cancers17223658

