Multi-Task Deep Learning on MRI for Tumor Segmentation and Treatment Response Prediction in an Experimental Model of Hepatocellular Carcinoma
Abstract
1. Introduction
2. Materials and Methods
2.1. Cell Lines and Cell Culture
2.2. Orthotopic Tumor Development
2.3. Therapeutic Strategies: NK Cell Immunotherapy and Sorafenib Administration
2.4. MRI Acquisition
2.5. Model Architecture
2.6. Training Procedure
2.7. Radiomics Analysis
2.8. Histological Analysis
2.9. Statistical Analysis
3. Results
3.1. Dataset and Radiomics Analysis of MRI Sequences
3.2. Tumor Segmentation Performance
3.3. Classification of Multi-Class Treatment Outcomes
3.4. Correlation of MRI-Derived Features with Histological Biomarkers
4. Discussion
5. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
| AI | Artificial Intelligence |
| AUROC | Area Under the Receiver Operating Characteristic |
| CE-T1w | Contrast-Enhanced T1-Weighted |
| CNN | Convolutional Neural Network |
| FA | Flip Angle |
| FOV | Field of View |
| HCC | Hepatocellular Carcinoma |
| IoU | Intersection Over Union |
| NK | Natural Killer |
| NSA | Number of Signals Acquisitions |
| RMSE | Root Mean Square Error |
| ST | Slice Thickness |
| TE | Echo Time |
| TR | Repetition Time |
| T1w | T1-Weighted |
| T2w | T2-Weighted |
References
- Siegel, R.L.; Miller, K.D.; Wagle, N.S.; Jemal, A. Cancer statistics, 2023. CA Cancer J. Clin. 2023, 73, 17–48. [Google Scholar] [CrossRef]
- Llovet, J.M.; Kelley, R.K.; Villanueva, A.; Singal, A.G.; Pikarsky, E.; Roayaie, S.; Lencioni, R.; Koike, K.; Zucman-Rossi, J. Hepatocellular carcinoma. Nat. Rev. Dis. Primers 2021, 7, 6. [Google Scholar] [CrossRef]
- Llovet, J.M.; Ricci, S.; Mazzaferro, V.; Hilgard, P.; Gane, E.; Blanc, J.F.; de Oliveira, A.C.; Santoro, A.; Raoul, J.-L.; Forner, A.; et al. Sorafenib in advanced hepatocellular carcinoma. N. Engl. J. Med. 2008, 359, 378–390. [Google Scholar] [CrossRef]
- Cheng, M.; Chen, Y.; Xiao, W.; Sun, R.; Tian, Z. NK cell-based immunotherapy for malignant diseases. Cell. Mol. Immunol. 2013, 10, 230–252. [Google Scholar] [CrossRef]
- Sprinzl, M.F.; Reisinger, F.; Puschnik, A.; Ringelhan, M.; Ackermann, K.; Hartmann, D.; Schiemann, M.; Weinmann, A.; Galle, P.R.; Schuchmann, M.; et al. Sorafenib perpetuates cellular anticancer effector functions by modulating the crosstalk between macrophages and natural killer cells. Hepatology 2013, 57, 2358–2368. [Google Scholar] [CrossRef]
- Eresen, A.; Pang, Y.; Zhang, Z.; Hou, Q.; Chen, Z.; Yu, G.; Wang, Y.; Yaghmai, V.; Zhang, Z. Sorafenib plus memory like natural killer cell combination therapy in hepatocellular carcinoma. Am. J. Cancer Res. 2024, 14, 344–354. [Google Scholar] [CrossRef]
- Eresen, A.; Zhang, Z.; Yu, G.; Hou, Q.; Chen, Z.; Yu, Z.; Yaghmai, V.; Zhang, Z. Sorafenib plus memory-like natural killer cell immunochemotherapy boosts treatment response in liver cancer. BMC Cancer 2024, 24, 1215. [Google Scholar] [CrossRef] [PubMed]
- Chiou, V.L.; Burotto, M. Pseudoprogression and Immune-Related Response in Solid Tumors. J. Clin. Oncol. 2015, 33, 3541–3543. [Google Scholar] [CrossRef]
- Seymour, L.; Bogaerts, J.; Perrone, A.; Ford, R.; Schwartz, L.H.; Mandrekar, S.; Lin, N.U.; Litière, S.; Dancey, J.; Chen, A.; et al. iRECIST: Guidelines for response criteria for use in trials testing immunotherapeutics. Lancet Oncol. 2017, 18, e143–e152, Erratum in Lancet Oncol. 2017, 18, e143–e152. [Google Scholar] [CrossRef]
- Wang, W.; Bai, W.; Wang, E.; Zhao, Y.; Liu, L.; Yang, M.; Cai, H.; Xia, D.; Zhang, L.; Niu, J.; et al. mRECIST response combined with sorafenib-related adverse events is superior to either criterion alone in predicting survival in HCC patients treated with TACE plus sorafenib. Int. J. Cancer 2016, 140, 390–399. [Google Scholar] [CrossRef] [PubMed]
- Hodi, F.S.; Ballinger, M.; Lyons, B.; Soria, J.-C.; Nishino, M.; Tabernero, J.; Powles, T.; Smith, D.; Hoos, A.; McKenna, C.; et al. Immune-Modified Response Evaluation Criteria In Solid Tumors (imRECIST): Refining Guidelines to Assess the Clinical Benefit of Cancer Immunotherapy. J. Clin. Oncol. 2018, 36, 850–858. [Google Scholar] [CrossRef]
- LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 521, 436–444. [Google Scholar] [CrossRef] [PubMed]
- Aggarwal, R.; Sounderajah, V.; Martin, G.; Ting, D.S.W.; Karthikesalingam, A.; King, D.; Ashrafian, H.; Darzi, A. Diagnostic accuracy of deep learning in medical imaging: A systematic review and meta-analysis. npj Digit. Med. 2021, 4, 65. [Google Scholar] [CrossRef] [PubMed]
- Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation; Springer: Cham, Switzerland, 2015; pp. 234–241. [Google Scholar] [CrossRef]
- Hosna, A.; Merry, E.; Gyalmo, J.; Alom, Z.; Aung, Z.; Azim, M.A. Transfer learning: A friendly introduction. J. Big Data 2022, 9, 102. [Google Scholar] [CrossRef] [PubMed]
- Srinivas, C.; S., N.P.K.; Zakariah, M.; Alothaibi, Y.A.; Shaukat, K.; Partibane, B.; Awal, H. Deep Transfer Learning Approaches in Performance Analysis of Brain Tumor Classification Using MRI Images. J. Health Eng. 2022, 2022, 3264367. [Google Scholar] [CrossRef]
- Zhou, Z.; Siddiquee, M.M.R.; Tajbakhsh, N.; Liang, J. UNet++: A Nested U-Net Architecture for Medical Image Segmentation. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2018; Volume 11045. [Google Scholar] [CrossRef]
- Tan, M.; Le, Q. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. In Proceedings of the 36th International Conference on Machine Learning, Long Beach, CA, USA, 9–15 June 2019; Volume 97, pp. 6105–6114. Available online: https://proceedings.mlr.press/v97/tan19a.html (accessed on 6 October 2025).
- Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V. Scikit-Learn: Machine Learning in Python. J. Mach. Learn. Res. 2011, 12, 2825–2830. [Google Scholar]
- van Griethuysen, J.J.M.; Fedorov, A.; Parmar, C.; Hosny, A.; Aucoin, N.; Narayan, V.; Beets-Tan, R.G.H.; Fillion-Robin, J.-C.; Pieper, S.; Aerts, H.J.W.L. Computational Radiomics System to Decode the Radiographic Phenotype. Cancer Res. 2017, 77, e104–e107. [Google Scholar] [CrossRef]
- Zhen, S.-H.; Cheng, M.; Tao, Y.-B.; Wang, Y.-F.; Juengpanich, S.; Jiang, Z.-Y.; Jiang, Y.-K.; Yan, Y.-Y.; Lu, W.; Lue, J.-M.; et al. Deep Learning for Accurate Diagnosis of Liver Tumor Based on Magnetic Resonance Imaging and Clinical Data. Front. Oncol. 2020, 10, 680. [Google Scholar] [CrossRef]
- Yu, G.; Zhang, Z.; Eresen, A.; Hou, Q.; Garcia, E.E.; Yu, Z.; Abi-Jaoudeh, N.; Yaghmai, V.; Zhang, Z. MRI radiomics to monitor therapeutic outcome of sorafenib plus IHA transcatheter NK cell combination therapy in hepatocellular carcinoma. J. Transl. Med. 2024, 22, 76. [Google Scholar] [CrossRef]
- Yu, G.; Eresen, A.; Zhang, Z.; Hou, Q.; Amirrad, F.; Webster, S.; Nauli, S.; Yaghmai, V.; Zhang, Z. AI-Derived MRI Biomarkers Using Vision Transformer for Predicting Combination Immunotherapy Outcomes in Liver Cancer. In Proceedings of the 2025 ISMRM & ISMRT Annual Meeting, Honolulu, HI, USA, 10–15 May 2025. [Google Scholar]
- von Schacky, C.E.; Wilhelm, N.J.; Schäfer, V.S.; Leonhardt, Y.; Gassert, F.G.; Foreman, S.C.; Gassert, F.T.; Jung, M.; Jungmann, P.M.; Russe, M.F.; et al. Multitask Deep Learning for Segmentation and Classification of Primary Bone Tumors on Radiographs. Radiology 2021, 301, 398–406. [Google Scholar] [CrossRef]
- Li, Y.; Xu, Z.; An, C.; Chen, H.; Li, X. Multi-Task Deep Learning Approach for Simultaneous Objective Response Prediction and Tumor Segmentation in HCC Patients with Transarterial Chemoembolization. J. Pers. Med. 2022, 12, 248. [Google Scholar] [CrossRef] [PubMed]
- Kowald, A.; Barrantes, I.; Möller, S.; Palmer, D.; Escobar, H.M.; Schwerk, A.; Fuellen, G. Transfer learning of clinical outcomes from preclinical molecular data, principles and perspectives. Briefings Bioinform. 2022, 23, bbac133. [Google Scholar] [CrossRef] [PubMed]
- Johnson, P.; Zhou, Q.; Dao, D.Y.; Lo, Y.M.D. Circulating biomarkers in the diagnosis and management of hepatocellular carcinoma. Nat. Rev. Gastroenterol. Hepatol. 2022, 19, 670–681. [Google Scholar] [CrossRef] [PubMed]






| Term | Description & Relevance in This Study |
|---|---|
| RECIST | Response Evaluation Criteria in Solid Tumors. A standardized guideline that quantifies tumor response to therapy based on measurable size changes. Mentioned to highlight traditional limitations of size-based criteria in evaluating therapeutic response. |
| mRECIST | Modified RECIST. Incorporates measurements of the viable enhancing portion of tumors after therapy. Discussed as an advancement over RECIST but still limited for immunotherapy evaluation. |
| imRECIST | Immune-modified RECIST. Adjusts RECIST criteria to account for immune-related responses such as pseudoprogression. |
| U-Net++ architecture | A convolutional encoder–decoder network with nested skip connections that improve segmentation accuracy. Used as the backbone for precise tumor delineation. |
| EfficientNet-B0 encoder | A pre-trained convolutional encoder optimized for efficiency and accuracy. Used here to extract multi-scale features from MRI data and improve generalization with limited samples. |
| Convolutional Neural Network (CNN) | A deep-learning model that captures spatial features in imaging data. Forms the fundamental structure underlying both the encoder and decoder parts of the network. |
| Multi-task deep learning model | A neural-network framework designed to perform multiple related tasks simultaneously—in this study, tumor segmentation and treatment outcome classification—to enhance efficiency and performance. |
| Transfer learning | Leveraging pre-trained weights from large-scale datasets to improve model performance on smaller medical datasets. |
| Dice coefficient | Statistical metric quantifying overlap between predicted and ground-truth segmentation masks. Used to assess segmentation accuracy. |
| AUROC (Area Under the Receiver Operating Characteristic Curve) | Measures model discrimination ability across classes. Applied to evaluate classification performance for treatment prediction. |
| Class | Precision | Recall | F1-Score |
|---|---|---|---|
| Control | 0.9899 | 0.8066 | 0.8889 |
| NK | 0.8249 | 0.8690 | 0.8464 |
| Sorafenib | 0.7441 | 0.9290 | 0.8263 |
| Combination | 0.8417 | 0.8069 | 0.8239 |
| Macro Average | 0.8501 | 0.8529 | 0.8464 |
| Weighted Average | 0.8647 | 0.8497 | 0.8515 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Yu, G.; Zhang, Z.; Eresen, A.; Hou, Q.; Yaghmai, V.; Zhang, Z. Multi-Task Deep Learning on MRI for Tumor Segmentation and Treatment Response Prediction in an Experimental Model of Hepatocellular Carcinoma. Diagnostics 2025, 15, 2844. https://doi.org/10.3390/diagnostics15222844
Yu G, Zhang Z, Eresen A, Hou Q, Yaghmai V, Zhang Z. Multi-Task Deep Learning on MRI for Tumor Segmentation and Treatment Response Prediction in an Experimental Model of Hepatocellular Carcinoma. Diagnostics. 2025; 15(22):2844. https://doi.org/10.3390/diagnostics15222844
Chicago/Turabian StyleYu, Guangbo, Zigeng Zhang, Aydin Eresen, Qiaoming Hou, Vahid Yaghmai, and Zhuoli Zhang. 2025. "Multi-Task Deep Learning on MRI for Tumor Segmentation and Treatment Response Prediction in an Experimental Model of Hepatocellular Carcinoma" Diagnostics 15, no. 22: 2844. https://doi.org/10.3390/diagnostics15222844
APA StyleYu, G., Zhang, Z., Eresen, A., Hou, Q., Yaghmai, V., & Zhang, Z. (2025). Multi-Task Deep Learning on MRI for Tumor Segmentation and Treatment Response Prediction in an Experimental Model of Hepatocellular Carcinoma. Diagnostics, 15(22), 2844. https://doi.org/10.3390/diagnostics15222844

