Next Article in Journal
Augmented Reality-Guided Extraction of Fully Impacted Lower Third Molars Based on Maxillofacial CBCT Scans
Next Article in Special Issue
Nuclei Detection and Segmentation of Histopathological Images Using a Feature Pyramidal Network Variant of a Mask R-CNN
Previous Article in Journal
Reduction of Oxygen Production by Algal Cells in the Presence of O-Chlorobenzylidene Malononitrile
Previous Article in Special Issue
MurSS: A Multi-Resolution Selective Segmentation Model for Breast Cancer
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Bioengineering
Volume 11
Issue 6
10.3390/bioengineering11060624
bioengineering-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Multi-Scale Digital Pathology Patch-Level Prostate Cancer Grading Using Deep Learning: Use Case Evaluation of DiagSet Dataset

by
Tanaya Kondejkar
1,
Salah Mohammed Awad Al-Heejawi
1,
Anne Breggia
2,
Bilal Ahmad
3,
Robert Christman
3,
Stephen T. Ryan
3 and
Saeed Amal
4,*
1
College of Engineering, Northeastern University, Boston, MA 02115, USA
2
MaineHealth Institute for Research, Scarborough, ME 04074, USA
3
Maine Medical Center, Portland, ME 04102, USA
4
The Roux Institute, Department of Bioengineering, College of Engineering, Northeastern University, Boston, MA 02115, USA
*
Author to whom correspondence should be addressed.
Bioengineering 2024, 11(6), 624; https://doi.org/10.3390/bioengineering11060624
Submission received: 6 May 2024 / Revised: 3 June 2024 / Accepted: 12 June 2024 / Published: 18 June 2024
(This article belongs to the Special Issue Computational Pathology and Artificial Intelligence)
Browse Figures
Review Reports Versions Notes

Abstract

:
Prostate cancer remains a prevalent health concern, emphasizing the critical need for early diagnosis and precise treatment strategies to mitigate mortality rates. The accurate prediction of cancer grade is paramount for timely interventions. This paper introduces an approach to prostate cancer grading, framing it as a classification problem. Leveraging ResNet models on multi-scale patch-level digital pathology and the Diagset dataset, the proposed method demonstrates notable success, achieving an accuracy of 0.999 in identifying clinically significant prostate cancer. The study contributes to the evolving landscape of cancer diagnostics, offering a promising avenue for improved grading accuracy and, consequently, more effective treatment planning. By integrating innovative deep learning techniques with comprehensive datasets, our approach represents a step forward in the pursuit of personalized and targeted cancer care.

1. Introduction

Prostate cancer stands as a major health concern globally, necessitating precise diagnostic and grading methodologies for effective therapeutic interventions. Prostate cancer is characterized by abnormal cell growth in the prostate gland. It is the second most common cancer in the world and the most common cancer in men. Each year, over 1.4 million new cases of prostate cancer are found, causing more than 375,000 deaths [1]. It typically starts in the cells of the prostate gland and can grow slowly, initially confined to the gland itself, or it may spread rapidly to other parts of the body if left untreated. Early detection through screening tests such as prostate-specific antigen (PSA) tests and digital rectal exams (DREs) is crucial for timely treatment and improved outcomes. The processing and scanning of tissue biopsies in the laboratory produce large whole-slide images (WSIs), enhancing workflow efficiency and reproducibility. One characteristic of PCa is its tendency to be “non-aggressive”, which can complicate treatment decisions and determine the necessity of more serious interventions. To address this challenge, the Gleason Grading System classifies tumors into numerical risk groups known as WHO/ISUP grade groups, as adopted by the International Society of Urological Pathology (ISUP) and the World Health Organization (WHO). The conventional grading of prostate cancer involves challenges marked by intra- and inter-observer variability, potentially leading to suboptimal treatment decisions. Advancements in imaging techniques and molecular biomarkers offer promising avenues for enhancing the precision of prostate cancer diagnosis and risk stratification, guiding personalized treatment strategies tailored to individual patients’ needs. Integrating these innovative approaches into clinical practice holds the potential to improve prognostic accuracy and optimize therapeutic outcomes for men diagnosed with prostate cancer. To address these complexities, the integration of machine learning (ML) and deep learning (DL) techniques has emerged as a promising frontier, revolutionizing the landscape of prostate cancer diagnosis and grading. These technologies can enhance the accuracy of diagnostic processes, reducing the variability associated with human interpretation. This, in turn, can lead to more reliable and personalized treatment decisions for individuals diagnosed with prostate cancer. One essential aspect of this diagnostic is the Gleason score grading system. The Gleason score provides crucial insights into the aggressiveness of prostate cancer by evaluating the patterns observed in cancerous tissue under a microscope. Higher Gleason scores correlate with more aggressive cancer, aiding clinicians in determining optimal treatment strategies tailored to individual patients. By integrating the Gleason score into diagnostic and treatment protocols, healthcare professionals can enhance precision and personalized care for individuals with prostate cancer. This integration not only facilitates more accurate grading but also allows for the development of predictive models that assist clinicians in tailoring treatment plans based on individual patient characteristics. By leveraging advanced computational methods, machine learning models can analyze vast amounts of medical data, including imaging and pathology reports, to identify subtle patterns and markers indicative of the severity of prostate cancer. This approach not only assists in more accurate grading but also allows for the development of predictive models that can help healthcare professionals tailor treatment plans based on individual patient characteristics. In this research paper, we perform a comparative study of ResNet models within the domain of prostate cancer diagnosis and grading. Our primary objective is to surpass the accuracies achieved in the previous literature by leveraging the advancements in deep learning techniques. Through our research, we wish to contribute to the ongoing efforts in revolutionizing the landscape of prostate cancer diagnosis and grading, with the goal of enhancing patient outcomes and personalized care. Recent advancements in artificial intelligence (AI) and machine learning (ML) have reshaped prostate cancer diagnosis and Gleason grading. Goldenberg et al. [2] provide an overview of AI and ML’s potential in prostate cancer management, showing performance comparable to traditional diagnostic methods. Abraham and Nair [3] developed a deep learning algorithm for histopathologic diagnosis and Gleason grading, focusing on the PROSTATEx-2 2017 dataset. A study in The Lancet Oncology also presents an AI system for prostate biopsies, demonstrating accuracy and performance similar to expert pathologists [4]. SVM-based analysis in the work by Bhattacharjee et al. [5] enriches the AI landscape, accurately classifying Gleason grading based on biopsy-derived image features. These studies highlight the promising trajectory of AI and ML applications, offering accurate and efficient methodologies for prostate cancer grading. Goldenberg et al. [2] explored ML and deep learning (DL) techniques such as support vector machines (SVMs) and convolutional neural networks (CNNs) for diagnostic imaging in prostate cancer; using the PROSTATEx challenge dataset, the study shows performance comparable to radiologists. In The Lancet Oncology [6], a study focuses on an AI system for prostate biopsies, achieving remarkable performance in distinguishing benign and malignant biopsy cores. Abraham and Nair [4] introduced a novel approach for grading prostate cancer using a VGG-16 convolutional neural network and an ordinal class classifier, contributing to histopathologic diagnosis and Gleason grading. Bhattacharjee et al. [5] presented a quantitative analysis of benign and malignant tumors in histopathology, predicting prostate cancer grading using SVM based on biopsy-derived images. The collective findings from these studies demonstrate AI and ML’s transformative potential in prostate cancer diagnosis and grading. In 2022, Kamal Hammouda et al. [7]. conducted a study at Radboud University Medical Center involving 3080 whole-slide images (WSIs). The study focused on multi-level binary classification into Gleason grades using a multi-stage classification-based deep learning approach. Their findings demonstrated an overall accuracy of 66.23%, showcasing the potential of deep learning methods in prostate cancer grading. These methods offer comparable performance to traditional approaches while presenting opportunities for automation, reducing workload, and providing expertise in resource-limited regions. In 2024, Wang et al. explored a novel method for diagnosing multiple types of cancer using an environmentally friendly and cost-effective approach [8]. This method addresses the need for accessible diagnostic tools, especially in remote or resource-limited regions, by using dried serum spots instead of traditional liquid blood storage. This technique not only reduces the environmental impact but also ensures the stability of metabolites. Table 1 summarizes the key findings in each paper.

2. Proposed Methods

The primary objective of our proposed work is to enhance the accuracy of prostate cancer diagnosis and Gleason grading using advanced machine learning techniques [9]. It aims to implement Resnet models and compare their accuracies for the better classification and grading of prostate cancer [10]. It also proposes a model which uses the segmentation model as a feature extractor for the CNN models trained. Our proposed work aims to advance prostate cancer diagnosis and Gleason grading through the utilization of the histopathological dataset, DiagSet. This dataset, comprising over 2.6 million tissue patches from 430 fully annotated scans, 4675 scans with binary diagnoses, and 46 scans independently diagnosed by histopathologists, provides a rich resource for in-depth analysis. It includes meticulously annotated tissue patches, binary diagnoses, and expert-assigned diagnoses (Accessible at https://github.com/michalkoziarski/DiagSet (accessed on 3 June 2024)) It provides a foundation for our research, ensuring a thorough investigation into the factors influencing model performance. Our approach is centered around ensembles of convolution neural networks operating on histopathological scans at different scales [11]. We implement a CNN framework tailored to the characteristics of DiagSet [12]. This involves the detection of cancerous tissue regions and the prediction of scan-level diagnosis [13,14].

Dataset

Each whole-slide image is systematically partitioned into 256 × 256 blocks, forming the basis for a detailed analysis using convolutional neural network (CNN) classifiers [13]. These classifiers categorize each block into one of nine distinct classes (Figure 1): scan background (BG); tissue background (T); normal, healthy tissue (N); acquisition artifact (A); or one of the five Gleason grades (R1R5) [14]. Leveraging the versatility of pre-trained ResNet models, specifically ResNet-18, ResNet-34, and ResNet-50, our methodology ensures a comprehensive examination of the dataset at varying levels of complexity [15,16]. This multi-model approach enhances the robustness of our analysis, capturing intricate features within each block and facilitating a nuanced understanding of prostate cancer pathology across different Gleason grades and tissue types [17,18,19,20]. Furthermore, this extends across multiple magnification levels, including 40×, 20×, 10×, and 5×. At each magnification, the systematic division and classification of 256 × 256 blocks are repeated, allowing for a comprehensive analysis of prostate cancer pathology at varying resolutions. By comparing the accuracies obtained at different magnifications, our approach aims to discern patterns and variations in classification performance, providing insights into the robustness and scalability of the proposed model across a spectrum of image resolutions. This multi-level evaluation enhances the reliability and generalizability of our findings, acknowledging the significance of adapting to diverse magnification contexts in the field of histopathological analysis.

3. Methodology

At each magnification level (40×, 20×, 10×, and 5×), structured methodology was implemented to prepare and assess the dataset with thoroughness and precision.

3.1. Dataset

Initially, the dataset underwent a partitioning process, meticulously dividing it into distinct training and testing sets to ensure an equitable distribution of images across various classes. This step was crucial in mitigating potential biases and ensuring the representativeness of the data. Moreover, to address potential class imbalances and enhance the efficacy of model training, a carefully curated subset consisting of 4000 images per class was meticulously selected. This deliberate curation aimed to provide the model with a diverse array of examples, thereby fortifying its ability to generalize proficiently across a wide spectrum of scenarios. By preparing the dataset in this manner, we aimed to ensure that our model could effectively learn and generalize from a comprehensive and representative set of histopathological images. This approach not only improved the model’s performance but also bolstered its robustness and reliability in real-world applications of prostate cancer diagnosis and Gleason grading.

3.2. Training Phase

The selection of ResNet models ResNet-18, ResNet-34, and ResNet-50 was based on their established effectiveness in capturing intricate features relevant to various Gleason grades and tissue types across various levels of magnification [21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38]. The decision to employ these architectures was based on their proven capabilities to handle the complexities of histopathological images effectively. We utilized transfer learning to enhance the accuracy and robustness of our prostate cancer grading models. By leveraging the pre-existing learned features from the initial training on general image datasets, the models could better capture intricate patterns and details specific to histopathological images of prostate cancer [38,39,40]. The fine-tuning process involved freezing the initial layers of the ResNet models to retain the general features learned and only training the later layers to adapt to the specific characteristics of the dataset. This approach improved the convergence speed and enhanced the models’ overall accuracy by focusing on the relevant medical imaging features unique to prostate cancer pathology. The training phase spanned 100 epochs and employed pre-trained ResNet models, including ResNet18, ResNet-34, and ResNet-50 architectures. Each ResNet model underwent extensive training on the curated dataset, with the objective of fine-tuning their parameters to accurately classify tissue patches into one of the nine distinct classes, including scan background; tissue background; normal, healthy tissue; acquisition artifact; and the five Gleason grades. During the training phase, a 5-fold cross-validation approach was used to enhance the model’s robustness [40]. This involved partitioning the dataset into five subsets, with each subset used as a validation set once while the remaining four subsets were used for training. This process was repeated five times, with each subset used exactly once as the validation set. Implementing 5-fold cross-validation helped to prevent overfitting by systematically dividing the dataset into multiple subsets for training and validation, ensuring that the model’s performance was robust and reliable on unseen data. It enhanced the model’s ability to generalize well to new, unseen data by evaluating its performance across diverse scenarios. Additionally, cross-validation mitigates biases or inconsistencies in the dataset, leading to a more stable and resilient model [41,42,43,44]. It also facilitates the optimization of model hyperparameters and provides more reliable evaluation metrics, instilling confidence in the model’s predictions. By iteratively training and validating the model on different subsets of the data, the 5-fold cross-validation technique helped ensure that the model’s performance was not overly dependent on any subset of the data, thus improving its generalization ability [45,46,47,48].

3.3. Testing Phase

For the testing phase, a stratified split function was employed to maintain class balance in the testing dataset; 7200 images were used. This strategic approach ensured a thorough evaluation of the model’s performance across diverse classes and magnification levels. Stratification was imperative to prevent any bias in the evaluation process and to guarantee that the model’s performance was assessed comprehensively across all classes [49]. The incorporation of pre-trained ResNet models, coupled with meticulous data preparation and testing strategies, underscores the resilience and adaptability of the proposed methodology [50,51,52]. This comprehensive approach contributes to the accuracy of prostate cancer diagnosis and Gleason grading, establishing a robust foundation for reliable histopathological analysis across different magnification contexts. In the next stage of model development, we integrated the DeepLabv3 segmentation model to augment the feature extraction process, thus enhancing the predictive capabilities of our classification models [53]. This integration is designed to strengthen the model’s reliability and accuracy. The process involves feeding input images to DeepLabv3, which is optimized to efficiently extract relevant features. The output produced by the segmentation model, which encapsulates detailed spatial and semantic information, is directed to a convolutional neural network (CNN) classifier. This CNN classifier utilizes the extracted features to generate predictions, leveraging the understanding provided by the DeepLabv3 segmentation model. Through this comprehensive approach, we aim to improve the accuracy and robustness of our predictive model, advancing its performance in classification tasks. The initialization of both the ResNet and DeepLabv3 models serves as the foundational step in this process. Parameter freezing is implemented to safeguard the integrity of both models during training, except for the final fully connected layer of ResNet models, which undergoes modification to accommodate the classification requirements of the dataset. By preserving the feature extraction capabilities of DeepLabv3 and subsequently fine-tuning the ResNet models using these extracted features, the combined model aims to harness the collective strengths of both models to achieve enhanced performance and generalization on the target classification task. This approach capitalizes on the transfer learning potential of ResNet and the feature-rich representations generated by DeepLabv3, thereby facilitating more accurate and robust predictions, particularly in scenarios with limited labeled data availability. The DeepLabv3 model is specifically used to improve the feature extraction process by capturing detailed spatial and semantic information from the histopathological images. This information is crucial for distinguishing between different tissue types and Gleason grades. This innovative approach offers several distinct advantages. Firstly, by leveraging the feature extraction capabilities of DeepLabv3, the model aptly captures the intricate and nuanced features inherent in the input images. This facilitates a richer representation of the data, enabling the CNN classifier to make more precise and accurate predictions [54]. Furthermore, the integration of a segmentation model as a feature extractor allows our model to discern spatial relationships and semantic context within the images. Additionally, the segmentation model helps in isolating the relevant regions of interest within the images, reducing noise and irrelevant data that could potentially hinder the classification performance [55].

4. Results

In this study, we conducted a comprehensive evaluation of our proposed model’s performance across various magnification levels (40×, 20×, 10×, and 5×) in the context of prostate cancer diagnosis and Gleason grading. Leveraging ResNet18, ResNet34, and ResNet50 architectures, our model highlighted remarkable adaptability and accuracy throughout the investigation. For ResNet18, testing accuracies ranged from 0.9956 at 10× magnification to 0.9992 at 20× magnification, indicating a high level of reliability in predictions across different image resolutions. The model’s performance was slightly lower at the 10× magnification, suggesting potential challenges in capturing finer details at this scale. This could be attributed to the relatively simpler architecture of ResNet18, which may not be as effective in identifying complex tissue patterns at lower magnifications. ResNet34 exhibited even higher testing accuracies, consistently achieving 0.9999 at multiple magnifications (40× and 20×) and only slightly lower accuracies at other magnifications (0.9957 at 10× and 0.9993 at 5×), showcasing its robust ability to handle a variety of inputs with exceptional accuracy. The higher number of layers in ResNet34 allows for better feature extraction, especially at higher magnifications where detailed tissue structures are more prominent. ResNet50 also performed impressively, achieving testing accuracies between 0.9915 at 10× magnification and 0.9981 at 5×, demonstrating its competence in maintaining high levels of performance across the varying levels of image magnifications. The deeper layers in ResNet50 likely contribute to its ability to generalize well across different resolutions, but the slight dip at 10× suggests that the added complexity may sometimes lead to overfitting on finer details. The graphs in Figure 2 show training accuracy, validation accuracy in training, and validation loss for Resnet34 model on the 20× magnification image set.
The following graphs show training loss for each fold, accuracies, and validation loss for the Resnet34 model on a 20×-magnification image set when 5-fold cross-validation is used. In the graph of training accuracy, the third and fourth folds show higher accuracies and extremely low losses from the initial epochs. A possible explanation is that the images in those folds are similar to the ones used while training the model. By evaluating the model’s performance under both scenarios, we were able to quantify the tangible benefits of employing the 5-fold cross-validation technique. The graphical representations above in Figure 2 and Figure 3 offer a clear understanding of the model’s learning trajectory and its performance dynamics during training and validation stages. The Figures S1–S11 in the Supplementary Materials show learning losses and accuracies for all Resnet architectures across all magnifications.
Table 2 shows the model’s robustness and generalization ability, highlighting improvements achieved through the validation process. Figure 3 depicts the training loss for each fold over a group of epochs (one set represents 15 epochs), along with accuracies and validation losses for the ResNet34 model on 20× magnification. The losses for each fold are different because despite having a balanced dataset, the stratification process during the creation of folds can still result in slight variations in class distribution within each fold, impacting the loss differently. Certain folds might contain more challenging samples, which can significantly affect the loss curves for those specific folds.
The performance differences across magnification levels and ResNet models correlate with the complexity of tissue patterns in the dataset. At higher magnifications (40× and 20×), detailed tissue structures are more pronounced, allowing deeper models like ResNet34 and ResNet50 to leverage their complex architectures for better feature extraction.
These findings collectively emphasize the adaptability, reliability, and versatility of ResNet18, ResNet34, and ResNet50 in prostate cancer diagnosis and Gleason grading across different magnification levels. The ability of these models to maintain high accuracy rates across varying resolutions is essential for ensuring accurate and consistent diagnostic assessments, thereby contributing to improved patient outcomes and clinical decision-making in the field of prostate cancer pathology.

5. Discussion

This study demonstrates the effectiveness of using ResNet models like ResNet18, ResNet34, and ResNet50 in diagnosing prostate cancer and grading Gleason scores. These models perform admirably across various magnification levels, indicating their proficiency in detecting crucial details in histopathological images. However, there is potential for further exploration into other algorithms. Exploring techniques like Vision Transformers could potentially improve the models’ ability to discern subtle variations, leading to enhanced diagnostic accuracy. Moreover, the success of the proposed approach in prostate cancer diagnosis suggests its potential applicability to other malignancies. Expanding the methodology to include datasets and histopathological images from different cancer types could facilitate more accurate diagnosis and grading across various cancers. This multi-cancer approach could contribute to a broader understanding of histopathological features and aid in the development of comprehensive diagnostic tools applicable across diverse oncological contexts. Moving forward, an intriguing prospect involves transforming these developed models into an accessible web application. This would democratize access to advanced prostate cancer diagnostic tools, enabling healthcare professionals to integrate machine learning into their clinical practice more seamlessly. The application could offer features such as the real-time analysis of histopathological images, automated Gleason grading, and seamless integration with existing electronic medical record systems for streamlined patient management. An important consideration for future work includes the potential expansion of modalities in electronic health record (EHR) data to further enhance the model’s capabilities. Integrating additional types of EHR data, such as radiological imaging, genetic information, and laboratory test results, could provide a more holistic approach to diagnosing and grading prostate cancer. This comprehensive utilization of patient data can improve model accuracy, leading to better patient outcomes and personalized treatment plans.

6. Conclusions

The proposed approach, framed as a classification problem and utilizing ResNet models in conjunction with the Diagset dataset, highlights its potential as a robust tool for prostate cancer diagnosis and Gleason grading. The integration of machine learning and deep learning techniques represents a pivotal advancement in prostate cancer diagnosis and grading. The proposed methodology, centered around convolutional neural networks operating on histopathological scans at multiple magnification levels, presents a comprehensive and robust approach to prostate cancer diagnosis and Gleason grading. Leveraging the rich resource provided by the DiagSet dataset, this research offers insights into the intricate features underlying prostate cancer pathology across different Gleason grades and tissue types. The comprehensive evaluation of the proposed model’s performance across various magnification levels reaffirms its adaptability and reliability in accurately assessing prostate cancer pathology. The consistent high accuracies achieved by the ResNet models further validate their stability and efficacy, positioning them as valuable assets in clinical practice for prostate cancer diagnosis and Gleason grading. Moving forward, an intriguing prospect involves transforming these models into an accessible web application, which would democratize access to advanced diagnostic tools. This application would allow healthcare professionals to seamlessly integrate machine learning into clinical practice, offering features such as the real-time analysis of histopathological images and automated Gleason grading. Additionally, the application would facilitate integration with electronic medical record systems, enabling streamlined patient management and improved clinical workflows. By providing an intuitive interface, the web application would ensure that sophisticated machine learning tools are accessible to clinicians without requiring extensive technical expertise, thereby enhancing diagnostic accuracy and efficiency in diverse healthcare settings.
This study establishes a solid foundation for advancing prostate cancer diagnosis and Gleason grading through machine learning. Future research should explore additional algorithms, such as Vision Transformers, and improve model efficiency to further enhance accuracy, scalability, and clinical utility. By addressing these areas, future work can expand the capabilities and applications of these models, ultimately benefiting healthcare providers and patients alike.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/bioengineering11060624/s1. Figure S1: Training loss (a) and training accuracy (b) for Resnet34 on 5× magnification; Figure S2: Training loss (a) and training accuracy (b) for Resnet34 on 10× magnification; Figure S3: Training loss (a) and training accuracy (b) for Resnet34 on 40× magnification; Figure S4: Training loss (a) and training accuracy (b) for Resnet18 on 5× magnification; Figure S5: Training loss (a) and training accuracy (b) for Resnet18 on 10× magnification; Figure S6: Training loss (a) and training accuracy (b) for Resnet18 on 20× magnification; Figure S7: Training loss (a) and training accuracy (b) for Resnet18 on 40× magnification; Figure S8: Training loss (a) and training accuracy (b) for Resnet50 on 5× magnification; Figure S9: Training loss (a) and training accuracy (b) for Resnet50 on 10× magnification; Figure S10: Training loss (a) and training accuracy (b) for Resnet50 on 20× magnification; Figure S11: Training loss (a) and training accuracy (b) for Resnet50 on 40× magnification.

Author Contributions

Conceptualization, S.A. and S.M.A.A.-H.; Methodology, S.A., T.K. and S.M.A.A.-H.; Software S.A., T.K. and S.M.A.A.-H.; Validation, S.A., T.K., S.M.A.A.-H., A.B., B.A., R.C. and S.T.R.; Formal analysis, S.A.; Investigation, S.A., S.M.A.A.-H., A.B., B.A., R.C. and S.T.R.; Writing—original draft, S.A and T.K.; Writing—review & editing, S.A. and S.M.A.A.-H.; Visualization, S.A.; Supervision, S.A. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The used datasets are publicly available as specified in their references.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Rice-Stitt, T.; Valencia-Guerrero, A.; Cornejo, K.M.; Wu, C.L. Updates in Histologic Grading of Urologic Neoplasms. Arch. Pathol. Lab. Med. 2020, 144, 335–343. [Google Scholar] [CrossRef] [PubMed]
  2. Goldenberg, S.L.; Nir, G.; Salcudean, S.E. A new era: Artificial intelligence and machine learning in prostate cancer. Nat. Rev. Urol. 2019, 16, 391–403. [Google Scholar] [CrossRef] [PubMed]
  3. Abraham, B.; Nair, M.S. Automated grading of prostate cancer using convolutional neural network and ordinal class classifier. Inform. Med. Unlocked 2019, 17, 100256. [Google Scholar] [CrossRef]
  4. Ström, P.; Kartasalo, K.; Olsson, H.; Solorzano, L.; Delahunt, B.; Berney, D.M.; Bostwick, D.G.; Evans, A.J.; Grignon, D.J.; Humphrey, P.A.; et al. Artificial intelligence for diagnosis and grading of prostate cancer in biopsies: A population-based, diagnostic study. Lancet Oncol. 2020, 21, 222–232. [Google Scholar] [CrossRef] [PubMed]
  5. Bhattacharjee, S.; Park, H.-G.; Kim, C.-H.; Prakash, D.; Madusanka, N.; So, J.-H.; Cho, N.-H.; Choi, H.-K. Quantitative Analysis of Benign Malignant Tumors in Histopathology: Predicting Prostate Cancer Grading Using SVM. Appl. Sci. 2019, 9, 2969. [Google Scholar] [CrossRef]
  6. Alzubaidi, L.; Zhang, J.; Humaidi, A.J.; Al-Dujaili, A.; Duan, Y.; Al-Shamma, O.; Santamaría, J.; Fadhel, M.A.; Al-Amidie, M.; Farhan, L. Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions. J. Big Data 2021, 8, 53. [Google Scholar] [CrossRef] [PubMed]
  7. Hammouda, K.; Khalifa, F.; Alghamdi, N.S.; Darwish, H.; El-Baz, A. Multi-Stage Classification-Based Deep Learning for Gleason System Grading Using Histopathological Images. Cancers 2022, 14, 5897. [Google Scholar] [CrossRef]
  8. Wang, R.; Yang, S.; Wang, M.; Zhou, Y.; Li, X.; Chen, W.; Liu, W.; Huang, Y.; Wu, J.; Cao, J.; et al. A sustainable approach to universal metabolic cancer diagnosis. Nat. Sustain. 2024, 7, 602–615. [Google Scholar] [CrossRef]
  9. Liu, B.; Wang, Y.; Weitz, P.; Lindberg, J.; Hartman, J.; Wang, W.; Egevad, L.; Grönberg, H.; Eklund, M.; Rantalainen, M. Using deep learning to detect patients at risk for prostate cancer despite benign biopsies. iScience 2022, 25, 104663. [Google Scholar] [CrossRef]
  10. Wang, J.; Yang, Y.; Mao, J.; Huang, Z.; Huang, C.; Xu, W. Cnn-rnn: A unified framework for multi-label image classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2016, Las Vegas, NV, USA, 27–30 June 2016; pp. 2285–2294. [Google Scholar]
  11. Kumar, M.D.; Babaie, M.; Zhu, S.; Kalra, S.; Tizhoosh, H.R. A comparative study of CNN, BoVW and LBP for classification of histopathological images. In Proceedings of the 2017 IEEE Symposium Series on Computational Intelligence (SSCI), Honolulu, HI, USA, 27 November–1 December 2017; pp. 1–7. [Google Scholar]
  12. LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 521, 436–444. [Google Scholar] [CrossRef] [PubMed]
  13. Pati, P.; Jaume, G.; Ayadi, Z.; Thandiackal, K.; Bozorgtabar, B.; Gabrani, M.; Goksel, O. Weakly supervised joint whole-slide segmentation and classification in prostate cancer. Med. Image Anal. 2023, 89, 102915. [Google Scholar] [CrossRef] [PubMed]
  14. Fakoor, R.; Ladhak, F.; Nazi, A.; Huber, M. Using deep learning to enhance cancer diagnosis and classification. In Proceedings of the International Conference on Machine Learning 2013, Atlanta, GA, USA, 16–21 June 2013; Volume 28, pp. 3937–3949. [Google Scholar]
  15. Xu, W.; Fu, Y.L.; Zhu, D. ResNet and its application to medical image processing: Research progress and challenges. Comput. Methods Programs Biomed. 2023, 240, 107660. [Google Scholar] [CrossRef] [PubMed]
  16. Deng, J.; Dong, W.; Socher, R.; Li, L.J.; Li, K.; Li, F.-F. Imagenet: A large-scale hierarchical image database. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20–25 June 2009; pp. 248–255. [Google Scholar]
  17. Samaratunga, H.; Delahunt, B.; Yaxley, J.; Srigley, J.R.; Egevad, L. From Gleason to International Society of Urological Pathology (ISUP) grading of prostate cancer. Scand. J. Urol. 2016, 50, 325–329. [Google Scholar] [CrossRef] [PubMed]
  18. Nir, G.; Hor, S.; Karimi, D.; Fazli, L.; Skinnider, B.F.; Tavassoli, P.; Turbin, D.; Villamil, C.F.; Wang, G.; Wilson, R.S.; et al. Automatic grading of prostate cancer in digitized histopathology images: Learning from multiple experts. Med. Image Anal. 2018, 50, 167–180. [Google Scholar] [CrossRef] [PubMed]
  19. Gleason, D.F. Classification of prostatic carcinomas. Cancer Chemother. Rep. 1966, 50, 125–128. [Google Scholar] [PubMed]
  20. Nagpal, K.; Foote, D.; Liu, Y.; Chen, P.-H.C.; Wulczyn, E.; Tan, F.; Olson, N.; Smith, J.L.; Mohtashamian, A.; Wren, J.H.; et al. Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer. NPJ Digit. Med. 2019, 2, 48. [Google Scholar] [CrossRef] [PubMed]
  21. Almoosawi, N.M.; Khudeyer, R.S. ResNet-34/DR: A residual convolutional neural network for the diagnosis of diabetic retinopathy. Informatica 2021, 45. [Google Scholar] [CrossRef]
  22. Guo, M.; Du, Y. Classification of thyroid ultrasound standard plane images using ResNet-18 networks. In Proceedings of the 2019 IEEE 13th International Conference on Anti-Counterfeiting, Security, and Identification (ASID), Xiamen, China, 25–27 October 2019; pp. 324–328. [Google Scholar]
  23. Yurtkulu, S.C.; Şahin, Y.H.; Unal, G. Semantic segmentation with extended DeepLabv3 architecture. In Proceedings of the 2019 27th Signal Processing and Communications Applications Conference (SIU), Sivas, Turkey, 24–26 April 2019; pp. 1–4. [Google Scholar]
  24. Heryadi, Y.; Irwansyah, E.; Miranda, E.; Soeparno, H.; Herlawati; Hashimoto, K. The effect of ResNet model as feature extractor network to performance of DeepLabV3 model for semantic satellite image segmentation. In Proceedings of the 2020 IEEE Asia-Pacific Conference on Geoscience, Electronics and Remote Sensing Technology (AGERS), Jakarta, Indonesia, 7–8 December 2020; pp. 74–77.
  25. Kayalibay, B.; Jensen, G.; van der Smagt, P. CNN-based segmentation of medical imaging data. arXiv 2017, arXiv:1701.03056. [Google Scholar]
  26. Mortazi, A.; Bagci, U. Automatically designing CNN architectures for medical image segmentation. In Proceedings of the Machine Learning in Medical Imaging: 9th International Workshop, MLMI 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, 16 September 2018; Proceedings 9. Springer International Publishing: Berlin/Heidelberg, Germany, 2018; pp. 98–106. [Google Scholar]
  27. Liu, F.; Lin, G.; Shen, C. CRF learning with CNN features for image segmentation. Pattern Recognit. 2015, 48, 2983–2992. [Google Scholar] [CrossRef]
  28. Sharma, P.; Berwal, Y.P.S.; Ghai, W. Performance analysis of deep learning CNN models for disease detection in plants using image segmentation. Inf. Process. Agric. 2020, 7, 566–574. [Google Scholar] [CrossRef]
  29. Dolz, J.; Gopinath, K.; Yuan, J.; Lombaert, H.; Desrosiers, C.; Ben Ayed, I. HyperDense-Net: A hyper-densely connected CNN for multi-modal image segmentation. IEEE Trans. Med. Imaging 2019, 38, 1116–1126. [Google Scholar] [CrossRef] [PubMed]
  30. Milosevic, M.; Jin, Q.; Singh, A.; Amal, S. Applications of AI in multi-modal imaging for cardiovascular disease. Front. Radiol. 2024, 3, 1294068. [Google Scholar] [CrossRef] [PubMed]
  31. Wang, X.; Deng, X.; Fu, Q.; Zhou, Q.; Feng, J.; Ma, H.; Liu, W.; Zheng, C. Weakly supervised framework for detecting lesions in medical images. In Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019; pp. 3514–3522. [Google Scholar]
  32. Hamida, A.B.; Devanne, M.; Weber, J.; Truntzer, C.; Derangère, V.; Ghiringhelli, F.; Forestier, G.; Wemmert, C. Deep learning for colon cancer histopathological images analysis. Comput. Biol. Med. 2021, 136, 104730. [Google Scholar] [CrossRef] [PubMed]
  33. Zerouaoui, H.; Idri, A. Deep hybrid architectures for binary classification of medical breast cancer images. Biomed. Signal Process. Control. 2022, 71, 103226. [Google Scholar]
  34. Cao, R.; Zhong, X.; Shakeri, S.; Bajgiran, A.M.; Mirak, S.A.; Enzmann, D.; Raman, S.S.; Sung, K. Prostate cancer detection and segmentation in multi-parametric MRI via CNN and conditional random field. In Proceedings of the 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), Venice, Italy, 8–11 April 2019; pp. 1900–1904. [Google Scholar]
  35. Soni, M.; Khan, I.R.; Babu, K.S.; Nasrullah, S.; Madduri, A.; Rahin, S.A. Light weighted healthcare CNN model to detect prostate cancer on multiparametric, M.R.I. Comput. Intell. Neurosci. 2022, 2022, 5497120. [Google Scholar] [PubMed]
  36. Tolkach, Y.; Dohmgörgen, T.; Toma, M.; Kristiansen, G. High-accuracy prostate cancer pathology using deep learning. Nat. Mach. Intell. 2020, 2, 411–418. [Google Scholar] [CrossRef]
  37. Abbasi, A.A.; Hussain, L.; Awan, I.A.; Abbasi, I.; Majid, A.; Nadeem, M.S.A.; Chaudhary, Q.-A. Detecting prostate cancer using deep learning convolution neural network with transfer learning approach. Cogn. Neurodyn. 2020, 14, 523–533. [Google Scholar] [CrossRef] [PubMed]
  38. De Vente, C.; Vos, P.; Hosseinzadeh, M.; Pluim, J.; Veta, M. Deep learning regression for prostate cancer detection and grading in bi-parametric MRI. IEEE Trans. Biomed. Eng. 2020, 68, 374–383. [Google Scholar] [CrossRef] [PubMed]
  39. Anguita, D.; Ghelardoni, L.; Ghio, A.; Oneto, L.; Ridella, S. The ‘K’ in K-fold Cross Validation. In Proceedings of the ESANN 2012, Bruges, Belgium, 25–27 April 2012; Volume 102, pp. 441–446. [Google Scholar]
  40. Li, Z.; Liu, F.; Yang, W.; Peng, S.; Zhou, J. A survey of convolutional neural networks: Analysis, applications, and prospects. IEEE Trans. Neural Netw. Learn. Syst. 2021, 33, 6999–7019. [Google Scholar] [CrossRef] [PubMed]
  41. Albawi, S.; Mohammed, T.A.; Al-Zawi, S. Understanding of a convolutional neural network. In Proceedings of the 2017 International Conference on Engineering and Technology (ICET), Antalya, Turkey, 21–23 August 2017; pp. 1–6. [Google Scholar] [CrossRef]
  42. Shah, R.B. Current perspectives on the Gleason grading of prostate cancer. Arch. Pathol. Lab. Med. 2009, 133, 1810–1816. [Google Scholar] [CrossRef] [PubMed]
  43. Tabesh, A.; Teverovskiy, M.; Pang, H.-Y.; Kumar, V.P.; Verbel, D.; Kotsianti, A.; Saidi, O. Multifeature prostate cancer diagnosis and Gleason grading of histological images. IEEE Trans. Med. Imaging 2007, 26, 1366–1378. [Google Scholar] [CrossRef] [PubMed]
  44. Kott, O.; Linsley, D.; Amin, A.; Karagounis, A.; Jeffers, C.; Golijanin, D.; Serre, T.; Gershman, B. Development of a deep learning algorithm for the histopathologic diagnosis and Gleason grading of prostate cancer biopsies: A pilot study. Eur. Urol. Focus 2021, 7, 347–351. [Google Scholar] [CrossRef] [PubMed]
  45. Sarwinda, D.; Paradisa, R.H.; Bustamam, A.; Anggia, P. Deep learning in image classification using residual network (ResNet) variants for detection of colorectal cancer. Procedia Comput. Sci. 2021, 179, 423–431. [Google Scholar] [CrossRef]
  46. Hou, L.; Samaras, D.; Kurc, T.M.; Gao, Y.; Davis, J.E.; Saltz, J.H. Patch-based convolutional neural network for whole slide tissue image classification. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 2424–2433. [Google Scholar]
  47. Koziarski, M.; Woźniak, M.; Krawczyk, B. Combined cleaning and resampling algorithm for multi-class imbalanced data with label noise. Knowl.-Based Syst. 2020, 204, 106223. [Google Scholar] [CrossRef]
  48. Litjens, G.; Sánchez, C.I.; Timofeeva, N.; Hermsen, M.; Nagtegaal, I.; Kovacs, I.; Hulsbergen-van de Kaa, C.; Bult, P.; Van Ginneken, B.; Van Der Laak, J. Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Sci. Rep. 2016, 6, 26286. [Google Scholar] [CrossRef] [PubMed]
  49. Srinidhi, C.L.; Ciga, O.; Martel, A.L. Deep neural network models for computational histopathology: A survey. Med. Image Anal. 2021, 67, 101813. [Google Scholar] [CrossRef] [PubMed]
  50. Lu, L.; Zheng, Y.; Carneiro, G.; Yang, L. Deep Learning and Convolutional Neural Networks for Medical Image Computing; Advances in Computer Vision and Pattern Recognition Series; Springer: Cham, Switzerland, 2017; Volume 10. [Google Scholar]
  51. Singh, A.; Wan, M.; Harrison, L.; Breggia, A.; Christman, R.; Winslow, R.L.; Amal, S. Visualizing Decisions and Analytics of Artificial Intelligence based Cancer Diagnosis and Grading of Specimen Digitized Biopsy: Case Study for Prostate Cancer. In Proceedings of the Companion 28th International Conference on Intelligent User Interfaces, New York, NY, USA, 27–31 March 2023; pp. 166–170. [Google Scholar] [CrossRef]
  52. Gan, Y.; Li, L.; Zhang, L.; Yan, S.; Gao, C.; Hu, S.; Qiao, Y.; Tang, S.; Wang, C.; Lu, Z. Association between shift work and risk of prostate cancer: A systematic review and meta-analysis of observational studies. Carcinogenesis 2018, 39, 87–97. [Google Scholar] [CrossRef] [PubMed]
  53. Sultana, F.; Sufian, A.; Dutta, P. Evolution of image segmentation using deep convolutional neural network: A survey. Knowl.-Based Syst. 2020, 201, 106062. [Google Scholar] [CrossRef]
  54. Huang, R.; Li, Y.; Wu, H.; Liu, B.; Zhang, X.; Zhang, Z. 68Ga-PSMA-11 PET/CT versus 68Ga-PSMA-11 PET/MRI for the detection of biochemically recurrent prostate cancer: A systematic review and meta-analysis. Front. Oncol. 2023, 13, 1216894. [Google Scholar] [CrossRef] [PubMed]
  55. Yang, C.; Sheng, D.; Yang, B.; Zheng, W.; Liu, C. A Dual-Domain Diffusion Model for Sparse-View CT Reconstruction. IEEE Signal Process. Lett. 2024, 31, 1279–1283. [Google Scholar] [CrossRef]
Bioengineering 11 00624 g001
Figure 1. Samples of classes in dataset.
Figure 1. Samples of classes in dataset.
Bioengineering 11 00624 g001
Bioengineering 11 00624 g002
Figure 2. Graphs for Resnet34 model on images of 20× magnification: (a) training accuracy, (b) training loss.
Figure 2. Graphs for Resnet34 model on images of 20× magnification: (a) training accuracy, (b) training loss.
Bioengineering 11 00624 g002
Bioengineering 11 00624 g003
Figure 3. Cross fold training loss (a) and training accuracies (b) for ResNet34 model on 20× magnification.
Figure 3. Cross fold training loss (a) and training accuracies (b) for ResNet34 model on 20× magnification.
Bioengineering 11 00624 g003
Table 1. Summary of methodologies and datasets used in prostate cancer studies [2,3,4,5].
Table 1. Summary of methodologies and datasets used in prostate cancer studies [2,3,4,5].
PaperJournal/ConferenceAuthor + YearMethodologyDataset
1.A new era: artificial intelligence and machine learning in prostate cancer (Nature Reviews Urology) [2]Goldenberg, S.L.; Nir, G.; Salcudean, S.E. 2019ML and DL techniques for diagnostic imaging, SVM, CNN-based DL networkPROSTATE-x challenge, mpMRI images
2.Automated grading of prostate cancer using CNN and ordinal class classifier (Informatics in Medicine Unlocked) [3]Abraham, B.; Nair, M.S. 2019VGG-16 CNN, Ordinal Class Classifier with J48
Achieved a moderate quadratic weighted kappa score of 0.4727 in grading PCA into 5 grade groups. Positive predictive value of 0.9079 in predicting clinically significant prostate cancer.		PROSTATEx-2 2017 grand challenge dataset
3.AI for diagnosis and grading of prostate cancer in biopsies (The Lancet Oncology) [4]Ström, P.; Kartasalo, K.; Olsson, H.; Solorzano, L.; Delahunt, B.; Berney, D.M.; Bostwick, D.G.; Evans, A.J.; Grignon, D.J.; Humphrey, P.A.; et al. 2020Deep neural networks for biopsy assessment.
AI system achieved high accuracy in distinguishing benign and malignant biopsy cores (AUC of 0.997 and 0.986 on respective datasets). 		STHLM3 diagnostic study, external validation dataset
4.Quantitative Analysis of Benign and Malignant Tumors in Histopathology: Predicting Prostate Cancer Grading Using SVM (Applied Sciences) [5]Bhattacharjee, S.; Park, H.-G.; Kim, C.-H.; Prakash, D.; Madusanka, N.; So, J.-H.; Cho, N.-H.; Choi, H.-K. 2019SVM classification, Image manipulation, K-means, Watershed algorithms. Accuracy of 88.7% for malignant vs. benign, 85.0% for Grade 3 vs. Grade 4, 5, and 92.5% for Grade 4 vs. Grade 5.Biopsy-derived images, Gleason grade groups (Grade 3, Grade 4, Grade 5, and benign)
Table 2. Models’ performance on 5-fold cross-validation.
Table 2. Models’ performance on 5-fold cross-validation.
MagnificationArchitectureAverage Training AccuracyTesting Accuracy
40×ResNet180.99950.9977
20×0.99960.9992
10×0.99930.9964
5×0.99950.9921
40×ResNet340.99920.9999
20×0.99930.9999
10×0.99981.0000
5×0.99930.9993
40×ResNet500.99930.9957
20×0.99910.9915
10×0.99560.9952
5×0.98930.9981
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Kondejkar, T.; Al-Heejawi, S.M.A.; Breggia, A.; Ahmad, B.; Christman, R.; Ryan, S.T.; Amal, S. Multi-Scale Digital Pathology Patch-Level Prostate Cancer Grading Using Deep Learning: Use Case Evaluation of DiagSet Dataset. Bioengineering 2024, 11, 624. https://doi.org/10.3390/bioengineering11060624

AMA Style

Kondejkar T, Al-Heejawi SMA, Breggia A, Ahmad B, Christman R, Ryan ST, Amal S. Multi-Scale Digital Pathology Patch-Level Prostate Cancer Grading Using Deep Learning: Use Case Evaluation of DiagSet Dataset. Bioengineering. 2024; 11(6):624. https://doi.org/10.3390/bioengineering11060624

Chicago/Turabian Style

Kondejkar, Tanaya, Salah Mohammed Awad Al-Heejawi, Anne Breggia, Bilal Ahmad, Robert Christman, Stephen T. Ryan, and Saeed Amal. 2024. "Multi-Scale Digital Pathology Patch-Level Prostate Cancer Grading Using Deep Learning: Use Case Evaluation of DiagSet Dataset" Bioengineering 11, no. 6: 624. https://doi.org/10.3390/bioengineering11060624

APA Style

Kondejkar, T., Al-Heejawi, S. M. A., Breggia, A., Ahmad, B., Christman, R., Ryan, S. T., & Amal, S. (2024). Multi-Scale Digital Pathology Patch-Level Prostate Cancer Grading Using Deep Learning: Use Case Evaluation of DiagSet Dataset. Bioengineering, 11(6), 624. https://doi.org/10.3390/bioengineering11060624

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop