Next Article in Journal
Post-Operative Complications and Risk Predictors Related to the Avulsion of Lower Impacted Third Molars
Previous Article in Journal
MRONJ of the Mandible—From Decortication to a Complex Jaw Reconstruction Using a CAD/CAM-Guided Bilateral Scapula Flap
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Medicina
Volume 59
Issue 3
10.3390/medicina59030536
medicina-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Deep Learning Prediction of TERT Promoter Mutation Status in Thyroid Cancer Using Histologic Images

by
Jinhee Kim
1,†,
Seokhwan Ko
2,3,†,
Moonsik Kim
1,
Nora Jee-Young Park
1,
Hyungsoo Han
2,4,
Junghwan Cho
2,* and
Ji Young Park
1,*
1
Department of Pathology, Kyungpook National University School of Medicine, Kyungpook National University Chilgok Hospital, Daegu 41404, Republic of Korea
2
Clinical Omics Institute, Kyungpook National University, Daegu 41405, Republic of Korea
3
Department of Biomedical Science, School of Medicine, Kyungpook National University, Daegu 41944, Republic of Korea
4
Department of Physiology, School of Medicine, Kyungpook National University, Daegu 41944, Republic of Korea
*
Authors to whom correspondence should be addressed.
†
These authors have contributed equally to this work.
Medicina 2023, 59(3), 536; https://doi.org/10.3390/medicina59030536
Submission received: 18 December 2022 / Revised: 2 February 2023 / Accepted: 6 March 2023 / Published: 9 March 2023
(This article belongs to the Section Oncology)
Browse Figures
Review Reports Versions Notes

Abstract

:
Background and objectives: Telomerase reverse transcriptase (TERT) promoter mutation, found in a subset of patients with thyroid cancer, is strongly associated with aggressive biologic behavior. Predicting TERT promoter mutation is thus necessary for the prognostic stratification of thyroid cancer patients. Materials and Methods: In this study, we evaluate TERT promoter mutation status in thyroid cancer through the deep learning approach using histologic images. Our analysis included 13 consecutive surgically resected thyroid cancers with TERT promoter mutations (either C228T or C250T) and 12 randomly selected surgically resected thyroid cancers with a wild-type TERT promoter. Our deep learning model was created using a two-step cascade approach. First, tumor areas were identified using convolutional neural networks (CNNs), and then TERT promoter mutations within tumor areas were predicted using the CNN–recurrent neural network (CRNN) model. Results: Using the hue–saturation–value (HSV)-strong color transformation scheme, the overall experiment results show 99.9% sensitivity and 60% specificity (improvements of approximately 25% and 37%, respectively, compared to image normalization as a baseline model) in predicting TERT mutations. Conclusions: Highly sensitive screening for TERT promoter mutations is possible using histologic image analysis based on deep learning. This approach will help improve the classification of thyroid cancer patients according to the biologic behavior of tumors.

1. Introduction

Thyroid cancer is one of the most common malignancies in humans [1]. Although the majority of thyroid cancers show indolent behavior [2], tumor recurrence and distant metastasis can occur [3,4]. The telomerase reverse transcriptase (TERT) gene, located on chromosome 5p15.33, is involved in telomere maintenance and associated with cellular senescence [5]. TERT promoter mutations have been repeatedly found in human cancer, particularly with high frequency in human melanoma and thyroid cancer [5,6]. Furthermore, TERT promoter mutations C228T and C250T have been known to occur quite frequently (mutation hotspots) [7,8]. Notably, TERT promoter mutations in thyroid cancer have been associated with aggressive clinical behavior [9,10,11]. Thus, the detection of TERT promoter mutations is important for prognostic stratification and patient management.
Evidence has shown that digital pathology with artificial intelligence (AI) can have a wide range of applications [12]. In fact, the use of digital pathologic images can improve quantitative analysis of certain histologic features, such as tumor-infiltrating lymphocytes [13]. Furthermore, current studies have been actively investigating methods for predicting the mutation status of genes with diagnostic and therapeutic implications using digital pathologic images [14,15,16,17]. Conventionally, a two-step approach is used for predicting genetic alternations in various cancer types [14,15,18]. First, typical tumor areas are distinguished using tissue slides, and subsequently, another deep neural network is applied to classify mutations at the tile level within tumor areas. Recent advances include attention-based multiple-instance learning performed by aggregating tile features with weight-scoring values learned by a neural network for slide-level prediction [19,20,21].
It is important to consider the color diversity of histopathological images when training AI for better tumor classification. Recent studies have introduced deep neural networks along with color normalization or transformation methods [22,23] as an image preprocessing step that reduces the generalization error.
To the best of our knowledge, this study is the first to evaluate the mutation status of the TERT promoter in thyroid cancer using our deep learning model. On the basis of the general perspective of medical doctors, we designed a two-step cascaded architecture to predict the mutation status of the TERT promoter in thyroid cancer. In the first step, the architecture predicted tumor areas using color transformation methods and convolutional neural networks (CNNs). To subsequently infer the TERT promoter mutation status, the combination of a CNN and recurrent neural network (RNN) model (CRNN) [24,25] was applied in the second step, which focuses on finding cell abnormalities associated with TERT promoter mutation status.

2. Materials and Methods

2.1. Study Population

We retrospectively evaluated 80 consecutive surgically resected thyroid cancer cases from 2016 to 2021 whose samples underwent TERT promoter polymerase chain reaction (PCR) testing and found 13 (16.3%) cases with TERT promoter mutations (either C228T or C250T). TERT promoter mutation status was confirmed via real-time PCR at the Department of Pathology, Kyungpook National University Chilgok Hospital. TERT promoter PCR testing was mainly performed for older patients (>55 years) with large tumors having widely infiltrative growth patterns and thyroid cancers showing aggressive clinical behavior [6,26]. We then randomly selected 12 surgically resected thyroid cancer cases having a wild-type TERT promoter during the same period. Considering the class-imbalance problem in training deep learning [27], we finally selected a number of TERT-negative cases that is similar to that of positive ones. The clinicopathologic data of the patients were retrieved from their medical records. This study was conducted in accordance with the guidelines of the Declaration of Helsinki. The requirement for written informed consent from the patients was waived because of the retrospective nature of the study.

2.2. Histologic Evaluation

Surgical specimens were fixed in 10% neutral-buffered formalin and embedded in paraffin blocks. The paraffin blocks were then cut into 4 μm thick sections and stained with hematoxylin and eosin. Two independent pathologists specializing in thyroid pathology (MSK and JYP) reviewed all available slides, and the representative slides were selected for scanning (Figure 1). Tumors were diagnosed and classified according to the fifth edition of the World Health Organization classification of thyroid neoplasms [28].

2.3. Dataset Preparation

2.3.1. Annotation of Tumor and TERT Positives

Each slide has been annotated according to three types of regions, as shown in Figure 2: normal regions (red contours), tumor regions (yellow contours), and TERT regions of interest (ROIs) within the tumor (bounding boxes). The tumor regions have been accurately delineated, while normal regions and TERT ROI boxes have been marked partially. Figure 2 shows an example of overall annotation tasks on whole-slide image (WSI) data.

2.3.2. Downsampling Ratio

To analyze patches in different WSI scales, the downsampling level was defined on the basis of the generic equation of 1/ 2 l e v e l   v a l u e . Thus, the original size was extracted from the level value 0. Because each step focuses on different features, we applied different level values to both steps of the deep learning model. Patch images scaled to ¼ the size of the original image resolution were used to classify tumor areas; however, patches used for predicting TERT promoter mutations were extracted using the original resolution.

2.4. Whole Architecture for TERT Prediction

We constructed a cascade deep learning model consisting of a tumor classifier and a TERT predictor that inferred mutation status according to the tile-based WSI input. Figure 3a shows how the CNN model recognizes tumor areas at low magnification levels with tiling patches. The predicted patches are rescaled to the original level to examine cytologic atypia at high resolutions and are delivered to the CRNN model for the prediction of TERT mutation (negative or positive) as shown in Figure 3b.

2.5. Data Split

We used 25 WSIs (13 for TERT positive, 12 for TERT negative) with the given dataset being split into 5 cross-validation sets at the slide level. To evenly split the TERT-positive and TERT-negative cases in each fold, each positive and negative slide was first separated, after which five-fold cross-validation was performed. Table 1a shows the number of patches for tumor classification. Because each WSI contains various tumor area distributions, each set has a different amount of data. Table 1b shows the distribution of TERT ROI bounding boxes made according to TERT-negative and TERT-positive slides in the second step.

2.6. Classification of Tumor Areas Using CNN

2.6.1. Patch Filtering

Given the enormous size of WSI data, each WSI was tiled using a patch size of 256 × 256 at level value 2 (i.e., downsized to ¼ of the original). Some patches include unnecessary components, such as void backgrounds observed as being white. To prevent the use of white background patches, we filtered out void patches on the basis of the grayscale pixel criteria. Each patch image was converted to 8-bit grayscale, and a binary image was generated by setting the valid pixel value threshold to <230 in order to identify the background areas in the patch image. After each patch was inspected using the value at the pixel level, only patches that had a background pixel rate exceeding 40% were excluded from the training dataset.

2.6.2. Color Transformation as Image Preprocessing

To account for the color diversity of the pathological images, such as those acquired from different scanners or using various staining conditions, color transformations, including hue–saturation–value (HSV) and hematoxylin–eosin–DAB (HED) methods, were applied for a better classification performance [22]. Color-augmentation strategies using HSV/HED-light and HSV/HED-strong have been investigated for tumor classification in Figure 4
α = R a n d o m   C h o i c e [ u n i f o r m   d i s t r i b u t i o n ( 1 − θ ,   1 + θ ) ]
β = R a n d o m   C h o i c e [ u n i f o r m   d i s t r i b u t i o n ( − θ ,   θ ) ]
i m a g e ′ = α ∗ i m a g e + β
where α is the slope, β is the intercept parameter, and i m a g e ′ is the color-transformed image. For HED-light and HED-strong, θ parameters of 0.05 and 0.2 were applied, respectively. Moreover, a hue value of 0.1 and saturation value of 1.0 were applied for HSV-light and HSV-strong, respectively. The θ , hue, and saturation parameters manipulate how much to jitter the HED or HSV color space. Color normalization using the mean and standard deviation values for the whole-image data was performed in order to apply pre-trained weights from the ImageNet dataset and determine optimal preprocessing methods.

2.6.3. CNN Model Training

For tumor classification, we applied three state-of-the-art CNN models: DenseNet161 [29], VGG16 [30], and EfficientNet_b4 [31]. Figure 5 shows an overview of the CNN training model architecture for tumor area prediction. Each CNN model was implemented using a Pytorch deep learning framework and used a pre-trained model generated on the ImageNet dataset. To address the class-imbalance problem, class weighting, which is the ratio of the number of samples in each class to the total training samples, was applied to the cross-entropy loss function. A total of 3 CNN training models were created on NVIDIA RTX A6000 GPUs, with data being loaded at a batch size of 64.
Because most training performances were saturated or overfitted after the 30th epoch, tasks were forcibly stopped early at that point. In the experiments, the initial learning rate and weight decay were determined to be 5.0e-5 and 1.0e-4, respectively, using an ADAM optimizer to perform a parameter sweep that would derive the best-performing architecture. The best model was screened on the basis of validation accuracy.
Each validation set was evaluated on the best-performing models having the highest accuracy scores. As the trained models were generated using 5 different color transformation methods (i.e., normal, HED-light, HED-strong, HSV-light, and HSV-strong) at each cross-validation set, the experiment had a total of 25 training operations for each CNN model.

2.7. Prediction of TERT Promoter Mutation Status Using CRNN

After classifying tumor areas at the first step, the second step predicted whether the patches were positive or negative for TERT mutations. Considering the diagnosis of the annotated ROI bounding-box region at high resolution, the bounding box was magnified to the original scale to determine cytologic atypia levels, after which the boxes were cropped into 24 fragments (see Figure 6). Because each annotated box differs in size, the patches were overlapped, with the overlap size being set to automatically fit the corresponding sizes.
To integrate features from the 24 fragments, a CRNN, which is a combination of CNN and RNN, was constructed, after which a multilayer perceptron (MLP) module was created as shown in Figure 6. To extract the features of each patch, we applied ResNet152 as a CNN module and added Long Short-Term Memory (LSTM) (which has three RNN layers) to integrate the features and establish a two-layer MLP module to make the final predictions regarding the TERT mutation status. Figure 6 shows that all patches were passed through the CNN module and delivered to the RNN module.
Model training was performed using the CRNN network using NVIDIA RTX A6000 GPUs, with the data being loaded at a batch size of 64. Each CNN module used a pre-trained model generated on the ImageNet dataset. To measure the performance of the model, cross-entropy loss was applied, which yielded the summed outputs. According to training performance tracking, most training performances were saturated or overfitted after the 50th epoch; thus, tasks forcibly stopped early at that point. By sweeping the hyperparameters to derive the best-performing architecture, the learning rate was determined to be 1.0e−3 using an ADAM optimizer, and the best model was selected according to validation accuracy.

3. Results

3.1. Patient Cohort

The median age of the patients was 53 years (range 22–79 years) with the cohort including 9 males and 16 females. The 13 thyroid cancer cases with a mutant TERT promoter comprised 5 conventional papillary thyroid carcinomas, 3 follicular thyroid carcinomas, 1 poorly differentiated thyroid carcinoma, and 3 anaplastic thyroid carcinomas. The 12 thyroid cancer cases with a wild-type TERT promoter comprised 8 conventional papillary thyroid carcinomas, 2 follicular thyroid carcinomas, and 2 poorly differentiated thyroid carcinomas. Detailed information regarding the patient cohort is included in Table S1.

3.2. Tumor Classification

Five metrics, namely, precision, recall, f1 score, accuracy, and area under the curve (AUC) score were utilized to evaluate tumor classification performance. Some metrics, such as precision, recall, and f1 score, have two classification results; therefore, their results were macro-averaged over normal and tumor classes.
Figure 7 summarizes the performance results of the tumor classification performed using the DenseNet161, VGG16, and EfficientNet_b4 CNN architecture with image channel-wise normalization (i.e., subtracting the mean and dividing by the standard deviation from ImageNet datasets) and four different color-transformation methods. As five cross-validations were conducted, each result was averaged and its standard deviations indicated as shown in Figure 7.
Most bar plots show that the performance scores of the CNN architecture using the color transformation methods were better than those using image normalization, resulting in an improvement of approximately >6% (±2%) in terms of both accuracy and AUC score. The figures show that DenseNet161, VGG16, and EfficientNet_b4 had the best performance results with HSV-strong, HED-strong, and HSV-light transformation methods, respectively. More detailed results are provided in Supplementary Materials Table S2.

3.3. TERT Classification Performance Results Using the CRNN Model

As the color transformation methods showed improved results in the first step, we implemented additional experiments using the HSV-strong method in the next step. Table 2 shows the prediction results of TERT mutation status as negative or positive using the CRNN (ResNet152 + LSTM) model. Accordingly, an accuracy of 0.92 and AUC score of 0.90 were obtained without applying any color transforms. However, after applying the HSV-strong method, an accuracy of 0.95 and AUC score of 0.94 were obtained, which were noticeably better scores. As shown in Table 2 a,b, all performance metric scores were better when using the HSV-strong method than when using a plain image-normalization scheme.
To examine the areas highlighted by the CRNN model in the tumor patches predicted to be TERT positive, we created attention maps for the CNN modules on the basis of the score values extracted from the last fully connected layers. Because the TERT ROI has 24 patch images, each attention map is displayed in Figure 8, where the deeper the green color, the higher the attention score. In general, tumor cells showing size enlargement and nuclear hyperchromasia with prominent nucleoli, which are usually associated with aggressive biologic behavior, are concentrated in deep green areas in each attention map.

3.4. Whole Inference Process

In this experiment, our cascaded architecture, comprising trained CNN and CRNN, recognizes tumors and finally predicts TERT mutation status according to the tile-based WSI input. The model identifies tumor areas at a downsampling level value 2 (¼ downscale) with a patch size of 256 × 256 and then adjusts the predicted patches to level value 0 (original scale) with a size of 1024 × 1024. The higher-resolution patch was cropped to 24 patches with a size of 256 × 256, which were delivered simultaneously to the CRNN model to predict TERT mutation status. As shown in Figure 9, the CNN model determined whether each patch belonged to non-tumor or tumor areas. Once classified as a tumor, patches correctly predicted as TERT positive were marked with a green color, but those predicted incorrectly were indicated with a purple color.
Table 3 presents the validation results of the whole process using our cascade architecture. Each validation slide passed through the first CNN models (i.e., DenseNet161, VGG16, and EfficientNet_b4) and was retrieved along with the best color transformation methods, such as HSV-strong, HED-strong, and HSV-light. Given that we performed five-fold cross-validation, each result is the mean value of five validation sets. Two different transformation methods, namely, image normalization and HSV-strong, were applied to the CRNN model, and the results of each combination in terms of both sensitivity and specificity are shown in Table 3. Each combination of the CNN and CRNN models along with the HSV-strong transformation provided better performance. Notably, we observed a 23%, 15%, and 6% improvement in sensitivity and a 37%, 22%, and 8% improvement in specificity, respectively, compared to image normalization (Norm) as a baseline model.

4. Discussion

To the best of our knowledge, this study is the first to demonstrate that TERT promoter mutation status in thyroid cancer is associated with histologic features detectable using the deep learning approach. Through our cascaded deep learning approach, we learned that TERT promoter mutation status is associated with tumor cell size enlargement and nuclear atypia with prominent nuclear atypia, which is often associated with aggressive tumor behavior. This is consistent with the results of previous studies, which have shown that TERT promoter mutations usually accompany morphological changes [6,26].
Several previous studies have demonstrated the prognostic significance of TERT promoter mutations in thyroid cancer. In conventional papillary thyroid carcinoma, TERT promoter mutations are often associated with subtypes showing aggressive clinical behavior, including the tall cell [32] and hobnail subtypes [33]. Moreover, differentiated high-grade follicular cell-derived, poorly differentiated, and anaplastic thyroid carcinomas frequently harbor TERT promoter mutations [28]. Real-time PCR testing [34] and next-generation sequencing [35] are currently being used to confirm TERT promoter mutation status. However, testing all thyroid cancers for the TERT promoter mutation might not be cost effective considering the low incidence of TERT promoter mutations in thyroid cancer [7]. Therefore, predicting TERT promoter mutations via histologic images using a deep learning approach can be a useful screening tool.
Generally, WSI data have different color tones, thus color normalization or transformation has been regarded as an essential step in histopathology image processing. Regarding color normalization, prediction performance can also be influenced by a particular reference template. Manual selection of the templates also adds to the work. Furthermore, a recent study showed that a proper color transformation scheme outperformed the color normalization method [19]. Therefore, the current study focuses on color diversity rather than normalization, which enables us to leverage color transformation to improve TERT mutation prediction.
This study has some limitations. First, aberrant functionality of the TERT gene can be acquired through other mechanisms, including TERT mRNA overexpression and aberrant promoter methylation patterns [36,37,38]. Hence, thyroid cancers designated as having wild-type promoters in the present study might have exhibited abnormal TERT functioning, although the possibility is quite small. Pathogenic mutations in the TP53, BRAF, RAS, and other genes can also promote histologic changes associated with thyroid cancer [39,40]. Indeed, among the 13 thyroid cancer cases with TERT promoter mutations, 5 cases subsequently underwent targeted next-generation sequencing at the request of the clinician. Other than TERT promoter hotspot mutations, pathogenic mutations in NRAS, TP53, BRAF, RB1 deletion, and NCOA4-RET fusion were detected in these cases. However, only TERT promoter mutations were recurrently detected in these cases. We, therefore, suggest that TERT promoter mutations were most closely associated with the histologic findings observed in the present study.
We mainly performed PCR testing for TERT promoter mutations in thyroid cancer cases presenting with large tumor sizes showing aggressive behavior [4,26], which might lead to selection bias. Subsequent studies that include a larger number of thyroid cancer cases with a wider morphological spectrum should be performed to further validate the findings presented in this study.
We did not perform a subgroup analysis of thyroid cancer because of the limited number of cases. In a future study, we are planning to consider subgroup analysis to better predict TERT promoter mutations according to subtypes, with a larger number of cases.
Intratumoral heterogeneity is a well-known phenomenon in thyroid cancer [41], and TERT promoter mutation status can differ across distinct tumor areas. However, TERT promoter mutations in four of the five cases who underwent targeted next-generation sequencing were found to be clonal events after considering tumor cellularity and variant allele frequency of the TERT promoter mutation (data not included), with only one case having a TERT promoter mutation determined to be a subclonal event.
Moreover, a relatively small number of TERT promoter mutations and wild-type cancers were used in the current study. Training a deep learning model requires a much larger number of cases. However, TERT promoter mutations occur infrequently in thyroid cancer. Thus, collecting a large number of thyroid cancer cases with TERT promoter mutations was difficult.
Given our knowledge of the issues regarding case numbers, we focused on smart-sized cases confirmed with PCR testing and an efficient learning approach using transfer learning. Furthermore, this work focuses on tile-level rather than case-level prediction using deep learning. Thus, the number of tile images (Table 1) was sufficient to train the deep learning model. Although many relevant studies assign the same label to every patch in the tumor region of a WSI [14,15,16,17,18], this approach suffers from noisy training [20].
Therefore, to obtain good quality data from a limited number of TERT-positive cases, the two pathologists who are experienced in thyroid pathology that were involved in our study conducted fine-grained TERT ROI annotation in tumor areas. Using a smart-sized and good data set, the deep learning approach was able to differentiate the morphologic features at the tile level with 0.99 sensitivity for TERT mutation positivity.

5. Conclusions

High-sensitivity screening for TERT promoter mutation status in thyroid cancer is possible through histologic analysis with the assistance of deep learning along with color transformation schemes. Thyroid cancer patients with a high probability of harboring TERT promoter mutations can thus be screened for confirmative TERT promoter mutation testing, such as real-time PCR or next-generation sequencing, which can ultimately reduce the medical costs shouldered by them. Further studies with a larger cohort might be required to validate the results presented in the current study.

Supplementary Materials

The following supporting information can be downloaded at https://www.mdpi.com/article/10.3390/medicina59030536/s1: Table S1: Clinicopathologic characteristics of patient cohort. Table S2 contains more detailed results of Figure 7.

Author Contributions

Conceptualization, J.Y.P. and H.H.; investigation, M.K. and N.J.-Y.P.; data curation, J.K. and S.K.; writing—original draft preparation, J.K., S.K. and J.C.; writing—review and editing, H.H., J.C. and J.Y.P. All authors have read and agreed to the published version of the manuscript.

Funding

This research was supported by a grant from the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (grant number: HI21C0940) and Brain Pool Program through the National Research Foundation of Korea funded by the Ministry of Science and ICT (No.NRF-2020H1D3A2A02102040 & 2022H1D3A2A01096490) and the Ministry of Education (2021R1I1A3056903).

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Daegu Institutional review board (DGIRB 2021-11-002, 24 November 2021).

Informed Consent Statement

Written informed consent from the patients was waived because of the retrospective nature of the study.

Data Availability Statement

All data generated and analyzed during this study are included in this article and its Supplementary Materials.

Acknowledgments

The authors are grateful for the support provided by and wish to thank the Kyungpook National University Chilgok Hospital molecular pathology laboratory.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Sung, H.; Ferlay, J.; Siegel, R.L.; Laversanne, M.; Soerjomataram, I.; Jemal, A.; Bray, F. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J. Clin. 2021, 71, 209–249. [Google Scholar] [CrossRef] [PubMed]
  2. Milano, A.F. Thyroid Cancer: 20-Year Comparative Mortality and Survival Analysis of Six Thyroid Cancer Histologic Subtypes by Age, Sex, Race, Stage, Cohort Entry Time-Period and Disease Duration (SEER*Stat 8.3.2) A Systematic Review of 145,457 Cases for Diagnosis Years 1993–2013. J. Insur. Med. 2018, 47, 143–158. [Google Scholar]
  3. Cipriani, N.A. Prognostic Parameters in Differentiated Thyroid Carcinomas. Surg. Pathol. Clin. 2019, 12, 883–900. [Google Scholar] [CrossRef] [PubMed]
  4. Liu, X.; Bishop, J.; Shan, Y.; Pai, S.; Liu, D.; Murugan, A.K.; Sun, H.; El-Naggar, A.K.; Xiog, M. Highly prevalent TERT promoter mutations in aggressive thyroid cancers. Endocr. Relat. Cancer. 2013, 20, 603–610. [Google Scholar] [CrossRef] [Green Version]
  5. Vinagre, J.; Almeida, A.; Populo, H.; Batista, R.; Lyra, J.; Pinto, V.; Coelho, R.; Celestino, R.; Prazeres, H.; Lima, L.; et al. Frequency of TERT promoter mutations in human cancers. Nat. Commun. 2013, 4, 2185. [Google Scholar] [CrossRef] [Green Version]
  6. Landa, I.; Ganly, I.; Chan, T.A.; Mitsutake, N.; Matsuse, M.; Ibrahimpasic, T.; Ghossein, R.A.; Fagin, J.A. Frequent somatic TERT promoter mutations in thyroid cancer: Higher prevalence in advanced forms of the disease. J. Clin. Endocrinol. Metab. 2013, 98, E1562–E15626. [Google Scholar] [CrossRef] [Green Version]
  7. Cancer Genome Atlas Research, N. Integrated genomic characterization of papillary thyroid carcinoma. Cell 2014, 159, 676–690. [Google Scholar]
  8. Killela, P.J.; Reitman, Z.J.; Jiao, Y.; Bettegowda, C.; Agrawal, N.; Diaz, L.A., Jr.; Friedman, A.H.; Friedman, H.; Gallia, G.L.; Giovanella, B.C.; et al. TERT promoter mutations occur frequently in gliomas and a subset of tumors derived from cells with low rates of self-renewal. Proc. Natl. Acad. Sci. USA 2013, 110, 6021–6026. [Google Scholar] [CrossRef] [Green Version]
  9. Kim, T.H.; Kim, Y.E.; Ahn, S.; Kim, J.Y.; Ki, C.S.; Oh, Y.L.; Kim, K.; You, J.W.; Park, W.-Y.; Choe, J.-H.; et al. TERT promoter mutations and long-term survival in patients with thyroid cancer. Endocr Relat Cancer. 2016, 23, 813–823. [Google Scholar] [CrossRef] [Green Version]
  10. Xing, M.; Liu, R.; Liu, X.; Murugan, A.K.; Zhu, G.; Zeiger, M.A.; Pai, S.; Bishop, J. BRAF V600E and TERT promoter mutations cooperatively identify the most aggressive papillary thyroid cancer with highest recurrence. J. Clin. Oncolog. Off. J. Am. Soc. Clin. Oncol. 2014, 32, 2718–2726. [Google Scholar] [CrossRef] [Green Version]
  11. Liu, X.; Qu, S.; Liu, R.; Sheng, C.; Shi, X.; Zhu, G.; Murugan, A.K.; Guan, H.; Yu, H.; Wang, Y.; et al. TERT promoter mutations and their association with BRAF V600E mutation and aggressive clinicopathological characteristics of thyroid cancer. J. Clin. Endocrinol. Metab. 2014, 99, E1130–E1136. [Google Scholar] [CrossRef] [Green Version]
  12. Baxi, V.; Edwards, R.; Montalto, M.; Saha, S. Digital pathology and artificial intelligence in translational medicine and clinical practice. Mod. Pathol. 2022, 35, 23–32. [Google Scholar] [CrossRef] [PubMed]
  13. Barrera, C.; Velu, P.; Bera, K.; Wang, X.; Prasanna, P.; Khunger, M.; Khunger, A.; Velcheti, V.; Romero, E.; Madabhushi, A. Computer-extracted features relating to spatial arrangement of tumor infiltrating lymphocytes to predict response to nivolumab in non-small cell lung cancer (NSCLC). J. Clin. Oncol. 2018, 36 (Suppl. 15), 12115. [Google Scholar] [CrossRef]
  14. Kather, J.N.; Heij, L.R.; Grabsch, H.I.; Loeffler, C.; Echle, A.; Muti, H.S.; Krause, J.; Niehues, J.M.; Sommer, K.A.J.; Bankhead, P.; et al. Pan-cancer image-based detection of clinically actionable genetic alterations. Nat. Cancer. 2020, 1, 789–799. [Google Scholar] [CrossRef]
  15. Fu, Y.; Jung, A.W.; Torne, R.V.; Gonzalez, S.; Vöhringer, H.; Shmatko, A.; Yates, L.R.; Jimenez-Linan, M.; Moore, L.; Gerstung, M. Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis. Nat. Cancer 2020, 1, 800–810. [Google Scholar] [CrossRef] [PubMed]
  16. Chen, M.; Zhang, B.; Topatana, W.; Cao, J.; Zhu, H.; Juengpanich, S.; Mao, Q.; Yu, H.; Cai, X. Classification and mutation prediction based on histopathology H&E images in liver cancer using deep learning. NPJ Precis Oncol. 2020, 4, 14. [Google Scholar]
  17. Kather, J.N.; Pearson, A.T.; Halama, N.; Jäger, D.; Krause, J.; Loosen, S.H.; Marx, A.; Boor, P.; Tacke, F.; Neumann, U.P.; et al. Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nat. Med. 2019, 25, 1054–1056. [Google Scholar] [CrossRef] [PubMed]
  18. Coudray, N.; Ocampo, P.S.; Sakellaropoulos, T.; Narula, N.; Snuderl, M.; Fenyö, D.; Moreira, A.L.; Razavian, N.; Tsirigos, A. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat. Med. 2018, 24, 1559–1567. [Google Scholar] [CrossRef]
  19. Schirris, Y.; Gavves, E.; Nederlof, I.; Horlings, H.M.; Teuwen, J. DeepSMILE: Contrastive self-supervised pre-training benefits MSI and HRD classification directly from H&E whole-slide images in colorectal and breast cancer. Med. Image Anal. 2022, 79, 102464. [Google Scholar]
  20. Lu, M.Y.; Williamson, D.F.K.; Chen, T.Y.; Chen, R.J.; Barbieri, M.; Mahmood, F. Data-efficient and weakly supervised computational pathology on whole-slide images. Nat. Biomed. Eng. 2021, 5, 555–570. [Google Scholar] [CrossRef]
  21. Lazard, T.; Bataillon, G.; Naylor, P.; Popova, T.; Bidard, F.-C.; Stoppa-Lyonnet, D.; Stern, M.-H.; Decencière, E.; Walter, T.; Vincent-Salomon, A. Deep Learning identifies new morphological patterns of Homologous Recombination Deficiency in luminal breast cancers from whole slide images. Cell Rep. Med. 2022, 3, 100872. [Google Scholar] [CrossRef]
  22. Tellez, D.; Litjens, G.; Bándi, P.; Bulten, W.; Bokhorst, J.M.; Ciompi, F.; Van Der Laak, J. Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology. Med. Image Anal. 2019, 58, 101544. [Google Scholar] [CrossRef] [Green Version]
  23. Roy, S.; Kumar Jain, A.; Lal, S.; Kini, J. A study about color normalization methods for histopathology images. Micron 2018, 114, 42–61. [Google Scholar] [CrossRef]
  24. Campanella, G.; Hanna, M.G.; Geneslaw, L.; Miraflor, A.; Werneck Krauss Silva, V.; Busam, K.J.; Brogi, E.; Reuter, V.E.; Klimstra, D.S.; Fuches, T.J. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 2019, 25, 1301–1309. [Google Scholar] [CrossRef] [PubMed]
  25. Shi, B.; Bai, X.; Yao, C. An End-to-End Trainable Neural Network for Image-Based Sequence Recognition and Its Application to Scene Text Recognition. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 2298–2304. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. Liu, T.; Wang, N.; Cao, J.; Sofiadis, A.; Dinets, A.; Zedenius, J.; Laesson, C.; Xu, D. The age- and shorter telomere-dependent TERT promoter mutation in follicular thyroid cell-derived carcinomas. Oncogene 2014, 33, 4978–4984. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  27. Johnson, J.M.; Khoshgoftaar, T.M. Survey on deep learning with class imbalance. J. Big Data 2019, 6, 27. [Google Scholar] [CrossRef] [Green Version]
  28. Baloch, Z.W.; Asa, S.L.; Barletta, J.A.; Ghossein, R.A.; Juhlin, C.C.; Jung, C.K.; LiVolsi, V.A.; Papotti, M.G.; Sobrinho–Simões, M.; Tillini, G.; et al. Overview of the 2022 WHO Classification of Thyroid Neoplasms. Endocr. Pathol. 2022, 33, 27–63. [Google Scholar] [CrossRef]
  29. Huang, G.; Liu, Z.; Maaten, L.V.D.; Weinberger, K.Q. Densely Connected Convolutional Networks. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 4700–4708. [Google Scholar]
  30. He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016. [Google Scholar]
  31. Tan, M.; Le, Q. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. In Proceedings of the 36th International Conference on Machine Learning, Long Beach, CA, USA, 9–15 June 2019; pp. 6105–6114. [Google Scholar]
  32. Wang, X.; Cheng, W.; Liu, C.; Li, J. Tall cell variant of papillary thyroid carcinoma: Current evidence on clinicopathologic features and molecular biology. Oncotarget 2016, 7, 40792. [Google Scholar] [CrossRef] [Green Version]
  33. Yang, J.; Gong, Y.; Yan, S.; Chen, H.; Qin, S.; Gong, R. Association between TERT promoter mutations and clinical behaviors in differentiated thyroid carcinoma: A systematic review and meta-analysis. Endocrine 2020, 67, 44–57. [Google Scholar] [CrossRef] [Green Version]
  34. Kim, H.S.; Kwon, M.J.; Song, J.H.; Kim, E.S.; Kim, H.Y.; Min, K.W. Clinical implications of TERT promoter mutation on IDH mutation and MGMT promoter methylation in diffuse gliomas. Pathol. Res. Pract. 2018, 214, 881–888. [Google Scholar] [CrossRef]
  35. Lee, H.; Lee, B.; Kim, D.G.; Cho, Y.A.; Kim, J.S.; Suh, Y.L. Detection of TERT Promoter Mutations Using Targeted Next-Generation Sequencing: Overcoming GC Bias through Trial and Error. Cancer Res. Treat. 2022, 54, 75–83. [Google Scholar] [CrossRef]
  36. McKelvey, B.A.; Zeiger, M.A.; Umbricht, C.B. Characterization of TERT and BRAF copy number variation in papillary thyroid carcinoma: An analysis of the cancer genome atlas study. Genes Chromosomes Cancer 2021, 60, 403–409. [Google Scholar] [CrossRef]
  37. Tanaka, A.; Matsuse, M.; Saenko, V.; Nakao, T.; Yamanouchi, K.; Sakimura, C.; Yano, H.; Nishihsrs, E.; Hirokawa, M.; Suzuki, K.; et al. TERT mRNA Expression as a Novel Prognostic Marker in Papillary Thyroid Carcinomas. Thyroid 2019, 29, 1105–1114. [Google Scholar] [CrossRef]
  38. Paulsson, J.O.; Mu, N.; Shabo, I.; Wang, N.; Zedenius, J.; Larsson, C.; Juhlin, C.C. TERT aberrancies: A screening tool for malignancy in follicular thyroid tumours. Endocr. Relat. Cancer 2018, 25, 723–733. [Google Scholar] [CrossRef]
  39. Dolezal, J.M.; Trzcinska, A.; Liao, C.-Y.; Kochanny, S.; Blair, E.; Agrawal, N.; Keutegen, X.M.; Angelos, P.; Cipriani, N.A.; Pearson, A.T. Deep learning prediction of BRAF-RAS gene expression signature identifies noninvasive follicular thyroid neoplasms with papillary-like nuclear features. Mod. Pathol. 2021, 34, 862–874. [Google Scholar] [CrossRef]
  40. Tsou, P.; Wu, C.J. Mapping Driver Mutations to Histopathological Subtypes in Papillary Thyroid Carcinoma: Applying a Deep Convolutional Neural Network. J. Clin. Med. 2019, 8, 1675. [Google Scholar] [CrossRef] [Green Version]
  41. Affinito, O.; Orlandella, F.M.; Luciano, N.; Salvatore, M.; Salvatore, G.; Franzese, M. Evolution of intra-tumoral heterogeneity across different pathological stages in papillary thyroid carcinoma. Cancer Cell Int. 2022, 22, 263. [Google Scholar] [CrossRef]
Medicina 59 00536 g001 550
Figure 1. Representative images of thyroid cancers harboring a TERT promoter mutation (a,b) and a wild-type TERT promoter (c,d). Thyroid cancer with a TERT promoter mutation shows (a) a solid architecture with (b) prominent nuclear atypia and frequent mitosis, whereas that with a wild-type TERT promoter shows (c) conventional papillary architecture with (d) a lesser degree of nuclear atypia.
Figure 1. Representative images of thyroid cancers harboring a TERT promoter mutation (a,b) and a wild-type TERT promoter (c,d). Thyroid cancer with a TERT promoter mutation shows (a) a solid architecture with (b) prominent nuclear atypia and frequent mitosis, whereas that with a wild-type TERT promoter shows (c) conventional papillary architecture with (d) a lesser degree of nuclear atypia.
Medicina 59 00536 g001
Medicina 59 00536 g002 550
Figure 2. Whole-slide image (WSI) annotation: (a) normal regions, (b) tumor regions, and (c) TERT ROIs are marked in red and yellow contours and purple bounding boxes, respectively.
Figure 2. Whole-slide image (WSI) annotation: (a) normal regions, (b) tumor regions, and (c) TERT ROIs are marked in red and yellow contours and purple bounding boxes, respectively.
Medicina 59 00536 g002
Medicina 59 00536 g003 550
Figure 3. Overview of the TERT prediction architecture consisting of (a) a tumor classifier and (b) a mutation predictor.
Figure 3. Overview of the TERT prediction architecture consisting of (a) a tumor classifier and (b) a mutation predictor.
Medicina 59 00536 g003
Medicina 59 00536 g004 550
Figure 4. Examples of HED and HSV color transformations: (a) original patches and (b) color-transformed patches.
Figure 4. Examples of HED and HSV color transformations: (a) original patches and (b) color-transformed patches.
Medicina 59 00536 g004
Medicina 59 00536 g005 550
Figure 5. Overview of the CNN training model architecture for tumor area prediction showing (a) image tiling of WSI and patch filtering out from the tiled dataset (the yellow mask represents tumor area annotation, blue dot boxes are normal patches, and red dot boxes are tumor patches) and (b) the color transformation of the filtered patches.
Figure 5. Overview of the CNN training model architecture for tumor area prediction showing (a) image tiling of WSI and patch filtering out from the tiled dataset (the yellow mask represents tumor area annotation, blue dot boxes are normal patches, and red dot boxes are tumor patches) and (b) the color transformation of the filtered patches.
Medicina 59 00536 g005
Medicina 59 00536 g006 550
Figure 6. Overview of the CRNN training model architecture for TERT mutation prediction containing modules for: (a) TERT ROI bounding-box extraction, (b) tiling to 256 × 256 patches, and (c) CRNN.
Figure 6. Overview of the CRNN training model architecture for TERT mutation prediction containing modules for: (a) TERT ROI bounding-box extraction, (b) tiling to 256 × 256 patches, and (c) CRNN.
Medicina 59 00536 g006
Medicina 59 00536 g007 550
Figure 7. Tumor classification results obtained via five different transformation methods using the (a) DenseNet161, (b) VGG16, and (c) EfficientNet_b4 CNN models.
Figure 7. Tumor classification results obtained via five different transformation methods using the (a) DenseNet161, (b) VGG16, and (c) EfficientNet_b4 CNN models.
Medicina 59 00536 g007
Medicina 59 00536 g008 550
Figure 8. Attention maps of TERT-positive cases on the CRNN model.
Figure 8. Attention maps of TERT-positive cases on the CRNN model.
Medicina 59 00536 g008
Medicina 59 00536 g009 550
Figure 9. Inference results for TERT based on (a) negative and (b) positive cases without color transformations (as a base model) and (c) negative and (d) positive cases with color transformations. True positive TERT predictions are marked with a green color, whereas all others are indicated with a purple color.
Figure 9. Inference results for TERT based on (a) negative and (b) positive cases without color transformations (as a base model) and (c) negative and (d) positive cases with color transformations. True positive TERT predictions are marked with a green color, whereas all others are indicated with a purple color.
Medicina 59 00536 g009
Table
Table 1. Training–validation data split information for tumor classification and TERT prediction: (a) data counts for tumor and non-tumor patches and (b) TERT-negative and TERT-positive ROI boxes in each cross-validation (CV) set.
Table 1. Training–validation data split information for tumor classification and TERT prediction: (a) data counts for tumor and non-tumor patches and (b) TERT-negative and TERT-positive ROI boxes in each cross-validation (CV) set.
(a) Tumor and Non-Tumor Patches(b) TERT ROI for Negative and Positive Cases
TrainingValidationTrainingValidation
NormalTumorNormalTumorNegativePositiveNegativePositive
CV Set 126794264171009956991452254583
CV Set 224963276751193044411492394169
CV Set 32861824019827580971432364772
CV Set 434142208252751112911682462262
CV Set 53305529528383825881552863522
Table
Table 2. Results of TERT mutation prediction on the CRNN (ResNet152 + LSTM) model showing the mean and standard deviation values following five-fold cross-validation.
Table 2. Results of TERT mutation prediction on the CRNN (ResNet152 + LSTM) model showing the mean and standard deviation values following five-fold cross-validation.
(a) CRNN with Normal Transform(b) CRNN with HSV-Strong
PrecisionRecallf1-ScorePrecisionRecallf1-Score
Negative0.93 (±0.13)0.84 (±0.19)0.87 (±0.12)0.97 (±0.03)0.89 (±0.18)0.92 (±0.11)
Positive0.93 (±0.09)0.96 (±0.08)0.94 (±0.07)0.95 (±0.09)0.98 (±0.02)0.96 (±0.04)
Accuracy 0.92 (±0.08) 0.95 (±0.06)
AUC score 0.90 (±0.09) 0.94 (±0.08)
Table
Table 3. Inference results for the cascaded CNN + CRNN(ResNet152+LSTM) architectures.
Table 3. Inference results for the cascaded CNN + CRNN(ResNet152+LSTM) architectures.
MethodsSensitivitySpecificity
DenseNet161(Norm) + CRNN(Norm)0.76 (±0.43)0.23 (±0.18)
DenseNet161(HSV-strong) + CRNN(Norm)0.96 (±0.12)0.55 (±0.32)
DenseNet161(HSV-strong) + CRNN(HSV-strong)0.99 (±0.00)0.60 (±0.31)
VGG16(Norm) + CRNN(Norm)0.78 (±0.34)0.33 (±0.29)
VGG16(HED-strong) + CRNN(Norm)0.89 (±0.28)0.37 (±0.31)
VGG16(HED-strong) + CRNN(HSV-strong)0.93 (±0.26)0.50 (±0.31)
EfficientNet_b4(Norm) + CRNN(Norm)0.92 (±0.22)0.51 (±0.30)
EfficientNet_b4(HSV-light) + CRNN(Norm)0.95 (±0.12)0.50 (±0.34)
EfficientNet_b4(HSV-light) + CRNN(HSV-strong)0.98 (±0.05)0.59 (±0.26)
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Kim, J.; Ko, S.; Kim, M.; Park, N.J.-Y.; Han, H.; Cho, J.; Park, J.Y. Deep Learning Prediction of TERT Promoter Mutation Status in Thyroid Cancer Using Histologic Images. Medicina 2023, 59, 536. https://doi.org/10.3390/medicina59030536

AMA Style

Kim J, Ko S, Kim M, Park NJ-Y, Han H, Cho J, Park JY. Deep Learning Prediction of TERT Promoter Mutation Status in Thyroid Cancer Using Histologic Images. Medicina. 2023; 59(3):536. https://doi.org/10.3390/medicina59030536

Chicago/Turabian Style

Kim, Jinhee, Seokhwan Ko, Moonsik Kim, Nora Jee-Young Park, Hyungsoo Han, Junghwan Cho, and Ji Young Park. 2023. "Deep Learning Prediction of TERT Promoter Mutation Status in Thyroid Cancer Using Histologic Images" Medicina 59, no. 3: 536. https://doi.org/10.3390/medicina59030536

APA Style

Kim, J., Ko, S., Kim, M., Park, N. J.-Y., Han, H., Cho, J., & Park, J. Y. (2023). Deep Learning Prediction of TERT Promoter Mutation Status in Thyroid Cancer Using Histologic Images. Medicina, 59(3), 536. https://doi.org/10.3390/medicina59030536

Article Metrics

Back to TopTop