Next Article in Journal
First-Line Immunotherapy with Check-Point Inhibitors: Prospective Assessment of Cognitive Function
Next Article in Special Issue
Deep Learning and Registration-Based Mapping for Analyzing the Distribution of Nodal Metastases in Head and Neck Cancer Cohorts: Informing Optimal Radiotherapy Target Volume Design
Previous Article in Journal
Systematic Review of the Short-Term versus Long-Term Duration of Antibiotic Management for Neutropenic Fever in Patients with Cancer
Previous Article in Special Issue
Diagnostic Performance of Various Ultrasound Risk Stratification Systems for Benign and Malignant Thyroid Nodules: A Meta-Analysis
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Cancers
Volume 15
Issue 5
10.3390/cancers15051613
cancers-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Clinical-Radiomics Nomogram Based on Contrast-Enhanced Ultrasound for Preoperative Prediction of Cervical Lymph Node Metastasis in Papillary Thyroid Carcinoma

by
Liqing Jiang
1,
Zijian Zhang
2,3,4,
Shiyan Guo
1,
Yongfeng Zhao
1 and
Ping Zhou
1,*
1
Department of Ultrasound, The Third Xiangya Hospital, Central South University, Changsha 410013, China
2
Xiangya Lung Cancer Center, Xiangya Hospital, Central South University, Changsha 410008, China
3
Department of Radiation Oncology, Xiangya Hospital, Central South University, Changsha 410008, China
4
National Clinical Research Center for Geriatric Disorders, Xiangya Hospital, Changsha 410008, China
*
Author to whom correspondence should be addressed.
Cancers 2023, 15(5), 1613; https://doi.org/10.3390/cancers15051613
Submission received: 7 February 2023 / Revised: 1 March 2023 / Accepted: 4 March 2023 / Published: 5 March 2023
(This article belongs to the Special Issue Head and Neck Cancer Imaging and Image Analysis)
Browse Figures
Versions Notes

Abstract

:

Simple Summary

Improving the precision of preoperative LNM assessment is crucial for determining the scope of PTC surgery, reducing complications, and preventing recurrence. Few studies have applied radiomics analysis based on contrast-enhanced ultrasound (CEUS) to the prediction of LNM in PTC. Our study found that CEUS-based radiomics, as a promising quantitative analysis, provides incremental value to clinical prediction and management of LNM in PTC. In addition, the developed clinical-radiomics nomogram demonstrated promising value for predicting LNM. It may be an effective, noninvasive tool for preoperative prediction of LNM in clinical use.

Abstract

This study aimed to establish a new clinical-radiomics nomogram based on ultrasound (US) for cervical lymph node metastasis (LNM) in papillary thyroid carcinoma (PTC). We collected 211 patients with PTC between June 2018 and April 2020, then we randomly divided these patients into the training set (n = 148) and the validation set (n = 63). 837 radiomics features were extracted from B-mode ultrasound (BMUS) images and contrast-enhanced ultrasound (CEUS) images. The maximum relevance minimum redundancy (mRMR) algorithm, least absolute shrinkage and selection operator (LASSO) algorithm, and backward stepwise logistic regression (LR) were applied to select key features and establish a radiomics score (Radscore), including BMUS Radscore and CEUS Radscore. The clinical model and clinical-radiomics model were established using the univariate analysis and multivariate backward stepwise LR. The clinical-radiomics model was finally presented as a clinical-radiomics nomogram, the performance of which was evaluated by the receiver operating characteristic curves, Hosmer–Lemeshow test, calibration curves, and decision curve analysis (DCA). The results show that the clinical-radiomics nomogram was constructed by four predictors, including gender, age, US-reported LNM, and CEUS Radscore. The clinical-radiomics nomogram performed well in both the training set (AUC = 0.820) and the validation set (AUC = 0.814). The Hosmer–Lemeshow test and the calibration curves demonstrated good calibration. The DCA showed that the clinical-radiomics nomogram had satisfactory clinical utility. The clinical-radiomics nomogram constructed by CEUS Radscore and key clinical features can be used as an effective tool for individualized prediction of cervical LNM in PTC.

1. Introduction

Thyroid cancer (TC) ranked ninth among the incidence of human malignancies worldwide [1], and papillary thyroid carcinoma (PTC) is the most common pathological type among TC, accounting for 80–90% of cases [2]. PTC often has a good prognosis and a low mortality rate [3]; however, some PTCs exhibit cervical lymph nodes metastasis (LNM), which increases the risk of local recurrence and decreases overall patient survival [4,5]. Surgery is the main treatment for PTC, but because there are no effective solutions for predicting cervical LNM before surgery, it is still debatable whether to do prophylactic cervical lymph node (LN) dissection in patients with PTC. Although some experts recommend prophylactic cervical LN dissection for PTC patients [6,7], other studies suggest that this procedure may increase the risk of postoperative complications rather than considerably improve survival [8,9]. Therefore, it is crucial to improve the precision of preoperative LNM assessment to determine the scope of PTC surgery and enhance patient survival.
Preoperatively noninvasive assessment of cervical LNM is challenging. Ultrasound (US) is commonly used and plays an important role [10], as radiologists can determine the presence of cervical LNM by observing the sonographic features of bilateral cervical LNs. However, the detection rate of metastatic LNs by conventional US is not satisfactory, especially in the central region [7]. Given this, some scholars have targeted their studies on the US characteristics of primary tumors. Liu et al. reported that intratumoral microcalcification and parenchyma microcalcification were independent risk factors for LNM of PTC [11]. Zhan et al. found that cervical LNM was associated with high or equal enhancement at the peak time of CEUS, heterogeneous enhancement, and PTC size [12]. However, no validated and accepted model has been established, and US evaluation is susceptible to differences in the professional experience of radiologists.
Radiomics offers the potential for noninvasively precise diagnosis and treatment by extracting high-throughput quantitative information from medical images to create models that forecast intrinsic heterogeneity to support clinical decision-making [13]. Several scholars have successfully applied the radiomics approach from the primary tumor to predict LNM in malignant tumors, such as cervical cancer [14], pancreatic ductal adenocarcinoma [15], and laryngeal squamous cell carcinoma [16]. Several studies have not only confirmed the feasibility of primary tumor-based radiomics analysis for predicting cervical LNM in patients with PTC but also have combined the ultrasound radiomics signature and clinical data to construct robust predictive models for predicting LNM. The radiomics in these studies are mostly based on B-mode ultrasound (BMUS) and ultrasound elastography [17,18,19]. Few studies applied CEUS-based radiomics from primary tumors to the prediction of LNM in PTC, and it is not clear whether integrating CEUS-based radiomics with clinical risk factors enhances the ability to predict LNM in PTC.
The nomogram is based on a multivariate regression analysis that integrates multiple predictors, then graphically depicts the numerical relationship between the specific disease and risk factors, and finally intuitively provides the numerical probability of an outcome event through a scoring system [20,21,22]. Therefore, we aim to construct and validate a nomogram model combining the use of CEUS radiomics and clinical data to provide tools for the non-invasive prediction of cervical LNM in PTC for clinicians, thereby achieving the goal of individualized medicine.

2. Materials and Methods

2.1. Study Population

This study was approved by the ethics committee at the Third Xiangya Hospital. Written informed consent was waived owing to the retrospective nature of this study. We retrospectively collected patients with thyroid nodules who underwent preoperative ultrasound examination at the ultrasound department of our hospital between June 2018 and April 2020. Inclusion criteria: (1) postoperative pathology confirmed PTC; (2) treated with thyroidectomy and cervical LN dissection; (3) primary and solitary thyroid carcinoma; (4) no preoperative anticancer treatment; (5) ultrasound examinations were performed within two weeks before surgery. Exclusion criteria: (1) the lesion displayed incompletely in the US image due to excessive size; (2) poor image quality. A total of 211 patients were finally included. According to postoperative pathology, the patients were divided into the LNM group and the non-LNM group.

2.2. Image Acquisition and Clinicoradiological Characteristics Collection

Ultrasonography was performed using a GE LOGIQ E9 color Doppler ultrasonic instrument with a 9L linear array probe (2–9 MHz). The US physician first performed a BMUS examination of the thyroid gland, and saved the BMUS image of the largest long-axis section of the lesion, then switched to real-time CEUS mode. Next, the US physician asked the patient to breathe calmly and tried to keep the observation section unchanged. The contrast agent used was the SonoVue (Bracco, Milan, Italy). The patient received a bolus injection of 2.4 mL contrast agent through the antecubital vein, followed immediately by 5 mL of normal saline. The US physician observed the dynamic perfusion process of the lesion continuously and stored the dynamic images. A frame of the image at the peak time of CEUS was selected to store. Finally, two images of each nodule (BMUS image and CEUS image) were exported in Dicom format.
The conclusion suggestive of “LNM” in the US report was considered to be US-reported LN status positive. The conclusions of “undetectable LN”, “reactive hyperplastic lymph nodes” and “visible LN“ in the absence of metastasis were considered to be US-reported LN status negative. According to the 2015 American Thyroid Association (ATA) guidelines [23], the suspicious US signs suggestive of cervical LNM included round shape (aspect ratio > 0.5), calcifications, cystic changes, hyperechogenicity, and peripheral blood flow signals. One or more LNs that met one or more of the five criteria would be considered positive.
The following ultrasound features were recorded: primary site (left lobe, right lobe, isthmus), location (sub-capsular, intra-thyroidal), tumor size, echogenicity (iso/hyperechoic, hypoechoic, marked hypoechoic), aspect ratio (>1, ≤1), margin (smooth, ill-defined, irregular), calcification (absent or present), enhancement pattern (hypo-enhancement, iso-enhancement, hyper-enhancement). Demographic characteristics including gender and age were collected from the medical records.

2.3. Image Segmentation and Feature Extraction

ITK-SNAP software (open source software; http://www.itksnap.org, accessed on 7 August 2020) was used to segment the nodules, and the region of interest (ROI) was outlined along the contour of the targeted lesion. To assess interobserver reproducibility, 30 cases were randomly selected from all cases, and the images were segmented by two US physicians (reader1 and reader2), respectively. One US physician (reader1) performed all image segmentation. Next, the radiomics plug-in of 3D-Slicer software was used to perform feature extraction of the thyroid nodules. Before extracting features, the images were normalized including resampling to a voxel size of 1 mm × 1 mm × 1 mm, setting the bin width parameter in 3D-Slicer at 25 HU to discretize the voxel intensity. 837 radiomics features were extracted from each BMUS image and each CEUS image, respectively, and feature categories included first-order statistics, gray level dependence matrix (GLDM), gray level co-occurrence matrix (GLCM), gray level run length matrix (GLRLM), gray level size zone matrix (GLSZM) and neighborhood gray tone difference matrix (NGTDM).

2.4. Feature Selection and Radiomics Score Construction

We divided the 211 patients into a training set (n = 148) and validation set (n = 63) by 7:3 stratified random sampling method, and then radiomics features in the training and validation sets were z-score normalized according to the mean and standard deviation of the training set.
The process of radiomics feature selection and radiomics score construction is as follows. First, we calculated the interclass correlation coefficient (ICC) based on the radiomics features extracted after image segmentation by the two US physicians, and highly reproducible (ICC > 0.75) features were retained. Then the redundant and irrelevant features were removed using the minimum redundancy maximum correlation (mRMR) algorithm, and the best top 30 features from each image were selected. Next, the radiomics features associated with LNM were obtained by using the least absolute shrinkage and selection operator (LASSO) algorithm. Finally, a backward stepwise logistic regression (LR) with Akaike information criterion (AIC) was used to select the features constituting the logistic regression model, and the model score was the radiomics score (Radscore). According to the above process, we obtained BMUS Radscore and CEUS Radscore, respectively. Then a Mann–Whitney U test was used to assess the association of the Radscore with LNM.

2.5. Development of the Clinical Model and the Clinical-Radiomics Nomogram

We constructed the model based on the training set and subsequently applied the model in the validation set to test its performance. In the training set, first, we performed a univariate analysis of clinical parameters (including demographic parameters and ultrasound features) and two Radscores. Stepwise multivariate LR analysis was then performed to develop a clinical model using clinical risk factors with p-value < 0.05 in the univariate analysis as candidate predictors.
Clinical risk factors and two Radscores were introduced into multivariate LR to build the clinical-radiomics combined model. A backward stepwise selection process with the AIC as the stopping rule was performed. A nomogram based on the clinical-radiomics model was drawn to visualize the logistic regression model for individualized assessment of patients’ risk of cervical LNM.

2.6. Model Validation

We plotted the receiver operating characteristic (ROC) curves and evaluated the predictive ability of the clinical-radiomics nomogram by the area under the ROC curve (AUC). Comparisons between the clinical model and the clinical-radiomics model were made using the integrated discrimination improvement (IDI) index.
We used the Hosmer–Lemeshow test and calibration curve to assess the calibration performance of the clinical-radiomics nomogram and used decision curve analysis (DCA) to assess the clinical utility of the clinical-radiomics nomogram by estimating the net benefit of the training set at each threshold probability.

2.7. Statistical Analysis

R software and associated packages were used for statistical analyses. Quantitative data were presented as mean ± standard deviation or median ± interquartile ranges. The t-test or Mann–Whitney U test was used to compare the differences in the measurement data between the two groups, and the Chi-square test or Fisher’s exact test was used to compare the differences in the enumeration data between the two groups. The difference between the two groups was statistically significant with p < 0.05.

3. Results

3.1. Clinicoradiological Characteristics

The study flowchart and radiomics workflow are reported in Figure 1. This study included 211 patients with solitary PTC, 88 of whom had positive cervical LNM results and 123 of whom had negative cervical LNM results. The patients were randomly divided in a 7:3 ratio, with 148 cases allocated to the training set and 63 cases allocated to the validation set, and the positive rate of cervical LNM was 39.9% and 46.0% in the training and validation sets, respectively, with no statistically significant difference (p = 0.406). Patients in the training and validation sets are listed by their clinical features in Table 1. Between the training and validation sets, there was no statistically significant difference in the clinical characteristics of patients (p > 0.05 for all), indicating that the baseline data were comparable for both sets. Table 2 shows the univariate analysis results between cervical LNM and candidate variables in the two groups. Age, tumor size, and US-reported LN status were associated with LNM in both the training and validation set (p < 0.05). Primary site, echogenicity, margin, microcalcification, and enhancement patterns were not associated with LNM in the training or validation set (p > 0.05). In the training set, males were more likely to have LNM (p < 0.05); but gender was not associated with LNM (p > 0.05) in the validation set. In the validation set, sub-capsular location was more likely to have LNM (p < 0.05); but tumor location was not associated with LNM (p > 0.05) in the training set.

3.2. Radiomics Score Building

After removing the less stable features with ICC ≤ 0.75, 768 and 775 features were kept from the BMUS and CEUS images of each patient, respectively; 30 features were retained in each image by the mRMR algorithm. After the LASSO regression (Figure 2), 2 features from BMUS images and 10 features from CEUS images were selected. After the backward stepwise logistic regression analysis, 1 radiomics feature from the BMUS and 5 radiomics features from CEUS images were found to associate with LNM and used to construct the BMUS radiomics score and CEUS radiomics score, respectively. The formulas for BMUS Radscore and CEUS Radscore were as follows:
BMUS Radscore = −0.4535 − 0.5901 × wavelet.HLH_glszm_ZonePercentage
CEUS Radscore = − 0.5624 − 0.5753 × wavelet.LHL_glcm_Idn − 0.4804 × wavelet.LHL_gldm_DependenceVariance + 0.3809 × wavelet.HHH_firstorder_Median + 0.5590 × original_glszm_SizeZoneNonUniformity − 0.9770 × wavelet.LHH_glrlm_GrayLevelNonUniformityNormalized

3.3. Model Building and Validation

The univariate results showed significant differences (p < 0.05) in age, gender, tumor size, and US-reported LN status between the LNM positive and negative groups in the training set (Table 2). After backward stepwise multivariate logistic regression analysis, age < 55 years and US-reported LN status positive were still identified to be significant factors (p < 0.05) for LNM (Table 3). The AUCs for the clinical model were 0.700 (0.617–0.784) and 0.763 (0.650–0.877) in the training and validation set, respectively (Figure 3).
Six factors, namely, age, gender, tumor size, US-reported LN status, BMUS Radscore, and CEUS Radscore were introduced into stepwise multivariate logistic regression. As a result, the clinical-radiomics combined model was constructed based on gender, age, US-reported LN status, and CEUS Radscore (Table 3). In the combined model, age (OR, 0.18; 95%CI, 0.05–0.70), US-reported LN status (OR, 5.16; 95%CI, 1.40–18.98), and CEUS Radscore (OR, 2.75; 95%CI, 1.79–4.23) were independently associated with LNM. The AUCs for the combined model were 0.820 (0.749–0.890) and 0.814 (0.707–0.922) in the training and validation set, respectively (Figure 3). Then we compared the clinical model and the clinical-radiomics model [IDI = 15.42% (9.15–21.69%), p < 0.001 in the training set; IDI = 8.59% (0.91–16.26%), p = 0.028 in the validation set], a notable improvement in discrimination was seen in the clinical-radiomics model. This might mean that the addition of CEUS Radiomics improved LNM risk discrimination beyond the clinical model. We visualized the clinical-radiomics model using a clinical-radiomics nomogram (Figure 4A). The calibration curves (Figure 4B,C) and the Hosmer–Lemeshow test revealed that there was no significant difference between the probability predicted by the clinical-radiomics nomogram and actual probabilities (the Hosmer–Lemeshow test: p-value = 0.569 in the training set; p-value = 0.558 in the validation set). The DCA (Figure 5) showed that a treatment plan based on the clinical-radiomics nomogram might be more beneficial than either the treat-all-patients strategy or the treat-none strategy, and the net benefit of the clinical-radiomics nomogram was higher than the clinical model across the majority of the range of threshold probabilities.

4. Discussion

Currently, surgery is the main treatment for PTC. However, it is controversial whether total thyroidectomy and prophylactic lymph node dissection can provide substantial benefits for patients with PTC. Furthermore, surgical demolitive interventions may be accompanied by more severe complications, such as recurrent laryngeal nerve paralysis, cervical hematoma, and hypoparathyroidism [24,25]. Therefore, preoperatively prognostic markers to assess the risk of cervical LNM in PTC are of great significance to effectively avoid overdiagnosis and improve prognosis. To this end, Vincenzo Marotta et al. found that germline VEGF-A single nucleotide polymorphisms (SNPs) were stable and accessible prognostic markers for DTC (Differentiated Thyroid Cancer; PTC accounts for 85% of DTC [23]) obtained by peripheral blood testing, and constitute promising tools to enhance prognostic stratification of DTC [26]. In addition, Zhang et al. analyzed the BRAFV600E mutation from thyroid nodule samples collected by Fine-Needle Aspiration (FNA) biopsy and found that BRAFV600E mutation was an independent prognostic marker of central cervical LNM in PTC [27]. Unlike the aforementioned markers, radiomics is a non-invasive, time-saving, and cost-effective prognostic marker which was confirmed by many recent studies [13,28,29,30].
In the current study, we developed and validated a clinical-radiomics nomogram that combines key clinical risk factors and CEUS radiomics features for the individualized prediction of LNM in PTC. Compared with the clinical model, the clinical-radiomics nomogram had the better diagnostic efficacy for predicting LNM, with AUCs of 0.820 and 0.814, in the training set and validation set, respectively. Thus, our study suggests that clinical-radiomics nomogram can be used to assess the risk of LNM for PTC patients preoperatively and non-invasively, and provide a reference for individualized treatment planning.
According to the TNM staging system of the AJCC 8th edition [23], 55 years was used as the cut-off value for the age of patients with PTC in this study. Both univariate and multivariate analyses showed that age was significantly and negatively associated with LNM, and young age was an independent risk factor for cervical LNM in patients with PTC. This is consistent with previous findings [11,31]. Therefore, during the preoperative US examination, it was crucial to carefully examine the LN status in young patients with PTC. Male PTC patients had a higher likelihood of LNM than female PTC patients [32,33,34], which was also confirmed by our findings. This may be related to sex hormones [35]. After selection with the backward stepwise method, gender was kept in the final model. Both univariate and multivariate analyses showed that US-reported LN status was significantly associated with LNM, so US-reported LN status was included in the final prediction model. This is similar to other studies [18,19], showing that US-reported LN status is an important part of the preoperative prediction. However, US has limitations in the assessment of cervical LNM. Central cervical LNM was easily missed due to its deep location and the thyroid gland that overlies it; some meta-analyses have shown that the sensitivity of ultrasound for the assessment of central cervical LNM is less than 35% [7,36]. In addition, US diagnosis is based on visual qualitative judgments that were constrained by US physicians’ experience differences. Therefore, complementary indicators are needed for a more efficient diagnosis.
In recent years, radiomics is one of the hot research topics in medical imaging. Radiomics analysis can overcome the possible strong subjectivity of traditional medical image interpretation and convert medical imaging data into quantitative biomarkers through innovative computational methods [37]. US-based radiomics techniques have developed rapidly and have been applied to the differential diagnosis of tumors and the assessment of tumor aggressiveness, including malignant parotid gland lesions [38], breast cancer [39,40], and renal cell carcinoma [41]. With regard to PTC, Enock Adjei Agyekum et al. [42] reported that the radiomics model based on preoperative US images provided promising results in assessing cervical LNM in patients with PTC. Some studies have explored the further application of multimodal US radiomics [19,43,44]. Our preliminary study found that radiomics analysis of BMUS and CEUS images has good diagnostic efficacy for discriminating thyroid nodules, and the diagnostic efficacy of the BMUS + CEUS radiomics model is superior, suggesting the potential application value of multimodal radiomics in identifying benign and malignant thyroid nodules [45]. CEUS can help for preoperative prediction of LNM in PTC by intravenously injecting a blood pool contrast agent to show tissue microperfusion [5]. In this study, we explored the use of combined BMUS and CEUS images for radiomics analysis in predicting LNM of PTC. It is noticeable that in univariate analysis, BMUS Radscore and CEUS Radscore were significantly associated with LNM, but BMUS Radscore did not enter into the final clinical-radiomics model, which is similar to the results of Jiang et al. [19], showing that the BMUS Radscore was excluded due to its insufficient predictive power for LNM. We discovered that in the final multivariate stepwise logistic regression, the superior discriminatory power of CEUS Radscore weakened the weight of the BMUS Radscore.
Previous studies have found that the CEUS enhancement pattern of tumors can help predict LNM and that hyper- or iso-enhancement can be an independent risk factor for LNM [12,46]. The enhancement pattern of CEUS in our study is not related to LNM, probably due to the different data set with the small sample size of high enhancement in our study; in addition, enhancement intensity, as a qualitative characteristic judged by the naked eye, involves a certain degree of subjectivity. This does not affect our inspiring finding that quantitative radiomics analysis based on the CEUS image is strongly associated with LNM. In contrast to visual inspection of enhancement intensity and homogeneity, radiomics may be able to quantitatively decode important information about the heterogeneity of tumor microcirculation, which is associated with intratumoral perfusion, vascular permeability, and angiogenesis [47,48,49]. The CEUS Radscore included five radiomics features, and 80% of the selected radiomics features were wavelet-based features. The wavelet transform can reveal the hidden features of medical images at multiple scales [50,51], amplify the heterogeneous information of target tumor texture features, and enhance the discriminative ability [52]. We also noticed that most selected CEUS radiomics features characterize the spatial distribution of lesion voxels, proving that PTCs with higher vascular heterogeneity are prone to exhibit aggressive biological behavior. These radiomics features, which are hard to identify with the naked eye, have the potential to be non-invasive biomarkers for the preoperative prediction of cervical LN status in PTC.
Finally, we combined the radiomics score and key preoperative clinical features to create a clinical-radiomics model, and for clinical application, a nomogram was created as a visualization of the logistic regression model. The AUC and clinical benefit of the clinical-radiomics nomogram were higher than using the clinical model. Combining key clinical features with CEUS Radscore resulted in a significant improvement in IDI, demonstrating the incremental value of CEUS-based radiomics for preoperative clinical prediction of LN status.
The limitations of our study should be acknowledged. 1: This study was a retrospective single-center study, so a prospective multicenter study with a large sample size is needed for further improvement before practical application. 2: Our radiomics analysis of the CEUS image was based on a single-frame image due to technological limitations, and much information might be missed compared to the analysis of the entire perfusion process. Further research is needed into image processing and feature extraction of the dynamic image. 3: The radiomics analysis in this study was based on the images of the primary tumors, and there are still few studies that establish a radiomics model based on LN sonograms for LNM prediction in PTC patients. Future research is needed to determine the feasibility and predictive value of radiomics analysis based on LN sonograms or a combination of the primary tumor and LN images.

5. Conclusions

In conclusion, we constructed a clinical-radiomics nomogram incorporating CEUS Radscore and key clinical features. It demonstrated favorable predictive ability for LNM in patients with PTC and can be used as an effective tool for individualized prediction of cervical LNM in PTC.

Author Contributions

Conceptualization, L.J., P.Z. and Y.Z.; methodology, L.J. and S.G.; software, L.J. and S.G.; validation, L.J., P.Z. and Z.Z.; formal analysis, L.J.; investigation, L.J.; resources, P.Z.; data curation, L.J.; writing—original draft preparation, L.J.; writing—review and editing, L.J., Z.Z. and P.Z.; visualization, L.J.; supervision, P.Z.; project administration, L.J.; funding acquisition, P.Z. and Y.Z. All authors have read and agreed to the published version of the manuscript.

Funding

This research was supported by the National Natural Science Foundation of China (No. 81871367), the Key Research and Development Program of Hunan Province, China (No. 2018SK21217), and the Natural Science Foundation of Hunan Province, China (No. 2021JJ31037).

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and approved by the Ethics Committee of the Third Xiangya Hospital (2021-S018, approval date: 8 March 2021).

Informed Consent Statement

Patient consent was waived by the ethics committee due to the retrospective nature of the study.

Data Availability Statement

The original contributions presented in the study are included in the article; further inquiries can be directed to the corresponding author.

Acknowledgments

Thanks to all members of the Department of Ultrasound, Third Xiangya Hospital, Central South University for their help in the manuscript preparation. This work was supported in part by the High Performance Computing Center of Central South University.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Bray, F.; Ferlay, J.; Soerjomataram, I.; Siegel, R.L.; Torre, L.A.; Jemal, A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J. Clin. 2018, 68, 394–424. [Google Scholar] [CrossRef] [Green Version]
  2. Grimm, D. Cell and Molecular Biology of Thyroid Disorders. Int. J. Mol. Sci. 2019, 20, 2895. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Povoa, A.A.; Teixeira, E.; Bella-Cueto, M.R.; Melo, M.; Oliveira, M.J.; Sobrinho-Simoes, M.; Maciel, J.; Soares, P. Clinicopathological Features as Prognostic Predictors of Poor Outcome in Papillary Thyroid Carcinoma. Cancers 2020, 12, 3186. [Google Scholar] [CrossRef]
  4. Adam, M.A.; Pura, J.; Goffredo, P.; Dinan, M.A.; Reed, S.D.; Scheri, R.P.; Hyslop, T.; Roman, S.A.; Sosa, J.A. Presence and Number of Lymph Node Metastases Are Associated With Compromised Survival for Patients Younger Than Age 45 Years With Papillary Thyroid Cancer. J. Clin. Oncol. 2015, 33, 2370–2375. [Google Scholar] [CrossRef] [PubMed]
  5. Xue, T.; Liu, C.; Liu, J.J.; Hao, Y.H.; Shi, Y.P.; Zhang, X.X.; Zhang, Y.J.; Zhao, Y.F.; Liu, L.P. Analysis of the Relevance of the Ultrasonographic Features of Papillary Thyroid Carcinoma and Cervical Lymph Node Metastasis on Conventional and Contrast-Enhanced Ultrasonography. Front. Oncol. 2021, 11, 794399. [Google Scholar] [CrossRef]
  6. Parvathareddy, S.K.; Siraj, A.K.; Ahmed, S.O.; DeVera, F.; Al-Sobhi, S.S.; Al-Dayel, F.; Al-Kuraya, K.S. Risk Factors for Central Lymph Node Metastases and Benefit of Prophylactic Central Lymph Node Dissection in Middle Eastern Patients With cN0 Papillary Thyroid Carcinoma. Front. Oncol. 2021, 11, 819824. [Google Scholar] [CrossRef] [PubMed]
  7. Zhao, H.; Li, H. Meta-analysis of ultrasound for cervical lymph nodes in papillary thyroid cancer: Diagnosis of central and lateral compartment nodal metastases. Eur. J. Radiol. 2019, 112, 14–21. [Google Scholar] [CrossRef] [PubMed]
  8. Yan, X.Q.; Zhang, Z.Z.; Yu, W.J.; Ma, Z.S.; Chen, M.L.; Xie, B.J. Prophylactic Central Neck Dissection for cN1b Papillary Thyroid Carcinoma: A Systematic Review and Meta-Analysis. Front. Oncol. 2021, 11, 803986. [Google Scholar] [CrossRef]
  9. Sanabria, A.; Betancourt, C.; Sanchez, J.G.; Garcia, C. Prophylactic Central Neck Lymph Node Dissection in Low-Risk Thyroid Carcinoma Patients Does not Decrease the Incidence of Locoregional Recurrence: A Meta-Analysis of Randomized Trials. Ann. Surg. 2022, 276, 66–73. [Google Scholar] [CrossRef]
  10. Wu, Y.; Rao, K.; Liu, J.; Han, C.; Gong, L.; Chong, Y.; Liu, Z.; Xu, X. Machine Learning Algorithms for the Prediction of Central Lymph Node Metastasis in Patients With Papillary Thyroid Cancer. Front. Endocrinol. 2020, 11, 577537. [Google Scholar] [CrossRef]
  11. Liu, J.; Jia, X.; Gu, Y.; Chen, X.; Guan, L.; Yan, J.; Zhai, H.; Zhou, N.; Dong, Y.; Zhan, W.; et al. Thyroid Parenchyma Microcalcifications on Ultrasound for Predicting Lymph Node Metastasis in Papillary Thyroid Carcinoma: A Prospective Multicenter Study in China. Front. Oncol. 2021, 11, 609075. [Google Scholar] [CrossRef] [PubMed]
  12. Zhan, J.; Zhang, L.H.; Yu, Q.; Li, C.L.; Chen, Y.; Wang, W.P.; Ding, H. Prediction of cervical lymph node metastasis with contrast-enhanced ultrasound and association between presence of BRAF(V600E) and extrathyroidal extension in papillary thyroid carcinoma. Ther. Adv. Med. Oncol. 2020, 12, 1758835920942367. [Google Scholar] [CrossRef] [PubMed]
  13. Lambin, P.; Leijenaar, R.T.H.; Deist, T.M.; Peerlings, J.; de Jong, E.E.C.; van Timmeren, J.; Sanduleanu, S.; Larue, R.; Even, A.J.G.; Jochems, A.; et al. Radiomics: The bridge between medical imaging and personalized medicine. Nat. Rev. Clin. Oncol. 2017, 14, 749–762. [Google Scholar] [CrossRef] [Green Version]
  14. Li, X.R.; Jin, J.J.; Yu, Y.; Wang, X.H.; Guo, Y.; Sun, H.Z. PET-CT radiomics by integrating primary tumor and peritumoral areas predicts E-cadherin expression and correlates with pelvic lymph node metastasis in early-stage cervical cancer. Eur. Radiol. 2021, 31, 5967–5979. [Google Scholar] [CrossRef]
  15. Shi, Z.; Ma, C.; Huang, X.; Cao, D. Magnetic Resonance Imaging Radiomics-Based Nomogram From Primary Tumor for Pretreatment Prediction of Peripancreatic Lymph Node Metastasis in Pancreatic Ductal Adenocarcinoma: A Multicenter Study. J. Magn. Reson. Imaging JMRI 2022, 55, 823–839. [Google Scholar] [CrossRef] [PubMed]
  16. Zhao, X.; Li, W.; Zhang, J.; Tian, S.; Zhou, Y.; Xu, X.; Hu, H.; Lei, D.; Wu, F. Radiomics analysis of CT imaging improves preoperative prediction of cervical lymph node metastasis in laryngeal squamous cell carcinoma. Eur. Radiol. 2022, 33, 1121–1131. [Google Scholar] [CrossRef]
  17. Tong, Y.; Li, J.; Huang, Y.; Zhou, J.; Liu, T.; Guo, Y.; Yu, J.; Zhou, S.; Wang, Y.; Chang, C. Ultrasound-Based Radiomic Nomogram for Predicting Lateral Cervical Lymph Node Metastasis in Papillary Thyroid Carcinoma. Acad. Radiol. 2021, 28, 1675–1684. [Google Scholar] [CrossRef]
  18. Zhou, S.C.; Liu, T.T.; Zhou, J.; Huang, Y.X.; Guo, Y.; Yu, J.H.; Wang, Y.Y.; Chang, C. An Ultrasound Radiomics Nomogram for Preoperative Prediction of Central Neck Lymph Node Metastasis in Papillary Thyroid Carcinoma. Front. Oncol. 2020, 10, 1591. [Google Scholar] [CrossRef]
  19. Jiang, M.; Li, C.; Tang, S.; Lv, W.; Yi, A.; Wang, B.; Yu, S.; Cui, X.; Dietrich, C.F. Nomogram Based on Shear-Wave Elastography Radiomics Can Improve Preoperative Cervical Lymph Node Staging for Papillary Thyroid Carcinoma. Thyroid 2020, 30, 885–897. [Google Scholar] [CrossRef] [PubMed]
  20. Kent, M.S.; Mandrekar, S.J.; Landreneau, R.; Nichols, F.; Foster, N.R.; DiPetrillo, T.A.; Meyers, B.; Heron, D.E.; Jones, D.R.; Tan, A.D.; et al. A Nomogram to Predict Recurrence and Survival of High-Risk Patients Undergoing Sublobar Resection for Lung Cancer: An Analysis of a Multicenter Prospective Study (ACOSOG Z4032). Ann. Thorac. Surg. 2016, 102, 239–246. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  21. Chen, L.T.; Macarulla, T.; Blanc, J.F.; Mirakhur, B.; Jong, F.A.; Belanger, B.; Bekaii-Saab, T.; Siveke, J.T. Nomogram for Predicting Survival in Patients Treated with Liposomal Irinotecan Plus Fluorouracil and Leucovorin in Metastatic Pancreatic Cancer. Cancers 2019, 11, 1068. [Google Scholar] [CrossRef] [Green Version]
  22. Iasonos, A.; Schrag, D.; Raj, G.V.; Panageas, K.S. How to build and interpret a nomogram for cancer prognosis. J. Clin. Oncol. 2008, 26, 1364–1370. [Google Scholar] [CrossRef] [PubMed]
  23. Haugen, B.R.; Alexander, E.K.; Bible, K.C.; Doherty, G.M.; Mandel, S.J.; Nikiforov, Y.E.; Pacini, F.; Randolph, G.W.; Sawka, A.M.; Schlumberger, M.; et al. 2015 American Thyroid Association Management Guidelines for Adult Patients with Thyroid Nodules and Differentiated Thyroid Cancer: The American Thyroid Association Guidelines Task Force on Thyroid Nodules and Differentiated Thyroid Cancer. Thyroid 2016, 26, 1–133. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  24. Gambardella, C.; Offi, C.; Romano, R.M.; De Palma, M.; Ruggiero, R.; Candela, G.; Puziello, A.; Docimo, L.; Grasso, M.; Docimo, G. Transcutaneous laryngeal ultrasonography: A reliable, non-invasive and inexpensive preoperative method in the evaluation of vocal cords motility-a prospective multicentric analysis on a large series and a literature review. Updat. Surg. 2020, 72, 885–892. [Google Scholar] [CrossRef] [PubMed]
  25. Canu, G.L.; Medas, F.; Cappellacci, F.; Giordano, A.B.F.; Gurrado, A.; Gambardella, C.; Docimo, G.; Feroci, F.; Conzo, G.; Testini, M.; et al. Risk of Complications in Patients Undergoing Completion Thyroidectomy after Hemithyroidectomy for Thyroid Nodule with Indeterminate Cytology: An Italian Multicentre Retrospective Study. Cancers 2022, 14, 2472. [Google Scholar] [CrossRef]
  26. Marotta, V.; Sciammarella, C.; Capasso, M.; Testori, A.; Pivonello, C.; Chiofalo, M.G.; Gambardella, C.; Grasso, M.; Antonino, A.; Annunziata, A.; et al. Germline Polymorphisms of the VEGF Pathway Predict Recurrence in Nonadvanced Differentiated Thyroid Cancer. J. Clin. Endocrinol. Metab. 2017, 102, 661–671. [Google Scholar] [CrossRef] [Green Version]
  27. Zhang, K.; Qian, L.; Chen, J.; Zhu, Q.; Chang, C. Preoperative Prediction of Central Cervical Lymph Node Metastasis in Fine-Needle Aspiration Reporting Suspicious Papillary Thyroid Cancer or Papillary Thyroid Cancer Without Lateral Neck Metastasis. Front. Oncol. 2022, 12, 712723. [Google Scholar] [CrossRef] [PubMed]
  28. Yan, M.; Wang, W. A Non-invasive Method to Diagnose Lung Adenocarcinoma. Front. Oncol. 2020, 10, 602. [Google Scholar] [CrossRef] [PubMed]
  29. Starkov, P.; Aguilera, T.A.; Golden, D.I.; Shultz, D.B.; Trakul, N.; Maxim, P.G.; Le, Q.T.; Loo, B.W.; Diehn, M.; Depeursinge, A.; et al. The use of texture-based radiomics CT analysis to predict outcomes in early-stage non-small cell lung cancer treated with stereotactic ablative radiotherapy. Br. J. Radiol. 2019, 92, 20180228. [Google Scholar] [CrossRef] [PubMed]
  30. Lambin, P.; Rios-Velazquez, E.; Leijenaar, R.; Carvalho, S.; van Stiphout, R.G.; Granton, P.; Zegers, C.M.; Gillies, R.; Boellard, R.; Dekker, A.; et al. Radiomics: Extracting more information from medical images using advanced feature analysis. Eur. J. Cancer 2012, 48, 441–446. [Google Scholar] [CrossRef] [Green Version]
  31. Liu, L.S.; Liang, J.; Li, J.H.; Liu, X.; Jiang, L.; Long, J.X.; Jiang, Y.M.; Wei, Z.X. The incidence and risk factors for central lymph node metastasis in cN0 papillary thyroid microcarcinoma: A meta-analysis. Eur. Arch. Oto-Rhino-Laryngol. 2017, 274, 1327–1338. [Google Scholar] [CrossRef]
  32. Huang, Y.; Huang, Z.; Cai, H.; Zhuge, L.; Wang, S.; Yan, D.; Zhang, X.; An, C.; Niu, L.; Li, Z. Evaluation of serum B7-H3 expression, ultrasound and clinical characteristics to predict the risk of cervical lymph node metastases in papillary thyroid carcinoma by nomogram. J. Clin. Lab. Anal. 2022, 37, e24811. [Google Scholar] [CrossRef] [PubMed]
  33. Feng, Y.; Min, Y.; Chen, H.; Xiang, K.; Wang, X.; Yin, G. Construction and validation of a nomogram for predicting cervical lymph node metastasis in classic papillary thyroid carcinoma. J. Endocrinol. Investig. 2021, 44, 2203–2211. [Google Scholar] [CrossRef]
  34. Mao, J.; Zhang, Q.; Zhang, H.; Zheng, K.; Wang, R.; Wang, G. Risk Factors for Lymph Node Metastasis in Papillary Thyroid Carcinoma: A Systematic Review and Meta-Analysis. Front. Endocrinol. 2020, 11, 265. [Google Scholar] [CrossRef]
  35. Suteau, V.; Munier, M.; Briet, C.; Rodien, P. Sex Bias in Differentiated Thyroid Cancer. Int. J. Mol. Sci. 2021, 22, 12992. [Google Scholar] [CrossRef] [PubMed]
  36. Xing, Z.; Qiu, Y.; Yang, Q.; Yu, Y.; Liu, J.; Fei, Y.; Su, A.; Zhu, J. Thyroid cancer neck lymph nodes metastasis: Meta-analysis of US and CT diagnosis. Eur. J. Radiol. 2020, 129, 109103. [Google Scholar] [CrossRef]
  37. Goncalves, M.; Gsaxner, C.; Ferreira, A.; Li, J.; Puladi, B.; Kleesiek, J.; Egger, J.; Alves, V. Radiomics in Head and Neck Cancer Outcome Predictions. Diagnostics 2022, 12, 2733. [Google Scholar] [CrossRef] [PubMed]
  38. Li, Q.; Jiang, T.; Zhang, C.; Zhang, Y.; Huang, Z.; Zhou, H.; Huang, P. A nomogram based on clinical information, conventional ultrasound and radiomics improves prediction of malignant parotid gland lesions. Cancer Lett. 2022, 527, 107–114. [Google Scholar] [CrossRef] [PubMed]
  39. Ma, Q.; Wang, J.; Xu, D.; Zhu, C.; Qin, J.; Wu, Y.; Gao, Y.; Zhang, C. Automatic Breast Volume Scanner and B-Ultrasound-Based Radiomics Nomogram for Clinician Management of BI-RADS 4A Lesions. Acad. Radiol. 2022. [Google Scholar] [CrossRef]
  40. Jiang, M.; Li, C.L.; Luo, X.M.; Chuan, Z.R.; Chen, R.X.; Tang, S.C.; Lv, W.Z.; Cui, X.W.; Dietrich, C.F. Radiomics model based on shear-wave elastography in the assessment of axillary lymph node status in early-stage breast cancer. Eur. Radiol. 2022, 32, 2313–2325. [Google Scholar] [CrossRef]
  41. Habibollahi, P.; Sultan, L.R.; Bialo, D.; Nazif, A.; Faizi, N.A.; Sehgal, C.M.; Chauhan, A. Hyperechoic Renal Masses: Differentiation of Angiomyolipomas from Renal Cell Carcinomas using Tumor Size and Ultrasound Radiomics. Ultrasound. Med. Biol. 2022, 48, 887–894. [Google Scholar] [CrossRef]
  42. Agyekum, E.A.; Ren, Y.Z.; Wang, X.; Cranston, S.S.; Wang, Y.G.; Wang, J.; Akortia, D.; Xu, F.J.; Gomashie, L.; Zhang, Q.; et al. Evaluation of Cervical Lymph Node Metastasis in Papillary Thyroid Carcinoma Using Clinical-Ultrasound Radiomic Machine Learning-Based Model. Cancers 2022, 14, 5266. [Google Scholar] [CrossRef] [PubMed]
  43. Liu, T.; Ge, X.; Yu, J.; Guo, Y.; Wang, Y.; Wang, W.; Cui, L. Comparison of the application of B-mode and strain elastography ultrasound in the estimation of lymph node metastasis of papillary thyroid carcinoma based on a radiomics approach. Int. J. Comput. Assist. Radiol. Surg 2018, 13, 1617–1627. [Google Scholar] [CrossRef]
  44. Wang, Y.G.; Xu, F.J.; Agyekum, E.A.; Xiang, H.; Wang, Y.D.; Zhang, J.; Sun, H.; Zhang, G.L.; Bo, X.S.; Lv, W.Z.; et al. Radiomic Model for Determining the Value of Elasticity and Grayscale Ultrasound Diagnoses for Predicting BRAF(V600E) Mutations in Papillary Thyroid Carcinoma. Front. Endocrinol. 2022, 13, 872153. [Google Scholar] [CrossRef]
  45. Guo, S.Y.; Zhou, P.; Zhang, Y.; Jiang, L.Q.; Zhao, Y.F. Exploring the Value of Radiomics Features Based on B-Mode and Contrast-Enhanced Ultrasound in Discriminating the Nature of Thyroid Nodules. Front. Oncol. 2021, 11, 738909. [Google Scholar] [CrossRef] [PubMed]
  46. Hong, Y.R.; Yan, C.X.; Mo, G.Q.; Luo, Z.Y.; Zhang, Y.; Wang, Y.; Huang, P.T. Conventional US, elastography, and contrast enhanced US features of papillary thyroid microcarcinoma predict central compartment lymph node metastases. Sci. Rep. 2015, 5, 7748. [Google Scholar] [CrossRef] [Green Version]
  47. Park, H.S.; Lee, K.S.; Seo, B.K.; Kim, E.S.; Cho, K.R.; Woo, O.H.; Song, S.E.; Lee, J.Y.; Cha, J. Machine Learning Models That Integrate Tumor Texture and Perfusion Characteristics Using Low-Dose Breast Computed Tomography Are Promising for Predicting Histological Biomarkers and Treatment Failure in Breast Cancer Patients. Cancers 2021, 13, 6013. [Google Scholar] [CrossRef]
  48. Zhan, W.W.; Zhou, P.; Zhou, J.Q.; Xu, S.Y.; Chen, K.M. Differences in sonographic features of papillary thyroid carcinoma between neck lymph node metastatic and nonmetastatic groups. J. Ultrasound Med. 2012, 31, 915–920. [Google Scholar] [CrossRef] [Green Version]
  49. Ng, F.; Ganeshan, B.; Kozarski, R.; Miles, K.A.; Goh, V. Assessment of primary colorectal cancer heterogeneity by using whole-tumor texture analysis: Contrast-enhanced CT texture as a biomarker of 5-year survival. Radiology 2013, 266, 177–184. [Google Scholar] [CrossRef] [Green Version]
  50. Zhou, Y.; Zhou, G.; Zhang, J.; Xu, C.; Zhu, F.; Xu, P. DCE-MRI based radiomics nomogram for preoperatively differentiating combined hepatocellular-cholangiocarcinoma from mass-forming intrahepatic cholangiocarcinoma. Eur. Radiol. 2022, 32, 5004–5015. [Google Scholar] [CrossRef] [PubMed]
  51. Xie, C.; Yang, P.; Zhang, X.; Xu, L.; Wang, X.; Li, X.; Zhang, L.; Xie, R.; Yang, L.; Jing, Z.; et al. Sub-region based radiomics analysis for survival prediction in oesophageal tumours treated by definitive concurrent chemoradiotherapy. EBioMedicine 2019, 44, 289–297. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  52. Jiang, Z.; Yin, J.; Han, P.; Chen, N.; Kang, Q.; Qiu, Y.; Li, Y.; Lao, Q.; Sun, M.; Yang, D.; et al. Wavelet transformation can enhance computed tomography texture features: A multicenter radiomics study for grade assessment of COVID-19 pulmonary lesions. Quant. Imaging Med. Surg. 2022, 12, 4758–4770. [Google Scholar] [CrossRef] [PubMed]
Cancers 15 01613 g001 550
Figure 1. (a) Study flowchart of clinical-radiomics nomogram modeling for the LNM prediction in patients with PTC; (b) Radiomics workflow. BMUS, B-mode ultrasound; CEUS, contrast-enhanced ultrasound; ICC, interclass correlation coefficient; mRMR, minimum redundancy maximum relevance; LASSO, least absolute shrinkage and selection operator; AIC, Akaike information criterion; Radscore, radiomics score.
Figure 1. (a) Study flowchart of clinical-radiomics nomogram modeling for the LNM prediction in patients with PTC; (b) Radiomics workflow. BMUS, B-mode ultrasound; CEUS, contrast-enhanced ultrasound; ICC, interclass correlation coefficient; mRMR, minimum redundancy maximum relevance; LASSO, least absolute shrinkage and selection operator; AIC, Akaike information criterion; Radscore, radiomics score.
Cancers 15 01613 g001
Cancers 15 01613 g002 550
Figure 2. Selection of radiomics features using the least absolute shrinkage and selection operator (LASSO) algorithm in the training set. (a,c) LASSO coefficient profiles of the BMUS (a) and CEUS (c) features. (b,d) The 10-fold cross-validation and the minimum criteria process were used to generate the optimal penalization coefficient lambda (λ) in the BMUS and CEUS LASSO models. Dotted vertical lines are drawn by using the minimum criteria and 1 standard error of the minimum criteria. As a result, λ values of 0.08255607 and 0.05676918 were selected for the BMUS (b) and CEUS (d) features, respectively.
Figure 2. Selection of radiomics features using the least absolute shrinkage and selection operator (LASSO) algorithm in the training set. (a,c) LASSO coefficient profiles of the BMUS (a) and CEUS (c) features. (b,d) The 10-fold cross-validation and the minimum criteria process were used to generate the optimal penalization coefficient lambda (λ) in the BMUS and CEUS LASSO models. Dotted vertical lines are drawn by using the minimum criteria and 1 standard error of the minimum criteria. As a result, λ values of 0.08255607 and 0.05676918 were selected for the BMUS (b) and CEUS (d) features, respectively.
Cancers 15 01613 g002
Cancers 15 01613 g003 550
Figure 3. The ROC curves of the clinical model and clinical-radiomics model. ROC, receiver operating characteristic. (a) The training set, (b) the validation set.
Figure 3. The ROC curves of the clinical model and clinical-radiomics model. ROC, receiver operating characteristic. (a) The training set, (b) the validation set.
Cancers 15 01613 g003
Cancers 15 01613 g004 550
Figure 4. Developed radiomic nomogram based on the clinical-radiomics model for the assessment of cervical LNM in PTC patients (a). Calibration curves of the clinical-radiomics nomogram in the training set (b) and validation set (c).
Figure 4. Developed radiomic nomogram based on the clinical-radiomics model for the assessment of cervical LNM in PTC patients (a). Calibration curves of the clinical-radiomics nomogram in the training set (b) and validation set (c).
Cancers 15 01613 g004
Cancers 15 01613 g005 550
Figure 5. Decision curve of clinical model and clinical-radiomics model. The decision curve analysis (DCA) measures the net benefit (y-axis) versus the model’s high-risk threshold (x-axis) for different models. The black horizontal line manifests no patients with LNM (none) and the grey line manifests all patients with LNM (all). The decision curves indicate that the net benefit of the clinical-radiomics nomogram was higher than the clinical model across the majority of the range of reasonable risk thresholds.
Figure 5. Decision curve of clinical model and clinical-radiomics model. The decision curve analysis (DCA) measures the net benefit (y-axis) versus the model’s high-risk threshold (x-axis) for different models. The black horizontal line manifests no patients with LNM (none) and the grey line manifests all patients with LNM (all). The decision curves indicate that the net benefit of the clinical-radiomics nomogram was higher than the clinical model across the majority of the range of reasonable risk thresholds.
Cancers 15 01613 g005
Table
Table 1. Baseline patient characteristics in training and validation sets.
Table 1. Baseline patient characteristics in training and validation sets.
CharacteristicTraining Set (n = 148)Validation Set (n = 63)p-Value
Lymph node metastasis 0.406
Negative89 (60.1)34 (54.0)
Positive 59 (39.9)29 (46.0)
Age 0.531
<55 years125 (84.5)51 (81.0)
≥55 years23 (15.5)12 (19.0)
Gender 0.495
Female111 (75.0)50 (79.4)
Male37 (25.0)13 (20.6)
Primary site 0.299
Left lobe61 (41.2)33 (52.4)
Right lobe78 (52.7)26 (41.3)
Isthmus9 (6.1)4 (6.3)
Tumor location 0.980
Intra-thyroidal35 (23.6)15 (23.8)
Sub-capsular113 (76.4)48 (76.2)
Tumor size 0.106
≤10 mm106 (71.6)38 (60.3)
>10 mm42 (28.4)25 (39.7)
Echogenicity 0.583
iso/hyperechoic7 (4.7)3 (4.8)
hypoechoic58 (39.2)20 (31.7)
marked hypoechoic83 (56.1)40 (63.5)
Aspect ratio > 1 0.757
Absent93 (62.8)41 (65.1)
Present55 (37.2)22 (34.9)
Margin 0.579
Smooth7 (4.7)3 (4.8)
Ill-defined12 (8.1)8 (12.7)
Irregular129 (87.2)52 (82.5)
Microcalcification 0.571
Absent39 (26.4)19 (30.2)
Present109 (73.6)44 (69.8)
Enhancement pattern 0.329
Hyper-enhancement7 (4.7)1 (1.6)
Iso-enhancement34 (23.0)11 (17.5)
Hypo-enhancement107 (72.3)51 (81.0)
US-reported LN status 0.062
Negative130 (87.8)49 (77.8)
Positive18 (12.2)14 (22.2)
BMUS Radscore, 0.662
Median (Interquartile range)−0.40 (−0.71, −0.07)−0.32 (−0.84, 0.10)
CEUS Radscore, 0.185
Median (Interquartile range)−0.54 (−1.20, 0.29)−0.37 (−0.87, 0.36)
Table
Table 2. Associations between the lymph node metastasis and patient characteristics in the training and validation sets.
Table 2. Associations between the lymph node metastasis and patient characteristics in the training and validation sets.
CharacteristicTraining SetValidation Set
LNM−LNM+p-ValueLNM−LNM+p-Value
Age 0.017 0.023
<55 years70 (78.7)55 (93.2) 24 (70.6)27 (93.1)
≥55 years19 (21.3)4 (6.8) 10 (29.4)2 (6.9)
Gender 0.042 0.060
Female72 (80.9)39 (66.1) 30 (88.2)20 (69.0)
Male17 (19.1)20 (33.9) 4 (11.8)9 (31.0)
Primary site 0.642 0.401
Left lobe34 (38.2)27 (45.8) 17 (50.0)16 (55.2)
Right lobe49 (55.1)29 (49.2) 16 (47.1)10 (34.5)
Isthmus6 (6.7)3 (5.1) 1 (2.9)3 (10.3)
Tumor location 0.118 0.020
Intra-thyroidal25 (28.1)10 (16.9) 12 (35.3)3 (10.3)
Sub-capsular64 (71.9)49 (83.1) 22 (64.7)26 (89.7)
Tumor size 0.002 <0.001
>10 mm72 (80.9)34 (57.6) 27 (79.4)11 (37.9)
≤10 mm17 (19.1)25 (42.4) 7 (20.6)18 (62.1)
Echogenicity 0.409 0.497
iso/hyperechoic5 (5.6)2 (3.4) 1 (2.9)2 (6.9)
hypoechoic31 (34.8)27 (45.8) 13 (38.2)7 (24.1)
marked hypoechoic53 (59.6)30 (50.8) 20 (58.8)20 (69.0)
Aspect ratio > 1 0.309 0.029
Absent53 (59.6)40 (67.8) 18 (52.9)23 (79.3)
Present36 (40.4)19 (32.2) 16 (47.1)6 (20.7)
Margin 1.000 0.146
Smooth4 (4.5)3 (5.1) 1 (2.9)2 (6.9)
Ill-defined7 (7.9)5 (8.5) 2 (5.9)6 (20.7)
Irregular78 (87.6)51 (86.4) 31 (91.2)21 (72.4)
Microcalcification 0.083 0.336
Absent28 (31.5)11 (18.6) 12 (35.3)7 (24.1)
Present61 (68.5)48 (81.4) 22 (64.7)22 (75.9)
Enhancement pattern 0.155 0.860
Hyper-enhancement2 (2.2)5 (8.5) 0 (0.0)1 (3.4)
Iso-enhancement23 (25.8)11 (18.6) 6 (17.6)5 (17.2)
Hypo-enhancement64 (71.9)43 (72.9) 28 (82.4)23 (79.3)
US-reported LN status <0.001 0.031
Negative85 (95.5)45 (76.3) 30 (88.2)19 (65.5)
Positive4 (4.5)14 (23.7) 4 (11.8)10 (34.5)
BMUS Radscore 0.001 0.004
Median (Interquartile range)−0.51 (−0.85, −0.21)−0.25 (−0.50, 0.04) −0.53 (−1.00, −0.16)−0.02 (−0.52, 0.28)
CEUS Radscore <0.001 0.002
Median (Interquartile range)−0.89 (−1.71, −0.28)0.12 (−0.54, 0.66) −0.66 (−1.18, −0.07)0.10 (−0.37, 0.75)
Table
Table 3. Clinical model and clinical-radiomics model based on stepwise multivariate analyses for prediction of LNM.
Table 3. Clinical model and clinical-radiomics model based on stepwise multivariate analyses for prediction of LNM.
CharacteristicsOdds Ratio (95%CI)p-Value
Clinical model
Gender (male vs. female)2.18 (0.95, 5.00) 0.067
Age (≥55 years vs. <55 years)0.30 (0.09, 0.96) 0.042
Tumor size (>10 mm vs. ≤10 mm)2.22 (1.00, 4.95) 0.051
US-reported LN status (positive vs. negative)4.86 (1.40, 16.83) 0.013
Clinical-radiomics model
Gender (male vs. female)2.22 (0.86, 5.74) 0.100
Age (≥55 years vs. <55 years)0.18 (0.05, 0.70) 0.013
US-reported LN status (positive vs. negative)5.16 (1.40, 18.98) 0.014
CEUS Radscore2.75 (1.79, 4.23) <0.001
Abbreviations: CI, confidence interval.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Jiang, L.; Zhang, Z.; Guo, S.; Zhao, Y.; Zhou, P. Clinical-Radiomics Nomogram Based on Contrast-Enhanced Ultrasound for Preoperative Prediction of Cervical Lymph Node Metastasis in Papillary Thyroid Carcinoma. Cancers 2023, 15, 1613. https://doi.org/10.3390/cancers15051613

AMA Style

Jiang L, Zhang Z, Guo S, Zhao Y, Zhou P. Clinical-Radiomics Nomogram Based on Contrast-Enhanced Ultrasound for Preoperative Prediction of Cervical Lymph Node Metastasis in Papillary Thyroid Carcinoma. Cancers. 2023; 15(5):1613. https://doi.org/10.3390/cancers15051613

Chicago/Turabian Style

Jiang, Liqing, Zijian Zhang, Shiyan Guo, Yongfeng Zhao, and Ping Zhou. 2023. "Clinical-Radiomics Nomogram Based on Contrast-Enhanced Ultrasound for Preoperative Prediction of Cervical Lymph Node Metastasis in Papillary Thyroid Carcinoma" Cancers 15, no. 5: 1613. https://doi.org/10.3390/cancers15051613

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop