Next Article in Journal
A Comprehensive Survey on the Detection, Classification, and Challenges of Neurological Disorders
Previous Article in Journal
In-Season Quantification and Relationship of External and Internal Intensity, Sleep Quality, and Psychological or Physical Stressors of Semi-Professional Soccer Players
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Radiomics Metrics Combined with Clinical Data in the Surgical Management of Early-Stage (cT1–T2 N0) Tongue Squamous Cell Carcinomas: A Preliminary Study

by
Umberto Committeri
1,
Roberta Fusco
2,
Elio Di Bernardo
2,
Vincenzo Abbate
1,
Giovanni Salzano
1,
Fabio Maglitto
1,
Giovanni Dell’Aversana Orabona
1,
Pasquale Piombino
1,
Paola Bonavolontà
1,
Antonio Arena
1,
Francesco Perri
3,
Maria Grazia Maglione
4,
Sergio Venanzio Setola
5,
Vincenza Granata
5,
Giorgio Iaconetta
6,
Franco Ionna
4,
Antonella Petrillo
5,*,† and
Luigi Califano
1,†
1
Maxillofacial Surgery Operative Unit, Department of Neurosciences, Reproductive and Odontostomatological Sciences, Federico II University of Naples, 80131 Naples, Italy
2
Medical Oncology Division, Igea SpA, 80013 Naples, Italy
3
Head and Neck Medical Oncology Unit, Istituto Nazionale dei Tumori IRCCS Fondazione G. Pascale, 80131 Naples, Italy
4
Division of Surgical Oncology Maxillo-Facial Unit, Istituto Nazionale Tumori-IRCCS-Fondazione G. Pascale, Via Mariano Semmola, 80131 Naples, Italy
5
Divisions of Radiology, Istituto Nazionale dei Tumori IRCCS Fondazione G. Pascale, 80131 Naples, Italy
6
Department of Neurosurgery, University of Salerno, 84084 Salerno, Italy
*
Author to whom correspondence should be addressed.
Co-last authors.
Biology 2022, 11(3), 468; https://doi.org/10.3390/biology11030468
Submission received: 16 February 2022 / Revised: 8 March 2022 / Accepted: 16 March 2022 / Published: 18 March 2022
(This article belongs to the Section Cancer Biology)

Abstract

:

Simple Summary

The purpose of this study is to predict the risk of metastatic lymph nodes related to oral tongue squamous cell carcinoma (OTSCC) and tumor grading through the combination of clinical data with radiomics metrics extracted from computed tomography (CT) images. We aimed to develop a supportive approach in the management of lymphatic cervical areas, with particular attention given to the early stages (T1−T2). Specifically, we evaluated the effectiveness of the radiomics and clinical features in the study of OTSCC and in the prediction of occult laterocervical metastatic lymph nodes. We concluded that radiomics features and clinical parameters have an important role in identifying tumor grading and metastatic lymph nodes. Machine learning approaches can be used as an easy-to-use tool to stratify patients with early-stage OTSCC, based on the identification of metastatic and non-metastatic lymph nodes

Abstract

Objective: To predict the risk of metastatic lymph nodes and the tumor grading related to oral tongue squamous cell carcinoma (OTSCC) through the combination of clinical data with radiomics metrics by computed tomography, and to develop a supportive approach in the management of the lymphatic cervical areas, with particular attention to the early stages (T1−T2). Between March 2016 and February 2020, patients with histologically confirmed OTSCC, treated by partial glossectomy and ipsilateral laterocervical lymphadenectomy and subjected to computed tomography (CT) before surgery, were identified by two centers: 81 patients (49 female and 32 male) with 58 years as the median age (range 19–86 years). Univariate analysis with non-parametric tests and multivariate analysis with machine learning approaches were used. Clinical, hematological parameters and radiological features extracted by CT were considered individually and in combination. All clinical parameters showed statistically significant differences (p < 0.05) for the Kruskal−Wallis test when discriminating both the tumor grading and the metastatic lymph nodes. DOI, PLR, SII, and SIRI showed an accuracy of 0.70 (ROC analysis) when identifying the tumor grading, while an accuracy ≥ 0.78 was shown by DOI, NLR, PLR, SII, and SIRI when discriminating metastatic lymph nodes. In the context of the analysis of radiomics metrics, the original_glszm_HighGrayLevelZoneEmphasis feature was selected for identifying the tumor grading (accuracy of 0.70), while the wavelet_HHH_glrlm_LowGrayLevelRunEmphasis predictor was selected for determining metastatic lymph nodes (accuracy of 0.96). Remarkable findings were also obtained when classifying patients with a machine learning approach. Radiomics features alone can predict tumor grading with an accuracy of 0.76 using a logistic regression model, while an accuracy of 0.82 can be obtained by running a CART algorithm through a combination of three clinical parameters (SIRI, DOI, and PLR) with a radiomics feature (wavelet_LLL_glszm_SizeZoneNonUniformityNormalized). In the context of predicting metastatic lymph nodes, an accuracy of 0.94 was obtained using 15 radiomics features in a logistic regression model, while both CART and CIDT achieved an asymptotic accuracy value of 1.00 using only one radiomics feature. Radiomics features and clinical parameters have an important role in identifying tumor grading and metastatic lymph nodes. Machine learning approaches can be used as an easy-to-use tool to stratify patients with early-stage OTSCC, based on the identification of metastatic and non-metastatic lymph nodes.

1. Introduction

Oral tongue squamous cell carcinoma (OTSCC) represents an aggressive tumor associated with a poor prognosis due to the high risk of local and locoregional invasiveness [1].
Tongue drainage involves nodes of I, II, and partially III level of the lymphatic system of the neck. About 35% of patients in stage I–II of oral cancer have a rather high risk of developing occult cervical metastases, with a significant reduction of survival rate of 50% [2].
The management of the neck lymphatic stations in T1−T2 CN0 OTSCC, indeed, represents a dilemma and the surgical decision is often based on individual clinical experience. Nowadays, the surgeon can opt for several therapeutic alternatives. In patients with early-stage tumors, a watchful waiting approach can be adopted, otherwise elective neck dissection or biopsy of sentinel lymph node have to be considered. In this regard, it is important to note that elective neck dissection is not free from complications (rate close to 29%); this approach, in fact, is burdened by greater morbidity, given by the second surgical site. In 70–80% of cases, this option can represent overtreatment [3]. Nevertheless, the sentinel lymph node biopsy has a sensitivity that varies in the literature between 75−100%, for identifying the first draining lymph node in squamous carcinomas of the oral cavity; there is also a not negligible percentage of false negatives of about 14% [4].
The eighth edition of the American Joint Commission on Cancer (AJCC) provides, as a novelty in the TNM classification of oral cavity carcinomas, the depth of invasion (DOI), which evaluates the primary tumor (T), drawing a perpendicular line passing through the basement membrane of the normal epithelium, contiguous to the neoplasm, up to the deepest tumor invasion point. It seems with a DOI cut-off between 3 and 4 mm there is a 20% risk of lymphatic micro metastases [5]. This condition could be a possible recommendation for elective neck levels I-I-II, which overrides the extension to IV if the tumor has a posterior origin.
Furthermore, the state of inflammation also has a relationship with cancerogenesis. Rudolf Virchow described for the first time a possible association between the state of inflammation and cancer [6]. However, only in the last few decades has a clear relationship between inflammation and carcinogenesis been established [6], thanks to the possibility of detecting the systemic inflammatory state through blood samples, namely through the increase in the neutrophil, monocyte, or platelet count.
In addition, an association between the inflammation indices and the possible decreased survival in different tumor models was found, including patients with squamous cell carcinomas of the head−neck area. On the other hand, the presence of lymphopenia reflects a possible reduction in immune response with an aggravation of the prognosis.
Recently, all these hematological parameters, indicative of a state inflammatory and/or immune failure, have been combined for obtaining new prognostic biomarkers. Among these, the best known are the neutrophils−lymphocytes ratio (NLR); the platelet−lymphocyte ratio (PLR); the lymphocytes−monocytes ratio (LMR); the Systemic Inflammation Response Index (SIRI), defined as neutrophil count × monocyte/lymphocyte count; and the Systemic Immune-inflammation Index (SII), calculated by the formula neutrophil × platelet/lymphocyte [7].
Biomarkers can be derived from a wide variety of diagnostic sources, such as from the biopsy of tissue specimens, from cell collections, and from radiological images. In the context of cancer research, particular attention has been increasingly paid to the quantitative analysis of high-performance images through radiomics. Radiomics extracts quantitative features from medical imaging with the aim of obtaining descriptors of the image contents. Moreover, these features can be combined with other characteristics of the patient, when available, for increasing the power of the decision support models. This can be performed with the appropriate use of techniques from machine learning and artificial intelligence areas [8].
The purpose of this study is to predict the risk of metastatic lymph nodes related to OTSCC and tumor grading through the combination of clinical data with radiomics metrics extracted from computed tomography (CT) images. By means of statistical analysis, we aimed to develop a supportive approach in the management of the lymphatic cervical areas, with particular attention to the early stages (T1−T2). Specifically, we evaluated the effectiveness of the radiomics and clinical features in the study of OTSCC and in the prediction of occult laterocervical metastatic lymph nodes. To this aim, the features were both considered individually and combined with each other.

2. Materials and Methods

2.1. Patient Selection

In the context of a multicenter retrospective study, 81 patients were recruited by the Maxillofacial Surgery of the Polyclinic Federico II University of Naples and Otolaryngology and the Surgery Maxillofacial of the National Cancer Institute IRCCS G. Pascale of Naples.
The study was performed in accordance with relevant guidelines and regulations; informed consent was not required by the Local Ethics Committee due to the retrospective nature of the study.
The analysis was conducted between March 2016 and February 2020. The patients (49 female and 32 male) had a median age of 58 years (range 19–86 years) and had received a diagnosis of OTSCC. The diagnosis was confirmed with histological analysis and all patients underwent CT before surgery (partial glossectomy or ipsilateral laterocervical lymphadenectomy).
Inclusion criteria were proven histology of OTSCC with a preoperative incisional biopsy, early stage cT1−T2 OTSCC, non-primary OTSCC previously treated, availability of clinical and hematological parameters and a minimum of 12 months of post-operative clinical follow-up. Exclusion criteria were radiotherapy or chemotherapy in the clinical history, previous tumor in other sites, advanced squamous cell carcinoma, chronic inflammatory pathologies capable of influencing the indices of inflammation (infections, serum virus markers, chronic or autoimmune inflammatory diseases, haematological disorders, and simultaneous or long-term anti-inflammatory or steroidal drug treatments).
Clinical and hematological parameters included: age; gender; DOI, NLR; PLR; LMR; SIRI; SII; T stage; grading; metastatic lymph nodes; perineural infiltration; vascular infiltration.

2.2. CT Protocol

A tomograph with 64 detectors (Optima 660, General Electric Healthcare, Milwaukee, WI, USA) was used to acquire the CT images.
The image acquisition was performed at 120–140 kVp, 200–600 mA, setting a slice thickness of 1.25–2.5 mm and a table speed/rotation of 0.938–0.984/1 mm. Head and neck contrast-enhanced CT images were acquired in the venous phase (start delay 70–80 s), after the intravenous injection of 2 mL/kg of a non-ionic contrast material (iodine concentration >350 mg/ml), followed by 40 mL of saline solution, using a semi-automated power injector (3.5–4 mL/s flow rate).

2.3. Image Processing

Regions of interest (ROIs) were manually drawn, slice-by-slice, by two expert radiologists with 22 and 15 years, respectively, of experience in head and neck imaging. The radiologists performed the selection of ROIs first separately and then jointly. Manual definition of the ROIs was made using the 3DSlicer segmentation tool (https://www.slicer.org/ accessed on 15 February 2021).
In the context of radiomics, 851 features were extracted using PyRadiomics [9], in compliance with feature definitions, as described by the Imaging Biomarker Standardization Initiative (IBSI) [10]. More precisely, two sets of 851 radiomics features were extracted from both the tumor region and the lymph node area. The radiomics features were subdivided into the following classes: First order statistics, shape-based (2D and 3D); gray level cooccurence matrix (24 features), gray level run length matrix (16 features), gray level size zone matrix, neighboring gray tone difference matrix, and gray level dependence matrix. Details about radiomics features can be found in the work of Kumarasamy [9].

2.4. Statistical Analysis

2.4.1. Univariate Analysis

The non-parametric Kruskal−Wallis test was performed to identify those features that were mostly involved in (individually) discriminating the following outcomes: G1 + G2 grading versus G3 grading, metastatic lymph nodes versus non-metastatic lymph nodes, presence or absence of perineural infiltration, and presence or absence of vascular infiltration. The receiver operating characteristic (ROC) analysis was also performed and the Youden index was used to identify the optimal cut-off value for each feature; the area under the ROC curve (AUC), sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy were then calculated to evaluate the performances.
The univariate analyses were performed using the Statistics Toolbox of MATLAB R2007a (MathWorks, Natick, MA, USA).

2.4.2. Multivariate Analysis

A multivariate analysis was performed in order to identify the combinations of features that best predicted the outcomes: G1 + G2 grading versus G3 grading, metastatic lymph nodes versus non-metastatic lymph nodes, presence or absence of perineural infiltration, and presence or absence of vascular infiltration.
Clinical parameters and radiomics features were considered both individually and in combination, and were extracted from both tumor and lymph node areas.
Given the high number of radiomics features, before proceeding with the analysis, a selection of variables was made on the basis of the results obtained from the univariate analysis, considering only the features that were significant in the Kruskal−Wallis test.
Three machine learning algorithms were appropriately designed and run on RStudio Version 1.3.959 (https://www.rstudio.com/ accessed on 15 February 2021), as described in the following. Here, 80% of the dataset was used to train the algorithms, while the remaining 20% was used to validate the models and compare the results. The dataset was split in a randomized way, using the createDataPartition R function.
Logistic Regression. Considering that all the outcomes were binary (i.e., only two values were allowed for those variables), logistic regression was executed and used as a baseline for the model comparison. Logistic regression was performed using the glm function on RStudio, with appropriate design of function attributes and data pre-processing (centering and scaling of input predictors).
Tree-based algorithms. Classification and Regression Trees (CART) and Conditional Inference Decision Trees (CIDT) were executed for classifying the outcome variables. The hyperparameters were tuned with a 10-fold cross validation, and were repeated 10 times. A decision chart was also obtained with the aim of supporting the surgeon in the decision process.

3. Results

3.1. Univariate Analysis

All clinical parameters showed statistically significant differences (p < 0.05) in the Kruskal−Wallis test on median values when discriminating both the tumor grading and the metastatic lymph nodes.
In the context of identifying the tumor grading, DOI, PLR, SII, and SIRI showed an accuracy of 0.70 and the Systemic Immune-inflammation Index achieved the highest AUC value (0.73), with appreciable values of specificity and sensitivity (Table 1). However, better results were achieved in the ROC analysis of clinical parameters when discriminating metastatic lymph nodes. In fact, DOI, NLR, PLR, SII, and SIRI showed an accuracy ≥ 0.78 (Table 1), with PLR (accuracy of 0.83) reaching an AUC of 0.76, with sensitivity and specificity values of 0.80 and 0.85, respectively.
Less remarkable results were found when grouping patients according to the presence/absence of both perineural and vascular infiltration. Five (DOI, PLR, LMR, SII, and SIRI) out of seven clinical parameters showed a statistically significant difference (p < 0.05) in the Kruskal−Wallis test of median values for evaluating the perineural infiltration, with both DOI and SII having an accuracy slightly above 0.70. DOI, NLR, SII, and SIRI were, instead, found to be statistically significant (Kruskal−Wallis test, p < 0.05) in discriminating the presence or the absence of vascular infiltration. In the latter case, the only variable that showed an accuracy of just 0.73 was NLR, with an AUC value of 0.64 (Table 1).
The same trend of results was also observed in the context of identifying the presence or absence of infiltration (both perineural and vascular) when performing the ROC analysis on radiomics features. As Table 2 shows, the best accuracy achievable was 0.68 with the wavelet_HHH_glcm_MaximumProbability feature (sensitivity of 0.69 and specificity of 0.67) related to perineural infiltration, while a slightly better result can be achieved (in term of accuracy) in the context of vascular infiltration; however, the sensitivity value was found to be significantly low (0.29). Even in the analysis conducted with the Kruskal−Wallis test, the results on both types of infiltrations were ae less noticeable; in fact, only 39 and 57 radiomics features were statistically significant for perineural and vascular infiltration, respectively.
Contrarily, the differentiation of tumor grading and metastatic lymph nodes gave more interesting results. Indeed, already in the case of the Kruskal−Wallis test, 158 out of 851 radiomics features were found to be statistically significant (p < 0.05) for discriminating G1 + G2 versus G3 grading, and 804 features showed an analogue significance (p < 0.05) when identifying metastatic from non-metastatic lymph nodes. Further remarkable results were also found in the context of the ROC analysis. The original_glszm_HighGrayLevelZoneEmphasis feature was selected (among the 158 significant features) given the good performance (accuracy of 0.70) achieved in identifying the tumor grading (Table 2). However, the most noteworthy results were reached when determining the metastatic lymph nodes; in fact, the wavelet_HHH_glrlm_LowGrayLevelRunEmphasis predictor was able to separate patients with metastatic lymph nodes from those with non-metastatic lymph nodes with an accuracy of 0.96 (sensitivity of 0.94, specificity of 0.98) and with a cut-off value equals to 0.39 (Table 2).

3.2. Multivariate Analysis

In the multivariate analysis, a machine learning approach was used to predict the tumor grading and the metastatic lymph nodes. The patients were classified with the algorithms and using both clinical and radiomics features, both individually and in combination. The results obtained are reported in Table 3 and Table 4, and the accuracies achieved with the logistic regression were used as a baseline.
Good performances were obtained when classifying patients according to tumor grading (G1 + G2 grading versus G3 grading). Specifically, radiomics features alone can predict tumor grading with an accuracy of 0.76 (sensitivity of 0.78, specificity of 0.75) and using a logistic regression model (Table 3). However, the predictive power of radiomics features can be further improved by combining them with clinical parameters. In fact, as shown in Table 3, the CART method achieved an accuracy of 0.82 (sensitivity of 0.78, specificity of 0.87) combining (left panel of Figure 1) three clinical parameters (SIRI, DOI, and PLR) with a radiomics feature extracted from tumor areas (wavelet_LLL_glszm_SizeZoneNonUniformityNormalized). This result was also confirmed by the CIDT method. In particular, despite the low accuracy value (0.65), it is possible to note that even CIDT (top panel of Figure 2) found the SIRI variable to be highly predictive (p = 0.003), with a cut-off value of 0.93.
Remarkable findings can be observed in the context of the prediction of metastatic lymph nodes. Considering a baseline accuracy of 0.94, using radiomics features in a logistic regression model (Table 4), both CART and CIDT were found to achieve an asymptotic accuracy value of 1.00. The results and decision charts obtained (Figure 1, right panel, and Figure 2, bottom panel) emphasize that metastatic lymph nodes can be predicted with only one radiomics feature. Specifically, CART used the wavelet_HHH_glrlm_LowGrayLevelRunEmphasis feature for classifying patients with or without metastatic lymph nodes, with a cut-off value of 0.41. On the other hand, the variable showing a significant conditionally dependence (p < 0.001) with the metastatic lymph nodes found using the CIDT algorithm was the wavelet_HLH_glcm_Idn (cut-off equal to 0.045).
Further analysis was also performed for the prediction of both perineural and vascular infiltration; however, as the machine learning approach did not improve the performance achieved in the univariate analysis, and considering that a more complex multivariate model would be useless, the results have not been reported or discussed.

4. Discussion

Currently, the management of the neck dissection in early-stage OTSCC remains a controversial topic, especially if evaluations are made in risk/benefit terms with respect to the possible oncological outcomes and the possible treatment morbidity.
Various options have been proposed for the early detection of lymph node metastases. In 2015, the Sentinel European Node Trial (SENT) assessed the identification of metastatic lymph nodes through the sentinel lymph node biopsy on a sample of 415 patients, and reported an overall sensitivity and negative predictive value of 86% and 95%, respectively [11]. Moreover, approaches based on artificial intelligence, machine learning, and radiomics metrics are largely reported in the literature in the context of oncological studies [12,13,14,15,16,17,18,19,20,21,22].
Different studies suggested several clinical parameters as predictive markers for the presence of occult laterocervical metastases; some of the most used are the degree of differentiation of the tumor, the depth of tumor invasion, the perineural invasion, the vascular invasion, and the intercourse neutrophils/lymphocytes. However, as already reported by Tsushima et al., these parameters are not always sufficient to identify patients with a high risk of occult metastatic lymph nodes, as there is a significant heterogeneity in the outcomes within single-stage groups [21].
In our study, we assessed the predictive power of both clinical parameters and radiomics features, considered both individually and in combination. The focus was the prediction of the risk of metastatic lymph nodes and the tumor grade related to OTSCC. Specifically, we considered, as clinical indicators, the most recent indices of inflammation (NLR, PLR, LMR, SII, and SIRI) and the DOI. In fact, it is now widely recognized that the inflammatory response plays a critical role in tumor promotion and progression, potentially influencing the survival outcomes [23,24,25,26,27,28,29,30,31,32,33,34].
Since the eighth edition of the AJCC staging system, DOI has been added to the TNM as a staging criterion of oral cancers [23]. According to the latest edition of TNM, a DOI higher than 5 mm allows for classifying the neoplasm as T2, even in the case of tumors smaller than 2 cm [24]. Furthermore, the National Comprehensive Cancer Network suggested one staging dissection of the neck when the DOI is greater than 4 mm. D’Cruz et al. [28] performed a randomized control study and demonstrated that staging neck dissection has a better prognosis compared to therapeutic neck dissection.
However, the main limitation is the absence of a standardized definition of the DOI, leading to confusing the infiltration thickness with the DOI [25,26,27,28].
In our study, DOI alone achieved an accuracy of 70% in discriminating the tumor grading, with a cut-off of 5.43 mm; a cut-off of 4.76 mm, instead, was found to be able to detect metastatic lymph nodes with an accuracy of 79%.
Moreover, NLR has recently established itself as a leading index of inflammation, recognized as an independent prognostic factor in many solid tumor studies, including head and neck sites.
Assuming the effectiveness of NLR as an independent prognostic factor in head and neck tumors, for the first time, Abbate et al. demonstrated its correlation with occult lymph node metastases in early-stage carcinomas of the tongue [32]. The results of these authors demonstrated that the neutrophil/lymphocyte ratio was not only an independent prognostic factor for survival, but was is also useful for identifying patients at highest risk of occult metastatic lymph nodes.
This result was also confirmed by Wu YY et al. in their study on 890 patients with cancers of the oral cavity [34].
Furthermore, Kumarasamy C et al. [35] confirmed, with their systematic review of 34 studies, that PLR and LMR have been recognized as valid prognostic factors in the therapeutic management of oncological pathologies of the head and neck [24]. On the other hand, Feng Y [24] obtained significant prognostic efficacy with the use of SII and SIRI indices. In line with this, Valero C et al. [36] found that SIRI has a prognostic capacity in squamous-cell tumors of the head and neck area, noticing a significant decrease in survival for patients with higher values of SIRI.
These findings were confirmed by our results, with particular reference to the detection of metastatic lymph nodes. In fact, SII and SIRI can identify patients with metastatic lymph nodes with an accuracy of 78%, while NLR reported and accuracy of 79%. In addition, it was possible to detect metastatic lymph nodes with an accuracy of 83% using PLR alone. Similar results were also found in the classification of patients based on tumor grading (G1 + G2 versus G3 grading); in fact, PLR, SII, and SIRI were able to classifying the patients with an accuracy of 70%, when considered individually.
However, in the univariate analysis, no good performance was obtained by a single parameter.
Higher performances were achieved when considering radiomics metrics; the wavelet_HHH_glrlm_LowGrayLevelRunEmphasis feature, in fact, showed an accuracy of 96% in the identification of metastatic lymph nodes. The ability of a single radiomics predictor in identifying metastatic lymph nodes was confirmed in the multivariate analysis; using the CART algorithm, in fact, it was possible to classify metastatic and non-metastatic lymph nodes using only the wavelet_HHH_glrlm_LowGrayLevelRunEmphasis feature, reaching an asymptotic accuracy value of 100%. The cut-off value identified by the algorithm (0.41) was almost equal to that identified in the ROC analysis (0.39). Comparable results were observed when using a combination of 15 radiomics features in a logistic regression models with an accuracy of 94%.
That a single radiomics feature is able to classify metastatic lymph nodes was also confirmed by the CIDT method, where the wavelet_HLH_glcm_Idn feature showed a high correlation with the outcome (asymptotic accuracy of 100%, p < 0.001).
Further good results were obtained for predicting the tumor grading with a combination of radiomics metrics and clinical parameters. The CART algorithm reached an accuracy of appropriately 82% when combining SIRI, DOI, and PLR with the wavelet_LLL-glszm_SizeSoneNonUniformityNormalized feature. In particular, it is interesting to make an observation regarding the SIRI variable and the cut-off values identified during the analyses related to tumor grading; in fact, the cut-off for the SIRI variable (0.95) identified by the CART method was almost equal to that identified in the ROC analysis (cut-off of 0.93) and with the CIDT method (cut-off of 0.93).
No significant results were, instead, found when predicting both the perineural and vascular infiltrations.
Given its preliminary nature, this study has some limitations. The manual segmentation of ROIs makes the data operator-dependent; this poses challenges in terms of study reproducibility. Moreover, despite the validation-based approach used in the multivariate analysis, the small sample size could introduce some bias in the results, although the cross-validation technique tends to mitigate this risk. Both these limits can be significantly reduced by increasing the number of samples and recruiting patients in additional medical centers. In this way, in fact, it is possible not only to obviate the limited training abilities of the machine learning algorithms due to the small sample size, but it is also possible to dilute the bias that affects the data, which is due to the manual segmentation of the radiologist. Semi-automated segmentation could also be considered in order to improve reproducibility of the results.

5. Conclusions

In conclusion, the results obtained in this preliminary study agree with the literature [37,38,39], and confirm that both radiomics features and clinical parameters have an important role in identifying tumor grading and metastatic lymph nodes. Machine learning approaches can be used as an easy-to-use tool to stratify patients with early-stage OTSCC, based on the identification of metastatic and non-metastatic lymph nodes. However, some limitations of this preliminary study need to be highlighted. This study has a retrospective nature and the segmentation of ROIs was manually performed by radiologists. Moreover, the small sample size certainly had an impact on the results discussed, especially in the context of the machine learning approach. Overcoming these limitations should be one of the main goals of further studies.

Author Contributions

Data curation, U.C. and F.P.; Formal analysis, U.C. and R.F.; Investigation, U.C., E.D.B., V.A., G.S., F.M., G.D.O., P.P., P.B., A.A., F.P., M.G.M., S.V.S., V.G., G.I., A.P. and L.C.; Methodology, U.C., E.D.B., V.A., G.S., F.M., G.D.O., P.P., P.B., A.A., F.P., M.G.M., S.V.S., V.G., G.I., F.I., A.P. and L.C.; Writing—original draft, U.C.; Writing—review & editing, U.C. and R.F. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was performed in accordance with relevant guidelines and regulations and was approved by the Local Ethics Committee, University of Naples Federico II Protocol of Ethical Committe n. 12/19.

Informed Consent Statement

Informed consent was not required by the Local Ethics Committee of the National Cancer Institute of Naples due to the retrospective nature of the study.

Data Availability Statement

Acknowledgments

The authors are grateful to Alessandra Trocino, librarian at the National Cancer Institute of Naples, Italy. Moreover, for the collaboration, authors are grateful for the re-search support to Paolo Pariate, Andrea Esposito and Eugenio Sorgente of Istituto Nazionale Tumori IRCCS Fondazione Pascale—IRCCS di Napoli, 80131 Napoli, Italy.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Nocini, R.; Capocasale, G.; Marchioni, D.; Zotti, F. A Snapshot of Knowledge about Oral Cancer in Italy: A 505 Person Survey. Int. J. Environ. Res. Public Health 2020, 17, 4889. [Google Scholar] [CrossRef] [PubMed]
  2. Lim, S.C.; Zhang, S.; Ishii, G.; Endoh, Y.; Kodama, K.; Miyamoto, S.; Hayashi, R.; Ebihara, S.; Cho, J.S.; Ochiai, A. Predictive markers for late cervical metastasis in stage I and II invasive squamous cell carcinoma of the oral tongue. Clin. Cancer Res. 2004, 10, 166–172. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Vandenbrouck, C.; Sancho-Garnier, H.; Chassagne, D.; Saravane, D.; Cachin, Y.; Micheau, C. Elective versus therapeutic radical neck dissection in epidermoid carcinoma of the oral cavity: Results of a randomized clinical trial. Cancer 1980, 46, 386–390. [Google Scholar] [CrossRef]
  4. Sagheb, K.; Sagheb, K.; Rahimi-Nedjat, R.; Taylor, K.; Al-Nawas, B.; Walter, C. Sentinel lymph node biopsy in T1/T2 squamous cell carcinomas of the tongue: A prospective study. Oncol. Lett. 2016, 11, 600–604. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Piazza, C.; Montalto, N.; Paderno, A.; Taglietti, V.; Nicolai, P. Is it time to incorporate ‘depth of infiltration’ in the T staging of oral tongue and floor of mouth cancer? Curr. Opin. Otolaryngol. Head Neck Surg. 2014, 22, 81–89. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Balkwill, F.B.; Mantovani, A. Inflammation and cancer: Back to Virchow? Lancet 2001, 357, 539–545. [Google Scholar] [CrossRef]
  7. Kumarasamy, C.; Tiwary, V.; Sunil, K.; Suresh, D.; Shetty, S.; Muthukaliannan, G.K.; Baxi, S.; Jayaraj, R. Prognostic Utility of Platelet-Lymphocyte Ratio, Neutrophil-Lymphocyte Ratio and Monocyte-Lymphocyte Ratio in Head and Neck Cancers: A Detailed PRISMA Compliant Systematic Review and Meta-Analysis. Cancers 2021, 13, 4166. [Google Scholar] [CrossRef] [PubMed]
  8. Avanzo, M.; Stancanello, J.; El Naqa, I. Beyond imaging: The promise of radiomics. Phys Med. 2017, 38, 122–139. Available online: https://pyradiomics.readthedocs.io/en/latest/features.html (accessed on 15 February 2021). [CrossRef]
  9. Zwanenburg, A.; Vallières, M.; Abdalah, M.A.; Aerts, H.J.W.L.; Andrearczyk, V.; Apte, A.; Ashrafinia, S.; Bakas, S.; Beukinga, R.J.; Boellaard, R.; et al. The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Im-age-based Phenotyping. Radiology 2020, 295, 328–338. [Google Scholar] [CrossRef] [Green Version]
  10. Chilling, C.; Stoeckli, S.J.; Haerle, S.K.; Broglie, M.A.; Huber, G.A.; Sorensen, J.A. Sentinel European Node Trial (SENT): 3-year results of sentinel node biopsy in oral cancer. Eur. J. Cancer 2015, 51, 2777–2784. [Google Scholar] [CrossRef]
  11. Van Assen, M.; Muscogiuri, G.; Caruso, D.; Lee, S.J.; Laghi, A.; De Cecco, C.N. Artificial intelligence in cardiac radiology. Radiol. Med. 2020, 125, 1186–1199. [Google Scholar] [CrossRef] [PubMed]
  12. Neri, E.; Coppola, F.; Miele, V.; Bibbolino, C.; Grassi, R. Artificial intelligence: Who is responsible for the diagnosis? Radiol. Med. 2020, 125, 517–521. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Belfiore, M.P.; Urraro, F.; Grassi, R.; Giacobbe, G.; Patelli, G.; Cappabianca, S.; Reginelli, A. Artificial intelligence to codify lung CT in Covid-19 patients. Radiol. Med. 2020, 125, 500–504. [Google Scholar] [CrossRef] [PubMed]
  14. Cristofaro, M.; Busi Rizzi, E.; Piselli, P.; Pianura, E.; Petrone, A.; Fusco, N.; Di Stefano, F.; Schinina, V. Image quality and radiation dose reduction in chest CT in pulmonary infection. Radiol. Med. 2020, 125, 451–460. [Google Scholar] [CrossRef] [PubMed]
  15. Gurgitano, M.; Angileri, S.A.; Rodà, G.M.; Liguori, A.; Pandolfi, M.; Ierardi, A.M.; Wood, B.J.; Carrafiello, G. Interventional Radiology ex-machina: Impact of Artificial Intelligence on practice. Radiol. Med. 2021, 126, 998–1006. [Google Scholar] [CrossRef] [PubMed]
  16. Cappabianca, S.; Fusco, R.; de Lisio, A.; Paura, C.; Clemente, A.; Gagliardi, G.; Lombardi, G.; Giacobbe, G.; Russo, G.M.; Belfiore, M.P.; et al. Correction to: Clinical and laboratory data, radiological structured report findings and quantitative evaluation of lung involvement on baseline chest CT in COVID-19 patients to predict prognosis. Radiol. Med. 2021, 126, 643, Erratum in Radiol. Med. 2021, 126, 29–39. [Google Scholar] [CrossRef]
  17. Coppola, F.; Faggioni, L.; Regge, D.; Giovagnoni, A.; Golfieri, R.; Bibbolino, C.; Miele, V.; Neri, E.; Grassi, R. Artificial intelligence: Radiologists’ expectations and opinions gleaned from a nationwide online survey. Radiol. Med. 2021, 126, 63–71. [Google Scholar] [CrossRef]
  18. Chianca, V.; Albano, D.; Messina, C.; Vincenzo, G.; Rizzo, S.; Del Grande, F.; Sconfienza, L.M. An update in musculoskeletal tumors: From quantitative imaging to radiomics. Radiol. Med. 2021, 126, 1095–1105. [Google Scholar] [CrossRef]
  19. Scapicchio, C.; Gabelloni, M.; Barucci, A.; Cioni, D.; Saba, L.; Neri, E. A deep look into radiomics. Radiol. Med. 2021, 126, 1296–1311. [Google Scholar] [CrossRef]
  20. Benedetti, G.; Mori, M.; Panzeri, M.M.; Barbera, M.; Palumbo, D.; Sini, C.; Muffatti, F.; Andreasi, V.; Steidler, S.; Doglioni, C.; et al. CT-derived radiomics features to discriminate histologic characteristics of pancreatic neuroendocrine tumors. Radiol. Med. 2021, 126, 745–760. [Google Scholar] [CrossRef]
  21. Nardone, V.; Reginelli, A.; Grassi, R.; Boldrini, L.; Vacca, G.; D’Ippolito, E.; Annunziata, S.; Farchione, A.; Belfiore, M.P.; Desideri, I.; et al. Delta radiomics: A systematic review. Radiol. Med. 2021, 126, 1571–1583. [Google Scholar] [CrossRef] [PubMed]
  22. Sakashita, T.; Homma, A.; Hatakeyama, H.; Kano, S.; Mizumachi, T.; Kakizaki, T.; Suzuki, T.; Fukuda, S. The role of prophylactic neck dissection and tumor thickness evaluation for patients with cN0 tongue squamous cell carcinoma. Eur. Arch. Oto-Rhino-Laryngol. 2016, 273, 3987–3992. [Google Scholar]
  23. Edge, S.B.; Byrd, D.R.; Compton, C.C.; Fritz, A.G.; Greenen, F.L.; Trotti, A. American Joint Committee on Cancer (AJCC). Cancer Staging Manual, 8th ed.; Springer: New York, NY, USA, 2017; pp. 1–1854. [Google Scholar]
  24. Pentenero, M.; Gandolfo, S.; Carrozzo, M. Importance of tumor thickness and depth of invasion in nodal involvement and prognosis of oral squamous cell carcinoma: A review of the literature. Head Neck 2005, 27, 1080–1091. [Google Scholar] [CrossRef] [PubMed]
  25. Byers, R.M.; El-Naggar, A.K.; Lee, Y.Y. Can we detect or predict the presence of occult nodal metastases in patients with squamous carcinoma of the oral tongue? Head Neck 1998, 20, 138–144. [Google Scholar] [CrossRef]
  26. Fukano, H.; Matsuura, H.; Hasegawa, Y.; Nakamura, S. Depth of invasion as a predictive factor for cervical lymph node metastasis in tongue carcinoma. Head Neck 1997, 19, 205–210. [Google Scholar] [CrossRef]
  27. Kuan, E.C.; Mallen-St Clair, J.; Badran, K.W.; St John, M.A. How does depth of invasion influence the decision to do a neck dissection in clinically N0 oral cavity cancer? Laryngoscope 2016, 126, 547–548. [Google Scholar] [CrossRef] [Green Version]
  28. D’Cruz, A.K.; Vaish, R.; Kapre, N. Elective versus therapeutic neck dissection in node-negative oral cancer. N. Engl. J. Med. 2015, 373, 521–529. [Google Scholar] [CrossRef]
  29. Templeton, A.J.; McNamara, M.G.; Šeruga, B.; Vera-Badillo, F.E.; Aneja, P.; Ocaña, A.; Leibowitz-Amit, R.; Sonpavde, G.; Knox, J.J.; Tran, B.; et al. Prognostic role of neutrophil-to-lymphocyte ratio in solid tumors: A systematic review and meta-analysis. J. Natl. Cancer Inst. 2014, 106, 122–124. [Google Scholar] [CrossRef] [Green Version]
  30. El-Hag, A.; Clark, R.A. Immunosuppression by activated human neutrophils. Dependence on the myeloperoxidase system. J. Immunol. 1987, 139, 2406–2413. [Google Scholar]
  31. Li, J.; Jiang, R.; Liu, W.S.; Liu, Q.; Xu, M.; Feng, Q.S.; Chen, L.Z.; Bei, J.X.; Chen, M.Y.; Zeng, Y.X. A large cohort study reveals the association of elevated peripheral blood lymphocyte-to-monocyte ratio with favorable prognosis in nasopharyngeal carcinoma. PLoS ONE 2013, 8, e83069. [Google Scholar] [CrossRef] [Green Version]
  32. Salzano, G.; Dell’Aversana Orabona, G.; Abbate, V.; Vaira, L.A.; Committeri, U.; Bonavolontà, P.; Piombino, P.; Maglitto, F.; Russo, C.; Russo, D.; et al. The prognostic role of the pre-treatment neutrophil to lymphocyte ratio (NLR) and tumor depth of invasion (DOI) in early-stage squamous cell carcinomas of the oral tongue. Oral Maxillofac. Surg. 2021, 26, 21–32. [Google Scholar] [CrossRef] [PubMed]
  33. Abbate, V.; Dell’Aversana Orabona, G.; Salzano, G.; Bonavolontà, P.; Maglitto, F.; Romano, A.; Tarabbia, F.; Turri-Zanoni, M.; Attanasi, F.; Di Lauro, A.E.; et al. Pre-treatment Neutrophil to Lymphocyte Ratio as a predictor for occult cervical metastasis in early stage (T1-T2 cN0) squamous cell carcinoma of the oral tongue. Surg. Oncol. 2018, 27, 503–507. [Google Scholar] [CrossRef] [PubMed]
  34. Wu, Y.Y.; Chang, K.P.; Ho, T.Y.; Chou, W.C.; Hung, S.P.; Fan, K.H.; Chiang, Y.Y.; Chou, Y.C.; Tsang, N.M. Comparative prognostic value of different preoperative complete blood count cell ratios in patients with oral cavity cancer treated with surgery and postoperative radiotherapy. Cancer Med. 2021, 10, 1975–1988. [Google Scholar] [CrossRef] [PubMed]
  35. Feng, Y.; Zhang, N.; Wang, S.; Zou, W.; He, Y.; Ma, J.A.; Liu, P.; Liu, X.; Hu, C.; Hou, T. Systemic Inflammation Response Index Is a Predictor of Poor Survival in Locally Advanced Nasopharyngeal Carcinoma: A Propensity Score Matching Study. Front. Oncol. 2020, 10, 575417. [Google Scholar] [CrossRef]
  36. Valero, C.; Pardo, L.; Sansa, A.; Garcia Lorenzo, J.; López, M.; Quer, M.; León, X. Prognostic capacity of Systemic Inflammation Response Index (SIRI) in patients with head and neck squamous cell carcinoma. Head Neck 2020, 42, 336–343. [Google Scholar] [CrossRef]
  37. Romeo, V.; Cuocolo, R.; Ricciardi, C.; Ugga, L.; Cocozza, S.; Verde, F.; Stanzione, A.; Napolitano, V.; Russo, D.; Improta, G.; et al. Prediction of Tumor Grade and Nodal Status in Oropharyngeal and Oral Cavity Squamous-cell Carcinoma Using a Radiomic Approach. Anticancer Res. 2020, 40, 271–280. [Google Scholar] [CrossRef]
  38. Shan, J.; Jiang, R.; Chen, X.; Zhong, Y.; Zhang, W.; Xie, L.; Cheng, J.; Jiang, H. Machine Learning Predicts Lymph Node Metastasis in Early-Stage Oral Tongue Squamous Cell Carcinoma. J. Oral Maxillofac. Surg. 2020, 78, 2208–2218. [Google Scholar] [CrossRef]
  39. Tomita, H.; Yamashiro, T.; Heianna, J.; Nakasone, T.; Kimura, Y.; Mimura, H.; Murayama, S. Nodal-based radiomics analysis for identifying cervical lymph node metastasis at levels I and II in patients with oral squamous cell carcinoma using contrast-enhanced computed tomography. Eur. Radiol. 2021, 31, 7440–7449. [Google Scholar] [CrossRef]
Figure 1. Decision trees obtained with the CART algorithm for the prediction of tumor grading (left panel) and metastatic lymph nodes (right panel).
Figure 1. Decision trees obtained with the CART algorithm for the prediction of tumor grading (left panel) and metastatic lymph nodes (right panel).
Biology 11 00468 g001
Figure 2. Decision trees obtained with the CIDT algorithm for the prediction of tumor grading (top panel) and metastatic lymph nodes (bottom panel). For each inner node, the Bonferroni-adjusted p-values are given, while the fraction of responsive patients is displayed for each terminal node.
Figure 2. Decision trees obtained with the CIDT algorithm for the prediction of tumor grading (top panel) and metastatic lymph nodes (bottom panel). For each inner node, the Bonferroni-adjusted p-values are given, while the fraction of responsive patients is displayed for each terminal node.
Biology 11 00468 g002
Table 1. Univariate analysis. Results of the ROC analysis of the following clinical parameters: depth of invasion (DOI), neutrophils−lymphocytes ratio (NLR), platelet−lymphocyte ratio (PLR), lymphocytes−monocytes ratio (LMR), Systemic Inflammation Response Index (SIRI), Systemic Immune-inflammation Index (SII), and tumor largest diameter (SIZE).
Table 1. Univariate analysis. Results of the ROC analysis of the following clinical parameters: depth of invasion (DOI), neutrophils−lymphocytes ratio (NLR), platelet−lymphocyte ratio (PLR), lymphocytes−monocytes ratio (LMR), Systemic Inflammation Response Index (SIRI), Systemic Immune-inflammation Index (SII), and tumor largest diameter (SIZE).
Tumor Grading
DOINLRPLRLMRSIISIRISIZE
AUC0.720.650.660.340.730.700.74
Sensitivity0.680.890.510.970.680.861.00
Specificity0.730.430.860.090.730.570.39
PPV0.680.570.760.470.680.630.58
NPV0.730.830.680.800.730.831.00
Accuracy0.700.640.700.490.700.700.67
Cut-off5.432.11153.332.67563.260.9319.00
Metastatic Lymph Nodes
DOINLRPLRLMRSIISIRISIZE
AUC0.820.730.760.220.720.740.70
Sensitivity0.890.740.800.060.770.740.86
Specificity0.720.830.850.960.780.800.57
PPV0.700.760.800.500.730.740.60
NPV0.890.810.850.570.820.800.84
Accuracy0.790.790.830.570.780.780.69
Cut-off4.762.92142.027.47563.261.4223.00
Perineural Infiltration
DOINLRPLRLMRSIISIRISIZE
AUC0.770.570.650.340.670.650.62
Sensitivity0.740.690.590.030.690.640.36
Specificity0.740.550.741.000.740.740.90
PPV0.730.590.681.000.710.690.78
NPV0.760.660.660.530.720.690.60
Accuracy0.740.620.670.530.720.690.64
Cut-off5.112.41145.557.52563.231.3732.00
Vascular Infiltration
DOINLRPLRLMRSIISIRISIZE
AUC0.690.640.530.400.640.650.58
Sensitivity0.930.500.540.960.640.680.93
Specificity0.490.850.620.080.720.700.25
PPV0.490.640.430.360.550.540.39
NPV0.930.760.720.800.790.800.87
Accuracy0.640.730.590.380.690.690.48
Cut-off3.643.26142.022.67578.011.4218.00
Table 2. Univariate analysis. Most significant results of the ROC analysis of radiomics features extracted from ROIs on both the tumor and lymph node areas.
Table 2. Univariate analysis. Most significant results of the ROC analysis of radiomics features extracted from ROIs on both the tumor and lymph node areas.
Performance ResultsTumor Gradin—Tumor Area Metastatic Lymph Nodes—Lymph Node AreaPerineural Infiltration—Tumor AreaVascular Infiltration—Tumor Area
Original_Glszm_HighgraylevelzoneemphasisWavelet_HHH_Glrlm_LowgraylevelrunemphasisWavelet_HHH_Glcm_MaximumprobabilityWavelet_LLL_Glszm_Highgraylevelzoneemphasis
AUC0.660.930.650.62
Sensitivity0.760.940.690.29
Specificity0.660.980.670.98
PPV0.650.970.660.89
NPV0.760.960.700.72
Accuracy0.700.960.680.74
Cut-off−0.190.394.970.03
Table 3. Multivariate analysis. Results of machine learning approaches (logistic regression and tree-based algorithms) in the prediction of tumor grading.
Table 3. Multivariate analysis. Results of machine learning approaches (logistic regression and tree-based algorithms) in the prediction of tumor grading.
Performance ResultsClinical FeaturesRadiomics FeaturesCombination of Both Clinical and Radiomics Features
Logistic RegressionLogistic RegressionLogistic RegressionCARTCIDT
Accuracy0.650.760.590.820.65
Sensitivity0.440.780.560.780.33
Specificity0.870.750.620.871.00
No. of features32644
Table 4. Multivariate analysis. Results of machine learning approaches (logistic regression and tree-based algorithms) in the prediction of metastatic lymph nodes.
Table 4. Multivariate analysis. Results of machine learning approaches (logistic regression and tree-based algorithms) in the prediction of metastatic lymph nodes.
Performance ResultsClinical FeaturesRadiomics FeaturesCombination of Both Clinical and Radiomics Features
Logistic RegressionLogistic RegressionLogistic RegressionCARTCIDT
Accuracy0.760.940.881.001.00
Sensitivity0.571.000.861.001.00
Specificity0.900.900.901.001.00
No. of features4152011
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Committeri, U.; Fusco, R.; Di Bernardo, E.; Abbate, V.; Salzano, G.; Maglitto, F.; Dell’Aversana Orabona, G.; Piombino, P.; Bonavolontà, P.; Arena, A.; et al. Radiomics Metrics Combined with Clinical Data in the Surgical Management of Early-Stage (cT1–T2 N0) Tongue Squamous Cell Carcinomas: A Preliminary Study. Biology 2022, 11, 468. https://doi.org/10.3390/biology11030468

AMA Style

Committeri U, Fusco R, Di Bernardo E, Abbate V, Salzano G, Maglitto F, Dell’Aversana Orabona G, Piombino P, Bonavolontà P, Arena A, et al. Radiomics Metrics Combined with Clinical Data in the Surgical Management of Early-Stage (cT1–T2 N0) Tongue Squamous Cell Carcinomas: A Preliminary Study. Biology. 2022; 11(3):468. https://doi.org/10.3390/biology11030468

Chicago/Turabian Style

Committeri, Umberto, Roberta Fusco, Elio Di Bernardo, Vincenzo Abbate, Giovanni Salzano, Fabio Maglitto, Giovanni Dell’Aversana Orabona, Pasquale Piombino, Paola Bonavolontà, Antonio Arena, and et al. 2022. "Radiomics Metrics Combined with Clinical Data in the Surgical Management of Early-Stage (cT1–T2 N0) Tongue Squamous Cell Carcinomas: A Preliminary Study" Biology 11, no. 3: 468. https://doi.org/10.3390/biology11030468

APA Style

Committeri, U., Fusco, R., Di Bernardo, E., Abbate, V., Salzano, G., Maglitto, F., Dell’Aversana Orabona, G., Piombino, P., Bonavolontà, P., Arena, A., Perri, F., Maglione, M. G., Setola, S. V., Granata, V., Iaconetta, G., Ionna, F., Petrillo, A., & Califano, L. (2022). Radiomics Metrics Combined with Clinical Data in the Surgical Management of Early-Stage (cT1–T2 N0) Tongue Squamous Cell Carcinomas: A Preliminary Study. Biology, 11(3), 468. https://doi.org/10.3390/biology11030468

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop