A CT-Based Clinical, Radiological and Radiomic Machine Learning Model for Predicting Malignancy of Solid Renal Tumors (UroCCR-75)

Background: Differentiating benign from malignant renal tumors is important for patient management, and it may be improved by quantitative CT features analysis including radiomic. Purpose: This study aimed to compare performances of machine learning models using bio-clinical, conventional radiologic and 3D-radiomic features for the differentiation of benign and malignant solid renal tumors using pre-operative multiphasic contrast-enhanced CT examinations. Materials and methods: A unicentric retrospective analysis of prospectively acquired data from a national kidney cancer database was conducted between January 2016 and December 2020. Histologic findings were obtained by robotic-assisted partial nephrectomy. Lesion images were semi-automatically segmented, allowing for a 3D-radiomic features extraction in the nephrographic phase. Conventional radiologic parameters such as shape, content and enhancement were combined in the analysis. Biological and clinical features were obtained from the national database. Eight machine learning (ML) models were trained and validated using a ten-fold cross-validation. Predictive performances were evaluated comparing sensitivity, specificity, accuracy and AUC. Results: A total of 122 patients with 132 renal lesions, including 111 renal cell carcinomas (RCCs) (111/132, 84%) and 21 benign tumors (21/132, 16%), were evaluated (58 +/− 14 years, men 74%). Unilaterality (100/111, 90% vs. 13/21, 62%; p = 0.02), necrosis (81/111, 73% vs. 8/21, 38%; p = 0.02), lower values of tumor/cortex ratio at portal time (0.61 vs. 0.74, p = 0.01) and higher variation of tumor/cortex ratio between arterial and portal times (0.22 vs. 0.05, p = 0.008) were associated with malignancy. A total of 35 radiomics features were selected, and “intensity mean value” was associated with RCCs in multivariate analysis (OR = 0.99). After ten-fold cross-validation, a C5.0Tree model was retained for its predictive performances, yielding a sensitivity of 95%, specificity of 42%, accuracy of 87% and AUC of 0.74. Conclusion: Our machine learning-based model combining clinical, radiologic and radiomics features from multiphasic contrast-enhanced CT scans may help differentiate benign from malignant solid renal tumors.


Introduction
Renal cell carcinomas (RCCs) account for approximately 70% of all cases of renal cancer. The most common subtypes are clear-cell renal cell carcinoma (ccRCC), papillary renal cell carcinoma (pRCC) and chromophobe renal cell carcinoma (chRCC), accounting for 70%, 15% and 5% of all RCCs, respectively. As these subtypes have different natural histories and prognoses, it is crucial to differentiate them accurately. Moreover, some common benign tumors show a similar presentation to that of RCCs. Oncocytoma, a benign renal tumor accounting for 5% of all renal masses, is occasionally mistaken for RCC; oncocytomas account for 4-10% of all nephrectomy cases [1]. Furthermore, 5% of angiomyolipomas remain challenging to differentiate from RCC on CT due to their fat-poor nature. This may lead to unnecessary surgical treatment, raising concerns regarding morbidity.
In some circumstances, i.e., when the malignancy status remains unclear before surgery, percutaneous biopsy can be performed. This procedure shows excellent diagnostic performance; it can differentiate between benign and malignant lesions with sensitivity and specificity values of approximately 95%. However, in 20% of cases, the results remain indeterminate, and the complication rate is 8% [2]. Furthermore, for some histological subtypes, pre-operative histological diagnosis is challenging. A recent study reported that 25% of oncocytomas suspected on biopsy were ultimately diagnosed as RCC after surgical removal; 12.5% of these were of the chRCC subtype [3]. There has been an increase in the number of biopsies, particularly on smaller and smaller tumors, with the risk of non-contributory biopsies [4], hence the need to develop imaging characterization.
Multiparametric MRI has been well described in the evaluation of more common subtypes of RCC; however, oncocytoma result in poor imaging diagnostic accuracy [5].
CT is used for renal mass characterization [6]. CT was chosen for the ease of access and spatial resolution for small lesions. In routine clinical practice, qualitative and semiquantitative parameters are used in combination to distinguish benign from malignant renal masses. Visual analysis of the tumor shape, size, content and enhancement is performed [7]. Some studies showed that an analysis of enhancement patterns on multiphasic contrastenhanced (MCE)-CT images has high diagnostic accuracy [8]. While enhancement analysis is quantitative, shape and texture analyses remain more subjective and are thus vulnerable to interpretation variability.
Large-scale quantitative parameters can be extracted from medical CT images and then subjected to texture analysis for the detection of local variation in pixel intensity. This has emerged as a novel technique to quantitively evaluate tumor heterogeneity, assess the histopathologic characteristics of carcinomas and help predict prognosis [9][10][11][12].
Radiomics features provide information about the tumor intensity, shape and texture, and application of machine learning analysis to improved imaging data interpretation.
Although recent studies have aimed to differentiate RCCs from benign renal tumors using radiomics [12][13][14], none of them used three-dimensional (3D) radiomic feature extraction combined with clinical and radiological conventional parameters to assess the performance of machine learning (ML) models. The 3D contour-focused segmentation showed a higher stable feature rate [15].
Therefore, the aim of this study was to evaluate the ability of ML models to differentiate between benign and malignant solid renal tumors via the MCE-CT 3D segmentation of extracted radiomic, radiological and clinical features.

Patients
In this retrospective analysis of prospectively collected data, we included all patients who underwent robot-assisted partial nephrectomy for solid renal tumors at our institution between January 2016 and December 2020. Ethics approval was granted by our institutional ethics review board (IRB DR-2013-206). Participants were enrolled from a national kidney cancer database. We included patients who had undergone pre-operative abdominal MCE-CT at our institution. CT examinations performed outside our institution, and those not conducted in accordance with our examination protocol or with low-quality images, were excluded. Patients with missing picture archiving system data were also excluded. Biological, clinical and histological data were extracted from the prospective database. The clinical and biological features analyzed included sex, age and body weight at the time of surgery, the Eastern Cooperative Oncology Group (ECOG) score, the glomerular filtration rate, the presence of urologic symptoms and the clinical tumor-node-metastasis (TNM) stage. Histological findings of interest included malignancy, the histological subtypes of benign and malignant tumors, the Fuhrman grade and the histopathological TNM stage.

CT Examinations
All imaging examinations were performed at our institution using the same 64-slice CT scanner (Optima CT660; GE Healthcare, Milwaukee, WI, USA) before the patients underwent nephrectomy. The CT parameters were as follows: 120 kV; automatic current selection; maximum current, 500 mAs; rotation time, 0.7 s; collimation detector size, 40 × 1.2 mm; field of view, 350 × 350 mm; matrix size, 512 × 512; and reconstruction section thickness, 1.5 mm. For the vascular anatomy analysis, the arterial phase reconstruction pixel size was 0.625 mm.
First, unenhanced CT was performed. Then, a specific enhanced acquisition protocol was applied, including three-phase CT. Nonionic contrast medium (350-400 µmol/L) was injected into the antecubital vein at a rate of 3.5-4 mL/s to a final volume of 80 mL. Arterial phase images were obtained using the scanner's automatic bolus tracking system (SmartPrep; GE Healthcare), beginning 10 s after the attenuation threshold of 100 UH was reached in the upper abdominal aorta; after an additional 100 s, portal phase images were acquired. Finally, excretory phase images were acquired (10-15 min after injection).

CT
The CT scans were analyzed by one radiologist-in-training (C.G.) with 5 years of image analysis experience. The radiologist was blinded to the clinical and histological findings. The radiological features of all renal tumors were recorded, including infiltration, demarcated contours, homogeneity, calcifications, fat and hemorrhagic components, necrosis, necrotic core, tumor implantation, venous extension, multifocality and bilaterality. Representative cases are shown in Figures 1 and 2.   The enhancement pattern was noted for each lesion. A two-dimensional (2D) region of interest was drawn around the lesion on a single slice for both arterial and portal phase images. A second 2D region of interest was manually drawn on the same slice in a homogenous part of the renal cortex, again for both arterial and portal phase images. Finally, the ratios of the cortical to tumoral intensity values and of the arterial to arterial phases were calculated.
All Digital Imaging and Communication in Medicine images were anonymized. Segmentation was performed by C.G. using SOPHiA DDM for Radiomics v2.1.21 (SOPHiA GENETICS, Saint-Sulpice, Switzerland). In accordance with previous studies, nephrographic phase images were segmented due to their favorable tumor/renal parenchymal contrast. First, the slice on which the tumor was clearest (axial, coronal or sagittal plane) was chosen, and the tumor contours were precisely drawn by hand. Next, a volumetric model of the tumor was constructed using a deformation algorithm. If necessary, the user could manually adjust the semi-automatically obtained contours of the lesion. Each 3D segmentation process took approximately 20 min. The user interface of the segmentation software is shown in Figure 3. The enhancement pattern was noted for each lesion. A two-dimensional (2D) region of interest was drawn around the lesion on a single slice for both arterial and portal phase images. A second 2D region of interest was manually drawn on the same slice in a homogenous part of the renal cortex, again for both arterial and portal phase images. Finally, the ratios of the cortical to tumoral intensity values and of the arterial to arterial phases were calculated.
All Digital Imaging and Communication in Medicine images were anonymized. Segmentation was performed by C.G. using SOPHiA DDM for Radiomics v2.1.21 (SOPHiA GENETICS, Saint-Sulpice, Switzerland). In accordance with previous studies, nephrographic phase images were segmented due to their favorable tumor/renal parenchymal contrast. First, the slice on which the tumor was clearest (axial, coronal or sagittal plane) was chosen, and the tumor contours were precisely drawn by hand. Next, a volumetric model of the tumor was constructed using a deformation algorithm. If necessary, the user could manually adjust the semi-automatically obtained contours of the lesion. Each 3D segmentation process took approximately 20 min. The user interface of the segmentation software is shown in Figure 3.
More than 200 radiomic features were automatically extracted from the 3D segmentation model of the tumor (nephrographic phase). Previously described radiomic parameters (shape, pixel intensity and texture features) were analyzed. Dimensionality reduction was then performed using Kendall's correlation coefficient to avoid redundant parameters. An example process of the radiomic feature extraction is displayed in Figure 4. More than 200 radiomic features were automatically extracted from the 3D segmentation model of the tumor (nephrographic phase). Previously described radiomic parameters (shape, pixel intensity and texture features) were analyzed. Dimensionality reduction was then performed using Kendall's correlation coefficient to avoid redundant parameters. An example process of the radiomic feature extraction is displayed in Figure 4.
To differentiate between benign and malignant solid renal tumors from multimodal (clinical, radiological and radiomic) data with the best predictive performance while ensuring interpretability, the following ML models were trained: logistic regression with LASSO regularization to avoid overfitting (Logit-LASSO), binary decision tree (rpart), support vector machine with linear kernel (svmLinear), bagging method via random forest (RandomForest), and boosting method via C5.0 decision tree (C5.0Tree and wC5.0Tree to deal with imbalanced outcome). Class weights in LASSO-logistic regression and C5.0 tree were also incorporated to deal with the imbalance outcome (Logit-LASSO and wC5.0Tree, respectively).  Trained models were tested using a 10-fold cross-validation method [16]. The models were compared in terms of their ability to distinguish malignant from benign tumors based on sensitivity, specificity, accuracy and area under the receiver operating characteristic curve (AUC) values. To differentiate between benign and malignant solid renal tumors from multimodal (clinical, radiological and radiomic) data with the best predictive performance while ensuring interpretability, the following ML models were trained: logistic regression with LASSO regularization to avoid overfitting (Logit-LASSO), binary decision tree (rpart), support vector machine with linear kernel (svmLinear), bagging method via random forest (RandomForest), and boosting method via C5.0 decision tree (C5.0Tree and wC5.0Tree to deal with imbalanced outcome). Class weights in LASSO-logistic regression and C5.0 tree were also incorporated to deal with the imbalance outcome (Logit-LASSO and wC5.0Tree, respectively).

Statistical Analyses
Trained models were tested using a 10-fold cross-validation method [16]. The models were compared in terms of their ability to distinguish malignant from benign tumors based on sensitivity, specificity, accuracy and area under the receiver operating characteristic curve (AUC) values.

Statistical Analyses
Clinical data are presented as the mean ± standard deviation for continuous variables and as numbers and percentages for categorial variables. The Bonferroni method was applied for multiple comparisons. Univariable (Wilcoxon and Fisher's tests) and multivariable (l Logit-LASSO logistic regression, to obtain odds ratios (ORs)) analyses were conducted. All reported p-values are two-sided, and p < 0.05 was taken to indicate statistical significance.

Patients and Tumors
A total of 122 patients were included who were surgically treated at our institution between January 2016 and December 2020 ( Figure 5). Overall, two had two renal tumors, one had three renal tumors, and one had seven renal tumors; therefore, there were 132 renal lesions in total. There were 111 RCCs: 79 ccRCCs, 16 chRCCs, 13 pRCCs and 3 other rare renal carcinomas. There were also 21 benign lesions: 18 oncocytomas, 2 fat-poor angiomyolipoma (fpAMLs) and 1 other rare benign renal tumor. The mean age at diagnosis was 58 ± 14 years. Renal tumors were more frequent in males (87/132; 65.9%). Nine patients (7.3%) had sporadic kidney cancer, and seven (5.5%) had a family history of renal cancer. Regarding the bio-clinical characteristics, the mean body mass index was 26.7 kg/m 2 (range: 17.7-46.9 kg/m 2 ) and the mean Cockcroft clearance at diagnosis was 88.4 mL/min. Twenty-five patients (18.9%) had urologic symptoms The mean age at diagnosis was 58 ± 14 years. Renal tumors were more frequent in males (87/132; 65.9%). Nine patients (7.3%) had sporadic kidney cancer, and seven (5.5%) had a family history of renal cancer. Regarding the bio-clinical characteristics, the mean body mass index was 26.7 kg/m 2 (range: 17.7-46.9 kg/m 2 ) and the mean Cockcroft clearance at diagnosis was 88.4 mL/min. Twenty-five patients (18.9%) had urologic symptoms at diagnosis, and the majority (84.7%) had no physical limitations (ECOG score = 0). Table 1 shows the characteristics of the overall study population as well as of the benign and malignant groups. Univariable analysis showed that the risk of malignancy was higher in males than females (73.9% vs. 23.8%, p = 0.001) and in those with a higher body weight at diagnosis (79.6% vs. 67.4%, p = 0.02) ( Table 1). In the multivariable analysis, the correlation between sex and malignancy risk remained significant (OR = 2.35). A previous history of cancer (OR = 1.07) also showed a significant correlation with malignancy ( Table 2). The 95% confidence intervals were estimated using percentile approach, and the p-values were obtained as the rate of non-selection of the feature, both over 1000 bootstrap replicates.
In the multivariable analysis, the difference in the tumor/cortex ratio between the arterial and portal phases was the only enhancement parameter to show a significant difference between RCCs and benign tumors (OR = 0.83) ( Table 2).

Three-Dimensional Radiomic Features
More than 200 radiomic parameters were extracted by 3D segmentation in the portal phase. Feature selection based on Kendall's correlation coefficient led to the retention of 35 radiomic variables in the final analysis. A full list of the retained features is provided in the Supplementary Materials.

Discussion
This study compared ML models in terms of their ability to differentiate benign from malignant solid renal lesions in patients undergoing pre-operative contrast-enhanced CT at a single institution. The radiomic features were extracted after 3D segmentation of tu-

Discussion
This study compared ML models in terms of their ability to differentiate benign from malignant solid renal lesions in patients undergoing pre-operative contrast-enhanced CT at a single institution. The radiomic features were extracted after 3D segmentation of tumors in the nephrographic phase, and eight ML models were trained on all of these features. The C5.0Tree model showed the best predictive performance with a sensitivity of 95%, accuracy of 87% and AUC of 0.74.
Recently, Yap et al. [13] reported a classifier with an AUC of 0.70-0.73 for distinguishing benign from malignant tumors. In that study, radiomic features, shape and texture were analyzed in a large cohort of 735 renal lesions. Thus, combined shape and texture analysis can provide good classification performance. Yang et al. [12] included 118 RCCs and 45 fpAMLs in their study and differentiated them using a radiomic model based on noncontrast CT examinations. An AUC of 0.90 was achieved, while for the analysis of fourphase contrast-enhanced CT images, the AUC was 0.88. Sun et al. [17] compared the ability of ML models to differentiate among several histological subtypes in an analysis of 290 renal tumors. Their classifiers achieved sensitivity values of 86-90% for distinguishing RCCs from benign lesions and ccRCCs from other malignancies; the respective accuracy values were 86% and 90%. Interestingly, the models did not always perform better than trained radiologists. That is the only study in the literature to compare the performance of radiological, radiomic and combined radiological/radiomic models; the model based on enhancement ratios and radiomics performed better than both the radiomic featuresonly model and expert radiologists. Sun et al. also trained the same support vector machine model to differentiate ccRCCs from pRCCs and chRCCs; ccRCCs from fpAMLs and oncocytomas; and pRCCs and chRCCs from benign lesions.
Erdim et al. [14] achieved very good performance for their random forest algorithm: 84 solid renal masses were correctly identified as malignant at an accuracy of 91.7%. Although this rate is higher than that in our study, they included a smaller patient cohort and artificially adjusted the groups in terms of malignancy to reduce the impact of malignancy on model performance.
The most recent meta-analysis on the use of radiomic features to characterize renal tumors, by Muhlbauer et al. [18], included 30 studies. The overall quality of the studies was relatively low, with a median radiomic quality score of 19.4%. The main reasons for this were insufficient use of feature reduction methods, a lack of internal and external validation and poor data availability. Moreover, ML models using radiomic features are susceptible to overfitting.
Notably, the present study involved all of the common histological subtypes of renal tumors and developed an ML model to differentiate ccRCCs from other malignant subtypes, ccRCCs from fpAMLs and oncocytomas, and all malignant subtypes from benign tumors. In the majority of recent studies, the predictive performance of ML models was tested without distinguishing among subtypes [12,[19][20][21][22]. Deng and Yang [12,19]  In this study, we used an innovative, semi-automatic 3D segmentation process (in the nephrographic phase) to obtain a volumetric tumor model. Radiomic parameters, including shape, pixel intensity and texture features, were extracted and analyzed. The vast majority of previous radiomic studies used 2D segmentation techniques [12,14,22] and limited data extraction processes, especially those involving heterogeneous tumors. Our semi-automatic method allowed us to analyze tumor margins in one slice and to obtain a volumetric model of the masses via a software-based, time-efficient deep learning algorithm.
Our study had several limitations. First, the cohort was imbalanced; malignant and benign tumors represented 84% and 16% of all tumors, respectively. We included patients who underwent surgical treatment for the removal of a renal lesion; before partial nephrectomy, the images were reviewed, and most of the benign lesions were excluded. Furthermore, fpAMLs accounted for only 1.5% of all solid renal tumors in our cohort, which is lower than the rates (6-13%) reported in other recent studies [13,17,19]. Furthermore, we only included patients who underwent pre-operative CT examination at our institution using a protocol specific for MCE-CT. This allowed us to obtain homogenous imaging data, thus increasing the robustness of the analysis of radiomic features. However, this also raises issues concerning the clinical relevance and generalizability of the results. Furthermore, the 3D segmentation technique automatically reduces the margins of the volumetric model by 1 mm. This method is widely used in radiomic studies for margin shrinkage, which frequently described manual reductions in tumor contours of 1-3 mm [14,21,22,26], because it reduces the vulnerability of radiomic features to partial volume effects. However, Kocak et al. [27] recently showed that segmentation had a non-negligible impact on radiomic features. Their "specific contour" segmentation method yielded AUC values of 0.85-0.98 when distinguishing benign from malignant tumors compared with 0.75-0.8 when using the margin shrinkage approach. Since this is time consuming, it is not adapted to current clinical practice at the moment.
Finally, our findings were not validated against external data. Regarding the relatively small number of renal tumors included in the cohort, we trained and evaluated the models using data from the entire cohort and the 10-fold cross-validation technique. Although this method is widely used in radiomic studies, larger cohorts and external validation sets should be used to assess the performance of the ML algorithm developed herein.

Conclusions
This study showed that ML models can help with the non-invasive differentiation of malignant from benign solid renal tumors. These classifiers can efficiently analyze clinical, conventional radiological and radiomic features extracted from MCE-CT images to help clinicians diagnose and treat renal tumors. Data Availability Statement: Data sharing is not applicable to this article.

Conflicts of Interest:
The authors declare no conflict of interest.