Next Article in Journal
Upper Airway Changes and OSAS Risk in Patients after Mandibular Setback Surgery to Treat III Class Skeletal Malocclusion
Next Article in Special Issue
Breast Cancer Surrogate Subtype Classification Using Pretreatment Multi-Phase Dynamic Contrast-Enhanced Magnetic Resonance Imaging Radiomics: A Retrospective Single-Center Study
Previous Article in Journal
Is There a Role for Intraoperative Neuromonitoring in Intradural Extramedullary Spine Tumors? Results and Indications from an Institutional Series
Previous Article in Special Issue
Dynomics: A Novel and Promising Approach for Improved Breast Cancer Prognosis Prediction
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Statistical Approach to Assess the Robustness of Radiomics Features in the Discrimination of Mammographic Lesions

by
Alfonso Maria Ponsiglione
*,
Francesca Angelone
,
Francesco Amato
and
Mario Sansone
*,†
Department of Information Technology and Electrical Engineering, University of Naples Federico II, 80125 Naples, Italy
*
Authors to whom correspondence should be addressed.
These authors contributed equally to this work.
J. Pers. Med. 2023, 13(7), 1104; https://doi.org/10.3390/jpm13071104
Submission received: 30 May 2023 / Revised: 1 July 2023 / Accepted: 5 July 2023 / Published: 7 July 2023
(This article belongs to the Special Issue Precision Medicine in Radiomics and Radiogenomics)

Abstract

:
Despite mammography (MG) being among the most widespread techniques in breast cancer screening, tumour detection and classification remain challenging tasks due to the high morphological variability of the lesions. The extraction of radiomics features has proved to be a promising approach in MG. However, radiomics features can suffer from dependency on factors such as acquisition protocol, segmentation accuracy, feature extraction and engineering methods, which prevent the implementation of robust and clinically reliable radiomics workflow in MG. In this study, the variability and robustness of radiomics features is investigated as a function of lesion segmentation in MG images from a public database. A statistical analysis is carried out to assess feature variability and a radiomics robustness score is introduced based on the significance of the statistical tests performed. The obtained results indicate that variability is observable not only as a function of the abnormality type (calcification and masses), but also among feature categories (first-order and second-order), image view (craniocaudal and medial lateral oblique), and the type of lesions (benign and malignant). Furthermore, through the proposed approach, it is possible to identify those radiomics characteristics with a higher discriminative power between benign and malignant lesions and a lower dependency on segmentation, thus suggesting the most appropriate choice of robust features to be used as inputs to automated classification algorithms.

1. Introduction

Breast cancer is a multifactorial disease, characterized by the uncontrolled multiplication of mammary gland cells [1]. The major risk factors associated with breast cancer can be classified into: (i) nonmodifiable risk factors (such as age, gender, genetic factors, a family history of breast cancer, previous breast cancer, and/or proliferative breast disease); (ii) modifiable risk factors (such as reproductive factors, radiation exposure, hormone replacement therapy, alcohol, and a high-fat diet); (iii) environmental factors (such as exposure to organochlorines, electromagnetic fields, and smoke) [2,3,4]. Although there is not an absolute correlation between the disease and the above-mentioned factors, they are assessed in the anamnesis phase and can influence the diagnosis [1]. According to national reports [5], breast cancer ranks first in terms of mortality level among the cancer pathologies affecting the female population worldwide, particularly in the 35–55 age group, with 6–7% of cases being metastatic at the time of the diagnosis [5]. Since the chances of recovery strongly depend on the tumour grade at the time it is diagnosed, early detection is of fundamental importance. Indeed, advanced-grade breast cancers could require invasive interventions followed by heavy radiotherapy and chemotherapy treatments [6,7,8]. On the other hand, early-stage tumours can often require less invasive therapeutic approaches, with increased success rates. Therefore, screening protocols and campaigns can serve as helpful approaches to foster the early diagnosis of breast cancer [9,10,11].
In this regard, mammography (MG) is among the most widespread techniques to perform screening exams for the detection of small nodules, potentially dangerous calcifications, and other types of suspicious lesions [12,13]. The screening exam consists of an X-ray scan of each breast in two projections (or views), namely the craniocaudal (CC) and the medial lateral oblique (MLO) projection, to improve anomaly visualization, thus obtaining four images per patient. Based on the signs/symptoms or the patient’s history, a further mammogram may be required. Therefore, MG results in an X-ray low-dose greyscale digital image representing the structure of the breast [14].
Despite its ability to detect 85–90% of breast cancers early, before a medical examination [15], several misdiagnosis cases and errors are still reported and can be attributable to the following main reasons: (i) the nature of the abnormality, which exhibits a high variability in its morphological characteristics; (ii) the similarity between potentially malignant masses and surrounding healthy tissues, especially occurring in young women with dense fibrous breast tissue, which appears opaque in MG. In order to overcome these issues, computer-aided detection and diagnosis (CAD) systems have been introduced to assist radiologists in the interpretation of the medical images and in segmenting, extracting, processing, and classifying imaging features within suspicious regions of interest (ROIs) to determine the phenotypic characteristics of the abnormalities, thus helping to differentiate malignant abnormalities from benign ones [16]. Indeed, by means of radiomics, which consists of an advanced extraction and analysis of textural features from medical images, quantitative parameters and measures of tumour heterogeneity can be obtained, thus supporting the interpretation of the radiology findings [17].
Studies from the literature have shown that a double reading from two independent radiologists can improve the diagnosis of breast lesions [18,19,20,21]; however, a double check has a high operational cost in terms of resource utilization and time, thus preventing its application in broader healthcare settings. In this sense, CAD systems, along with reproducible and standardized radiomics workflows, could offer the opportunity to obtain a “second opinion” from an automated tool, thus improving the diagnostic accuracy with less operational cost and within a reduced timeframe [22,23]. The sensitivity of such automated tools, i.e., the ability to find and properly characterize suspicious areas, is, therefore, of fundamental importance to increase the reliability and clinical translation of CAD systems and radiomics workflows in most radiology applications [24,25,26,27,28,29,30,31,32], which can be then combined with artificial intelligence (AI) approaches, such as machine learning (ML) and deep learning (DL) techniques, to further boost the automated diagnostic performance [33,34,35,36,37,38,39,40,41].
To ensure the reliability, robustness, and reproducibility of CAD systems to process and classify medical images, and to facilitate and support classification tasks, an accurate and appropriate choice of radiomics features is of utmost importance, by selecting those that better describe and reflect the tissue properties and lesion characteristics. In this context, the main objective of the present work is to examine radiomics features’ robustness and variability depending on the segmented ROI of breast masses and calcification from a public database of MG images. An automated technique is adopted to generate artificial ROIs, namely underestimated and overestimated regions, starting from the original ROI associated with the MG image. A statistical approach is carried out to assess the variability of radiomics features across the original and the artificially generated ROI, and a robustness score is provided.

2. Methods

2.1. Dataset

The study is carried out on a sample of 622 digitized mammograms from the public database “Curated Breast Imaging Subset of Digital Database for Screening Mammography” (CBIS-DDSM) [42,43], provided by The Cancer Imaging Archive (TCIA) [44]. The images are DICOM images obtained through a lossless decompression of the original Digital Database for Screening Mammography (DDSM) images, distributed in a Lossless Joint Photographic Experts Group (LJPEG) format. The database includes instances with certified pathology data for normal, benign, and cancerous conditions, and is designed to serve as a tool for testing and improving decision support and CAD systems. For the purpose of this work, only breast projections containing abnormalities were considered. The type of anomaly between calcifications and masses, or both, is specified in the metadata associated with the database, and the classification of the abnormalities between malignant and benign is also indicated. Out of the 622 images considered in this study, 268 concerned abnormalities classified as calcifications (146 MLO views and 122 CC views), and 354 concerned abnormalities classified as masses (188 MLO views and 166 CC views).

2.2. Methodological Workflow

The main steps of the adopted methodological workflow are shown in Figure 1.
A preprocessing phase was followed by the generation of two artificial ROI based on the original segmentation provided with the input image. Both overestimated and underestimated segmentations were produced and given as inputs to a radiomics feature extraction process based on PyRadiomics [45]. Six groups of features were extracted from each segmentation and a statistical analysis phase was then performed to assess the variability in the features across the different ROIs and to derive a robustness score based on the results of the statistical analysis.
In the following subsections, a detailed explanation of each phase of the methodological workflow displayed in Figure 1 is provided.

2.3. Preprocessing

The examination of mammograms for breast cancer screening and diagnosis requires a preprocessing step, which has been shown to be of utmost importance to reduce the incidence of false-positive results [46]. In this work, image preprocessing was carried out by means of contrast limited adaptive histogram equalization (CLAHE), as presented and suggested by Tripathya and Swarnkarb in [46] and by Zuiderveld Karel [47].
Figure 2 shows a comparison between the original and preprocessed images.
The adopted preprocessing algorithm enhances the contrast in rectangular discrete sections in which the original image is divided and implements a bilinear interpolation to join the adjacent sections. The algorithm is implemented in MatLab (R2021b, The Mathworks, Inc., Natick, MA, USA). Subsequently, the preprocessed image, along with the associated ROI binary mask representing the original segmentation, is passed to an automated algorithm for the generation of synthetic ROI binary masks based on the original one, as detailed in the following subsection.

2.4. Generation of Artificial ROI

An automated algorithm for the generation of artificial ROI binary masks based on the overestimation and underestimation of the original segmentation was proposed and carried out in MatLab (R2021b, The Mathworks, Inc., Natick, MA, USA). The algorithm takes the original image and associated original segmentation as inputs and automatically applies both a binary morphological dilation and an erosion operation on the input ROI, taking advantage of the decomposition of a structuring element as described in [48,49], in order to generate both an overestimated and an underestimated ROI binary mask. An octagonal structuring element is chosen for both dilation and erosion operations. The extent or the scale factor of the morphological operation, i.e., the distance between the structuring element origin and its sides, is proportional to the original ROI main dimension, taken as the length (expressed in pixels) of the major axis of the ellipse that has the same normalized second central moments as the original ROI. The scale factor of the dilation and erosion was chosen to be proportional to the original ROI main dimension. The constant of proportionality between the scale factor of the morphological operation and the major axis of the original ROI was empirically set to 0.06 (i.e., the original ROI was dilated/eroded to an extent equal to 6% of its major axis). This allowed us to achieve perceptible changes with respect to the original ROI while avoiding excessive dilation or erosion.
Figure 3 provides a graphical representation of the contours of the artificial segmentations compared with the original one.
In Figure 4, original and artificially generated ROI masks and contours are reported for mammograms in both MLO and CC views.
Both original, overestimated, and underestimated segmentations were given as inputs to a radiomics feature extraction process based on PyRadiomics [45], as detailed in the following subsection.

2.5. Radiomics Feature Extraction

Once artificial ROIs are generated, a feature extraction process using PyRadiomics version 3.0.1 was carried out. PyRadiomics is an open-source Python library [45] for the extraction of radiomics data from medical images in compliance with the Imaging Biomarker Standardisation Initiative (IBSI). Values of different statistical and textural features are calculated on the preprocessed MG image within each ROI, both corresponding to the original and artificial segmentations. Radiomics features are usually divided according to the characteristic to which they refer: those related to the morphology of the lesion and those related to the texture. In the former case, the size and shape of the lesion are studied, while the latter, being linked to the image texture, are based on the histogram of the grey levels and the corresponding matrix of the pixels forming the image. Here, six groups of radiomics features are computed. These consist of first-order (FO) features, and the following second-order features: grey-level co-occurrence matrix (GLCM), grey-level dependence matrix (GLDM), grey-level run length matrix (GLRLM), grey-level size zone matrix (GLSZM), and neighbourhood grey tone difference matrix (NGTDM).
The FO operators are statistical operators based on the image brightness histogram and, thus, describe the distribution of values related to individual pixels. Although not providing any information on a morphological level, they allow obtaining a first estimate of the brightness and homogeneity of the image. Indeed, based on the histogram, statistical parameters can be considered to estimate the radiomics features, such as the mean (representing the average intensity that the grey levels can assume), the variance (representing a measure of the spread of the distribution with respect to the mean, in other words, an indirect measure of the image contrast), the skewness (i.e., the symmetry of a distribution with respect to the mean), the kurtosis (representing the peak shape of the ROI values’ distribution, with higher kurtosis), and the entropy (representing the randomness in the pixel values). Unlike the FO features, the second-order ones were calculated using second-order statistics and provided an evaluation of the spatial dependence between the values associated with individual pixels. The descriptors were calculated on the co-occurrence matrix, or GLCM, which was calculated starting from an image I of dimensions n × m by determining how many times a pixel with intensity i was spatially related with a pixel of value j. Each element (i, j) in the resulting matrix was equal to the sum of the occurrences in which the i-th pixel was related to j-th pixel of the image. Concerning the GLDM, it represents the grey-level dependencies in an image, i.e., the number of pixels connected to and dependent on the centre pixel of the image within a given distance, δ. As far as the GLRLM is concerned, it records the occurrence of all various combinations of grey-level values and grey-level runs in an ROI for a given direction, typically along four principal directions (i.e., horizontal, antidiagonal, vertical, and diagonal). Regarding the GLSZM, it is a matrix representation of a 2D histogram of the total number of zones of size j and of intensity value i. Finally, the NGTDM stores the sum of absolute differences for a given grey-level i by quantifying the difference between the reference grey value and the average grey value of its neighbours within a given distance δ.
The feature values obtained for all the groups were collected in a CSV file before undergoing a statistical analysis, as discussed in the following section.

2.6. Statistical Analysis and Radiomics Robustness Score

The data obtained from the feature extractor were then collected in an CSV file, with the first columns reporting the metadata associated with each processed record and patient, and the subsequent columns including the values for each calculated radiomics feature. A statistical analysis was carried out on both abnormality type (masses or calcifications) and pathology type (benign or malignant) separately. The data were grouped according to the following segmentation classes: original ROI, overestimated ROI, and underestimated ROI, in order to assess the robustness of the radiomics features across the three examined ROI. A first analysis was performed on the whole sample, i.e., considering both MLO and CC views. Then, the same analysis was reiterated, taking into account each image view (MLO and CC) separately in order to investigate the robustness and variability of radiomics features at each specific projection.
A Shapiro–Wilk test (α = 0.05) was carried out to assess the normality of the distribution of the radiomics features across the groups to be compared. Based on the results of the normality checks, a parametric (or nonparametric) test, namely ANOVA (or Kruskal–Wallis), was performed (α = 0.05) to compare the radiomics features distributions across the three segmentation classes according to pathology type (benign and malignant), abnormality type (masses and calcifications), and view type (MLO or CC). Based on the p-value of the statistical test, an “overall robustness score”, ranging from 0 to 1, was defined as equal to the significance of the test. In this way, the closer to 1 the overall score is, the less statistically significant the test (high p-value) is, thereby reflecting a higher robustness in the examined radiomics feature, whose distribution did not show significant variations across the ROI segmentations. For example, if the Kruskal–Wallis statistical test carried out on a specific radiomics feature returned a p-value equal to 0.02 (i.e., less than the significance level α), it could be concluded that a statistically significant difference existed among the median values assumed by the examined radiomics features in the three segmentations (original vs. underestimated vs. overestimated). As a consequence, we could hypothesize that the examined radiomics feature varied significantly across the different segmentations of the same ROI and, therefore, it lacked robustness. With the aim of quantifying the robustness, the “overall robustness score” attributed to the examined radiomics feature was chosen to be equal to 0.02, i.e., equal to the probability value (p-value) of the employed statistical test. Conversely, if the p-value was equal to 0.80, it could be concluded that the median values of the examined radiomics feature did not significantly differ across the three explored segmentations, leading us to the hypothesis that the examined feature is robust, with an “overall robustness score” of 0.80.
Since the above-mentioned test indicated that at least one difference existed among the classes, but did not identify where the stochastic difference occurred, a parametric (or nonparametric) test, namely Student’s t-test (or the Mann–Whitney U-test), was performed to make a pairwise comparison of the radiomics feature distributions between the original ROI and the overestimated ROI as well as between the original ROI and the underestimated ROI, in order to assess the feature robustness at both dilation and erosion transformation of the segmented area. Like the overall robustness score, a “segmentation-specific robustness score” (hereafter referred to as the “robustness score”) is defined as equal to the p-value of the test. For example, as for the overall robustness score, if the p-value of 0.02 (i.e., less than the significance level α) is obtained for the Mann–Whitney U-test to compare the median values of certain radiomics features between the original ROI and the overestimated ROI, then we can conclude that a statistically significant difference exists between the feature median values in the two classes (original vs. overestimated ROI). Therefore, we could hypothesize that the examined radiomics feature lacks robustness and, thereby, we could attribute a low robustness score (0.02) equal to the p-value of the test. Conversely, if a p-value of 0.80 is obtained with the same test, it can be concluded that the median values of the feature measured on the original ROI do not significantly vary on the overestimated ROI, leading us to the hypothesis that the examined feature is robust against ROI overestimation, with an “overall robustness score” of 0.80. The definition of a more specific score is necessary to achieve a better and wiser assessment of the robustness by considering the effects of ROI dilation and erosion separately.
Finally, boxplots were used to visually represent the obtained results in terms of robustness scores. In particular, the variability of the scores calculated for the six different feature classes (FO, GLCM, GLDM, GLRLM, GLSZM, and NGTDM) was shown through boxplots by grouping the data according to the abnormality type (calcifications or masses), pathology type (benign or malignant), and view type (MLO or CC). Boxplots were also used to carry out a visual analysis on a representative set of radiomics features to illustrate the variability at different robustness levels (high vs. low, qualitatively defined according to the robustness score) and at different levels of discriminative power (high vs. low, qualitatively defined according to the ability to discriminate between benign and malignant lesions).
All the feature elaborations and statistical tests were carried out in R language within the RStudio environment.

3. Results

In Figure 5, the distributions of the overall robustness scores calculated for both masses and calcifications are represented. The results are differentiated according to the pathology type (benign or malignant) and according to the image view (MLO and CC).
It can be observed that the radiomics features calculated for calcifications showed higher overall robustness scores, especially for the FO, GLCM, GLRLM, and GLSZM categories (Figure 5a), compared to the corresponding ones computed for masses (Figure 5b). This is far more evident when comparing the scores obtained for CC and MLO views in the calcification subset (Figure 5c,e) with those in the mass subset (Figure 5d,f).
Figure 6 displays the distributions of the robustness scores in the different radiomics feature classes calculated for masses and calcifications. From this figure, it is possible to investigate the segmentation-specific robustness by separately and independently observing the behaviour of the robustness score after ROI dilation (overestimated ROI) and ROI erosion (underestimated ROI). The results are differentiated according to the pathology type (benign or malignant) and according to the image view (MLO or CC).
It can be observed that the radiomics features calculated on the underestimated ROI in for calcifications (Figure 6a) and masses (Figure 6b) were far more robust than the respective features calculated on the overestimated ROI, thus confirming a nondesirable impact of the dilation operation on the consistency of the radiomics feature distributions. This consideration appears to be applicable to all feature categories in the calcification subset (Figure 6a), while only to GLCM, GLDM, GLRLM, and GLSZM classes in the mass subset (Figure 6b). Similar behaviour was also observable when grouping the data according to image view type (MLO and CC), with higher robustness scores in the calcification dataset (Figure 6c,e) compared to the corresponding ones computed for masses (Figure 6d,f).
A binary qualitative robustness level (low or high) is defined by ranking the calculated radiomics features based on their robustness score. Similarly, a binary qualitative discriminative power level (low or high) is defined through a visual observation of the feature values’ distribution across benign and malignant abnormalities. In Figure 7, some representative examples of radiomics features in the FO category with different robustness levels and different discriminative power are shown for both calcifications and masses.
Among the FO radiomics features, the kurtosis showed to be the one with the highest robustness score in both the calcification (Figure 7a) and mass (Figure 7e) subsets. Indeed, especially for calcification (Figure 7a), it can be observed that the distributions were consistent across the segmentations for both benign and malignant groups. Despite its robustness, the kurtosis showed a rather low discriminative power in distinguishing between benign and malignant abnormalities, as can be visually deduced from the boxplots in Figure 7a,e, with both showing almost overlapping median values between the benign and malignant groups in each segmentation. Similarly, the variance showed to be the one with the lowest discriminative power score in both the calcification (Figure 7c) and mass (Figure 7g) subsets. However, differently from the kurtosis, it also showed low robustness as the distribution values showed significant changes across the different segmentations in both benign and malignant cases. Moreover, further differences can be observed between the calcification and mass subsets. Indeed, the features showing both high robustness and high discriminative power were the energy in the calcification dataset (Figure 7b) and the maximum in the mass dataset (Figure 7f), while the features showing both low robustness and high discriminative power were the mean absolute deviation in the calcification dataset (Figure 7d) and the root mean square in the mass dataset (Figure 7h).

4. Discussion

Breast cancer accounts for the highest mortality rate among cancer pathologies affecting female population globally. In recent years, the mortality rate has decreased thanks to early diagnosis, especially through mammography (MG) screening campaigns for the detection of tumour lesions in the early stages to avoid invasive interventions and unnecessary therapies. However, despite MG being the most widespread technique to perform breast cancer screening, lesion detection and classification between benign and malignant tissues remains a complicated issue due to the great variability in the shape and size of lesions and their similarity to healthy tissues. The automated extraction of morphological, statistical, and textural features of medical images within a radiomics workflow proved to be helpful in aiding the early diagnosis of diseases.
Although several studies have investigated the repeatability of radiomics features, most of them focused on the following: a comparison across different manufacturers [50,51] and different image storage formats (raw vs. processed) [52] in full-field digital mammography (FFDM); a comparison between various AI-based feature extraction and segmentation approaches, e.g., radiomics and DL in magnetic resonance imaging (MRI) [53], manual and DL segmentation in in MRI [54,55], and computed tomography (CT) [56]; and the study of the impact of both clinical and radiomics features in contrast-enhanced spectral mammography (CESM) [57,58], in MG [59], and in FFDM [60]. Despite these valuable research efforts, and although the IBSI initiative addresses the issue of the reproducibility of radiomics characteristics, it is still difficult to interpret radiomics signatures and most studies have concentrated on repeatability using various imaging devices or reconstruction methods but employing one segmentation, which might not be accessible in clinical practice. Instead, from a clinical perspective, it is of utmost importance to assess the robustness of radiomics features regarding segmentation variability in order to identify those features to be included in the clinical evaluation.
In this study, the variability and robustness of radiomics features was investigated as a function of the lesion segmentation in MG images from a public database. A statistical analysis was carried out to assess feature variability and a radiomics robustness score was introduced based on the significance of the statistical tests performed. The major contributions of this work are three-fold: (i) the proposal of an approach to investigate and evaluate the variability in radiomics features as a function of the ROI size; (ii) the quantification of an overall robustness score based on the p-value of the statistical analysis of the radiomics features; (iii) the investigation of robustness across different radiomics categories, namely FO, GLCM, GLDM, GLRLM, GLSZM, and NGTDM.
As can be observed from the obtained results (Figure 5 and Figure 6), variability was observable not only as a function of the abnormality type (calcification and masses), but also among the feature categories (first-order and second-order), image view (craniocaudal and medial lateral oblique), and pathology (benign and malignant). The representative case of a FO radiomics feature (Figure 7) highlights that the energy was able to provide both an adequate robustness level and a good degree of discrimination between benign and malignant calcifications. The energy reflects the magnitude of pixel values in an image. A larger value, which is observed in malignant calcifications, could imply a higher intensity in the abnormality region. Similarly, the maximum was identified among the most robust and with higher discriminative power among the FO radiomics features calculated on breast masses. As in the case of the energy, the maximum reflects the intensity of the lesion area, and it showed larger values in malignant abnormalities compared to benign ones. The wise selection of most robust and predictive features along with their biological interpretation could serve as a basis to build resilient radiomics workflows in medical imaging and could provide novel insights in the definition of reliable indicators and biomarkers to detect malignancies and support in-depth analysis and diagnosis.
As far as the limitations of the present study are concerned, it is worth underlining that the statistical analysis carried out was univariate, as each variable was analysed individually without considering the interplay among the different extracted features. In future works, in order to improve the evidence provided by the present study, a multivariate analysis will be carried out by considering all the radiomics features simultaneously and performing a robustness assessment based on the obtained results. Furthermore, as the size of the dataset adopted in this study was limited to images from a public database referring to the same institution and image acquisition protocol, further studies could be focused on assessing the robustness at a multicentre level, including MG images acquired with different protocols and instruments, potentially providing additional insights into the variability and robustness of the radiomics features. Last but not least, future studies will also address the robustness of ML models across different segmentations by exploiting the workflow proposed here.

5. Conclusions

Radiomics is a promising method for identifying image-based indicators to support personalized diagnosis and treatment. The aim of this work was to investigate the robustness of radiomics features in MG as a function of the lesion segmentation to provide an automatic tool for designing a resilient and robust radiomics workflow based on the selection of segmentation-independent features with higher discriminative power to be used in abnormality characterization and classification tasks. In order to investigate and understand which radiomics characteristics can be considered most robust in MG, an algorithm was implemented to generate artificial segmentations based on an originally defined ROI, statistical analysis was carried out to assess the feature variability across different abnormality segmentations, and a radiomics robustness score was introduced based on the significance of the statistical tests performed. The results show that variability was observable not only as a function of the abnormality type (calcification and masses) but also among feature categories (first-order and second-order), image view (craniocaudal and medial lateral oblique), and pathology (benign and malignant). In particular, radiomics features were shown to be more robust while classifying breast calcifications rather than breast masses (as shown in Figure 5a,b and Figure 6a,b). Considering craniocaudal (CC) and mediolateral oblique (MLO) views separately, this difference between feature robustness across calcification and masses was even more evident (as shown in Figure 5c–f and Figure 6c–f). In order to investigate the predictive power of the features and their relative robustness, a representative example of the results obtained for the radiomics features of the first order is provided in Figure 7 of the manuscript. Indeed, in the case of breast calcifications, Figure 7a–d show how the energy turned out to be the most robust radiomics feature and the one with the highest discriminative power between benign and malignant lesions; conversely, in the case of breast masses, Figure 7e–h show how the maximum turned out to be the most robust radiomics feature and the one with the highest discriminative power between benign and malignant lesions. These results show that a preliminary selection of segmentation-independent and predictive features in MG images can be achieved and could suggest links between either statistical or textural image patterns and the biological/anatomical characteristics of the abnormalities, thus building a route for the definition of reliable descriptors to be adopted in CAD systems and to be used to test automated classification algorithms.

Author Contributions

Conceptualization, F.A. (Francesco Amato) and M.S.; methodology, A.M.P., F.A. (Francesca Angelone), M.S. and F.A. (Francesco Amato); validation, A.M.P., F.A. (Francesca Angelone), M.S. and F.A. (Francesco Amato); formal analysis, A.M.P. and F.A. (Francesca Angelone); investigation, A.M.P. and F.A. (Francesca Angelone); data curation, A.M.P. and F.A. (Francesca Angelone); writing—original draft preparation, A.M.P., F.A. (Francesca Angelone), M.S. and F.A. (Francesco Amato); writing—review and editing, A.M.P., F.A. (Francesca Angelone), M.S. and F.A. (Francesco Amato); visualization, A.M.P., F.A. (Francesca Angelone); supervision, M.S. and F.A. (Francesco Amato); F.A. (Francesco Amato) and M.S. equally contributed to the work. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Nindrea, R.D.; Aryandono, T.; Lazuardi, L.; Dwiprahasto, I. Diagnostic Accuracy of Different Machine Learning Algorithms for Breast Cancer Risk Calculation: A Meta-Analysis. Asian Pac. J. Cancer Prev. APJCP 2018, 19, 1747–1752. [Google Scholar] [CrossRef]
  2. Davis, J.; Liang, J.; Petterson, M.B.; Roh, A.T.; Chundu, N.; Kang, P.; Matz, S.L.; Connell, M.J.; Gridley, D.G. Risk Factors for Late Screening Mammography. Curr. Probl. Diagn. Radiol. 2019, 48, 40–44. [Google Scholar] [CrossRef] [PubMed]
  3. Takkar, N.; Kochhar, S.; Garg, P.; Pandey, A.K.; Dalal, U.R.; Handa, U. Screening methods (clinical breast examination and mammography) to detect breast cancer in women aged 40–49 years. J. Life Health 2017, 8, 2–10. [Google Scholar] [CrossRef] [PubMed]
  4. Ho, P.J.; Lau, H.S.H.; Ho, W.K.; Wong, F.Y.; Yang, Q.; Tan, K.W.; Tan, M.-H.; Chay, W.Y.; Chia, K.S.; Hartman, M.; et al. Incidence of breast cancer attributable to breast density, modifiable and non-modifiable breast cancer risk factors in Singapore. Sci. Rep. 2020, 10, 503. [Google Scholar] [CrossRef] [Green Version]
  5. Tumori, A.I.R. I numeri del cancro in italia 2021. Intermedia Ed. 2021, 64–66. Available online: https://www.aiom.it/wp-content/uploads/2021/10/2021_NumeriCancro_web.pdf (accessed on 1 May 2023).
  6. Tubiana-Hulin, M.; Stevens, D.; Lasry, S.; Guinebretière, J.M.; Bouita, L.; Cohen-Solal, C.; Cherel, P.; Rouëssé, J. Response to neoadjuvant chemotherapy in lobular and ductal breast carcinomas: A retrospective study on 860 patients from one institution. Ann. Oncol. 2006, 17, 1228–1233. [Google Scholar] [CrossRef] [PubMed]
  7. Swanick, C.W.; Smith, B.D. Indications for adjuvant radiation therapy in breast cancer: A review of the evidence and recommendations for clinical practice. Chin. Clin. Oncol. 2016, 5, 38. [Google Scholar] [CrossRef]
  8. Riaz, N.; Jeen, T.; Whelan, T.J.; Nielsen, T.O. Recent Advances in Optimizing Radiation Therapy Decisions in Early Invasive Breast Cancer. Cancers 2023, 15, 1260. [Google Scholar] [CrossRef]
  9. Gøtzsche, P.C.; Jørgensen, K.J. Screening for breast cancer with mammography. Cochrane Database Syst. Rev. 2013, 2013, CD001877. [Google Scholar] [CrossRef]
  10. Burrell, H.C.; Sibbering, D.M.; Wilson, A.R.; Pinder, S.E.; Evans, A.J.; Yeoman, L.J.; Elston, C.W.; Ellis, I.O.; Blamey, R.W.; Robertson, J.F. Screening interval breast cancers: Mammographic features and prognosis factors. Radiology 1996, 199, 811–817. [Google Scholar] [CrossRef]
  11. Mann, R.M.; Hooley, R.; Barr, R.G.; Moy, L. Novel Approaches to Screening for Breast Cancer. Radiology 2020, 297, 266–285. [Google Scholar] [CrossRef] [PubMed]
  12. Dang, L.-A.; Chazard, E.; Poncelet, E.; Serb, T.; Rusu, A.; Pauwels, X.; Parsy, C.; Poclet, T.; Cauliez, H.; Engelaere, C.; et al. Impact of artificial intelligence in breast cancer screening with mammography. Breast Cancer 2022, 29, 967–977. [Google Scholar] [CrossRef]
  13. Tari, D.U.; Pinto, F. Mammography in Breast Disease Screening and Diagnosis. J. Pers. Med. 2023, 13, 228. [Google Scholar] [CrossRef] [PubMed]
  14. Houn, F.; Brown, M.L. Current practice of screening mammography in the United States: Data from the National Survey of Mammography Facilities. Radiology 1994, 190, 209–215. [Google Scholar] [CrossRef] [PubMed]
  15. Melekoodappattu, J.G.; Subbian, P.S. A Hybridized ELM for Automatic Micro Calcification Detection in Mammogram Images Based on Multi-Scale Features. J. Med. Syst. 2019, 43, 183. [Google Scholar] [CrossRef] [PubMed]
  16. Rodríguez-Ruiz, A.; Krupinski, E.; Mordang, J.-J.; Schilling, K.; Heywang-Köbrunner, S.H.; Sechopoulos, I.; Mann, R.M. Detection of Breast Cancer with Mammography: Effect of an Artificial Intelligence Support System. Radiology 2019, 290, 305–314. [Google Scholar] [CrossRef] [PubMed]
  17. Korte, J.C.; Cardenas, C.; Hardcastle, N.; Kron, T.; Wang, J.; Bahig, H.; Elgohari, B.; Ger, R.; Court, L.; Fuller, C.D.; et al. Radiomics feature stability of open-source software evaluated on apparent diffusion coefficient maps in head and neck cancer. Sci. Rep. 2021, 11, 17633. [Google Scholar] [CrossRef]
  18. Dinnes, J.; Moss, S.; Melia, J.; Blanks, R.; Song, F.; Kleijnen, J. Effectiveness and cost-effectiveness of double reading of mammograms in breast cancer screening: Findings of a systematic review. In Database of Abstracts of Reviews of Effects (DARE): Quality-assessed Reviews [Internet]; Centre for Reviews and Dissemination: Edinburgh, UK, 2001. [Google Scholar]
  19. Posso, M.; Puig, T.; Carles, M.; Rué, M.; Canelo-Aybar, C.; Bonfill, X. Effectiveness and cost-effectiveness of double reading in digital mammography screening: A systematic review and meta-analysis. Eur. J. Radiol. 2017, 96, 40–49. [Google Scholar] [CrossRef] [Green Version]
  20. Taplin, S.; Abraham, L.; Barlow, W.E.; Fenton, J.J.; Berns, E.A.; Carney, P.A.; Cutter, G.R.; Sickles, E.A.; Carl, D.; Elmore, J.G. Mammography Facility Characteristics Associated with Interpretive Accuracy of Screening Mammography. JNCI J. Natl. Cancer Inst. 2008, 100, 876–887. [Google Scholar] [CrossRef] [Green Version]
  21. Taplin, S.H.; Rutter, C.M.; Elmore, J.G.; Seger, D.; White, D.; Brenner, R.J. Accuracy of Screening Mammography Using Single Versus Independent Double Interpretation. Am. J. Roentgenol. 2000, 174, 1257–1262. [Google Scholar] [CrossRef]
  22. Andreadis, I.; Nikita, K.; Spyrou, G. Investigating the performance of a CADx scheme for mammography in specific BIRADS categories. In Proceedings of the 2014 IEEE International Conference on Imaging Systems and Techniques (IST) Proceedings, Santorini, Greece, 14–17 October 2014; pp. 335–339. [Google Scholar]
  23. Li, L.; Zheng, Y.; Zhang, L.; Clark, R.A. Anisotropic diffusion filtering of mammographic image for CAD in digital mammography. In Proceedings of the WCC 2000—ICSP 2000. 2000 5th International Conference on Signal Processing Proceedings. 16th World Computer Congress 2000, Beijing, China, 21–25 August 2000; Volume 2, pp. 1159–1162. [Google Scholar]
  24. Baker, J.A.; Rosen, E.L.; Lo, J.Y.; Gimenez, E.I.; Walsh, R.; Soo, M.S. Computer-Aided Detection (CAD) in Screening Mammography: Sensitivity of Commercial CAD Systems for Detecting Architectural Distortion. Am. J. Roentgenol. 2003, 181, 1083–1088. [Google Scholar] [CrossRef] [PubMed]
  25. Warren Burhenne, L.J.; Wood, S.A.; D’Orsi, C.J.; Feig, S.A.; Kopans, D.B.; O’Shaughnessy, K.F.; Sickles, E.A.; Tabar, L.; Vyborny, C.J.; Castellino, R.A. Potential Contribution of Computer-aided Detection to the Sensitivity of Screening Mammography. Radiology 2000, 215, 554–562. [Google Scholar] [CrossRef] [PubMed]
  26. Malich, A.; Fischer, D.R.; Böttcher, J. CAD for mammography: The technique, results, current role and further developments. Eur. Radiol. 2006, 16, 1449–1460. [Google Scholar] [CrossRef] [PubMed]
  27. Wei, R.; Lin, K.; Yan, W.; Guo, Y.; Wang, Y.; Li, J.; Zhu, J. Computer-Aided Diagnosis of Pancreas Serous Cystic Neoplasms: A Radiomics Method on Preoperative MDCT Images. Technol. Cancer Res. Treat. 2019, 18, 1533033818824339. [Google Scholar] [CrossRef] [Green Version]
  28. Yang, S.K.; Moon, W.K.; Cho, N.; Park, J.S.; Cha, J.H.; Kim, S.M.; Kim, S.J.; Im, J.-G. Screening Mammography–detected Cancers: Sensitivity of a Computer-aided Detection System Applied to Full-Field Digital Mammograms. Radiology 2007, 244, 104–111. [Google Scholar] [CrossRef]
  29. Chen, W.; Liu, B.; Peng, S.; Sun, J.; Qiao, X. Computer-Aided Grading of Gliomas Combining Automatic Segmentation and Radiomics. Int. J. Biomed. Imaging 2018, 2018, e2512037. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  30. Lo, C.-M.; Weng, R.-C.; Cheng, S.-J.; Wang, H.-J.; Hsieh, K.L.-C. Computer-aided diagnosis of isocitrate dehydrogenase genotypes in glioblastomas from radiomic patterns. Medicine 2020, 99, e19123. [Google Scholar] [CrossRef]
  31. Suter, Y.; Knecht, U.; Alão, M.; Valenzuela, W.; Hewer, E.; Schucht, P.; Wiest, R.; Reyes, M. Radiomics for glioblastoma survival analysis in pre-operative MRI: Exploring feature robustness, class boundaries, and machine learning techniques. Cancer Imaging 2020, 20, 55. [Google Scholar] [CrossRef]
  32. Belfiore, M.P.; Sansone, M.; Monti, R.; Marrone, S.; Fusco, R.; Nardone, V.; Grassi, R.; Reginelli, A. Robustness of Radiomics in Pre-Surgical Computer Tomography of Non-Small-Cell Lung Cancer. J. Pers. Med. 2023, 13, 83. [Google Scholar] [CrossRef]
  33. Fusco, R.; Piccirillo, A.; Sansone, M.; Granata, V.; Rubulotta, M.R.; Petrosino, T.; Barretta, M.L.; Vallone, P.; Di Giacomo, R.; Esposito, E.; et al. Radiomics and Artificial Intelligence Analysis with Textural Metrics Extracted by Contrast-Enhanced Mammography in the Breast Lesions Classification. Diagnostics 2021, 11, 815. [Google Scholar] [CrossRef]
  34. Angelone, F.; Ricciardi, C.; Gatta, G.; Sansone, M.; Ponsiglione, A.M.; Belfiore, M.P.; Amato, F.; Grassi, R. Breast Density Analysis on Mammograms: Application of Machine Learning with Textural Features. In Proceedings of the 2022 IEEE International Conference on Metrology for Extended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE), Rome, Italy, 26–28 October 2022; pp. 295–300. [Google Scholar]
  35. Sansone, M.; Ponsiglione, A.M.; Angelone, F.; Amato, F.; Grassi, R. Effect of X-ray scatter correction on the estimation of attenuation coefficient in mammography: A simulation study. In Proceedings of the 2022 IEEE International Conference on Metrology for Extended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE), Rome, Italy, 26–28 October 2022; pp. 323–328. [Google Scholar]
  36. Sansone, M.; Fusco, R.; Grassi, F.; Gatta, G.; Belfiore, M.P.; Angelone, F.; Ricciardi, C.; Ponsiglione, A.M.; Amato, F.; Galdiero, R.; et al. Machine Learning Approaches with Textural Features to Calculate Breast Density on Mammography. Curr. Oncol. Tor. Ont 2023, 30, 839–853. [Google Scholar] [CrossRef] [PubMed]
  37. Costa, M.G.F.; Campos, J.P.M.; de Aquino e Aquino, G.; de Albuquerque Pereira, W.C.; Costa Filho, C.F.F. Evaluating the performance of convolutional neural networks with direct acyclic graph architectures in automatic segmentation of breast lesion in US images. BMC Med. Imaging 2019, 19, 85. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  38. Fleury, E.; Marcomini, K. Performance of machine learning software to classify breast lesions using BI-RADS radiomic features on ultrasound images. Eur. Radiol. Exp. 2019, 3, 34. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  39. Mahmood, T.; Li, J.; Pei, Y.; Akhtar, F.; Imran, A.; Yaqub, M. An Automatic Detection and Localization of Mammographic Microcalcifications ROI with Multi-Scale Features Using the Radiomics Analysis Approach. Cancers 2021, 13, 5916. [Google Scholar] [CrossRef] [PubMed]
  40. Shen, L.; Margolies, L.R.; Rothstein, J.H.; Fluder, E.; McBride, R.; Sieh, W. Deep Learning to Improve Breast Cancer Detection on Screening Mammography. Sci. Rep. 2019, 9, 12495. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  41. Wang, Y.; Aghaei, F.; Zarafshani, A.; Qiu, Y.; Qian, W.; Zheng, B. Computer-aided Classification of Mammographic Masses Using Visually Sensitive Image Features. J. X-ray Sci. Technol. 2017, 25, 171–186. [Google Scholar] [CrossRef]
  42. Sawyer-Lee, R.; Gimenez, F.; Hoogi, A.; Rubin, D. Curated Breast Imaging Subset of Digital Database for Screening Mammography (CBIS-DDSM). 2016. Available online: https://wiki.cancerimagingarchive.net/x/lZNXAQ (accessed on 1 May 2023).
  43. Lee, R.S.; Gimenez, F.; Hoogi, A.; Miyake, K.K.; Gorovoy, M.; Rubin, D.L. A curated mammography data set for use in computer-aided detection and diagnosis research. Sci. Data 2017, 4, 170177. [Google Scholar] [CrossRef] [Green Version]
  44. Clark, K.; Vendt, B.; Smith, K.; Freymann, J.; Kirby, J.; Koppel, P.; Moore, S.; Phillips, S.; Maffitt, D.; Pringle, M.; et al. The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository. J. Digit. Imaging 2013, 26, 1045–1057. [Google Scholar] [CrossRef] [Green Version]
  45. van Griethuysen, J.J.M.; Fedorov, A.; Parmar, C.; Hosny, A.; Aucoin, N.; Narayan, V.; Beets-Tan, R.G.H.; Fillion-Robin, J.-C.; Pieper, S.; Aerts, H.J.W.L. Computational Radiomics System to Decode the Radiographic Phenotype. Cancer Res. 2017, 77, e104–e107. [Google Scholar] [CrossRef] [Green Version]
  46. Tripathy, S.; Swarnkar, T. Unified Preprocessing and Enhancement Technique for Mammogram Images. Procedia Comput. Sci. 2020, 167, 285–292. [Google Scholar] [CrossRef]
  47. Zuiderveld, K. Contrast Limited Adaptive Histogram Equalization; Elsevier: Amsterdam, The Netherlands, 1994; pp. 474–485. [Google Scholar]
  48. van den Boomgaard, R.; van Balen, R. Methods for fast morphological image transforms using bitmapped binary images. CVGIP Graph. Model. Image Process. 1992, 54, 252–258. [Google Scholar] [CrossRef]
  49. Gonzalez, R.C.; Woods, R.E.; Eddins, S.L. Digital Image Processing Using MATLAB. 3; Gatesmark Publishing: Knoxville, Tennessee, 2020. [Google Scholar]
  50. Mendel, K.R.; Li, H.; Lan, L.; Cahill, C.M.; Rael, V.; Abe, H.; Giger, M.L. Quantitative texture analysis: Robustness of radiomics across two digital mammography manufacturers’ systems. J. Med. Imaging 2018, 5, 011002. [Google Scholar] [CrossRef] [PubMed]
  51. Robinson, K.; Li, H.; Lan, L.; Schacht, D.; Giger, M. Radiomics robustness assessment and classification evaluation: A two-stage method demonstrated on multivendor FFDM. Med. Phys. 2019, 46, 2145–2156. [Google Scholar] [CrossRef] [PubMed]
  52. Sansone, M.; Grassi, R.; Belfiore, M.P.; Gatta, G.; Grassi, F.; Pinto, F.; La Casella, G.V.; Fusco, R.; Cappabianca, S.; Granata, V.; et al. Radiomic features of breast parenchyma: Assessing differences between FOR PROCESSING and FOR PRESENTATION digital mammography. Insights Imaging 2021, 12, 147. [Google Scholar] [CrossRef]
  53. Rashid, H.U.; Ibrikci, T.; Paydaş, S.; Binokay, F.; Çevik, U. Analysis of breast cancer classification robustness with radiomics feature extraction and deep learning techniques. Expert Syst. 2022, 39, e13018. [Google Scholar] [CrossRef]
  54. Niu, S.; Jiang, W.; Zhao, N.; Jiang, T.; Dong, Y.; Luo, Y.; Yu, T.; Jiang, X. Intra- and peritumoral radiomics on assessment of breast cancer molecular subtypes based on mammography and MRI. J. Cancer Res. Clin. Oncol. 2022, 148, 97–106. [Google Scholar] [CrossRef]
  55. Militello, C.; Rundo, L.; Dimarco, M.; Orlando, A.; D’Angelo, I.; Conti, V.; Bartolotta, T.V. Robustness Analysis of DCE-MRI-Derived Radiomic Features in Breast Masses: Assessing Quantization Levels and Segmentation Agreement. Appl. Sci. 2022, 12, 5512. [Google Scholar] [CrossRef]
  56. Caballo, M.; Pangallo, D.R.; Mann, R.M.; Sechopoulos, I. Deep learning-based segmentation of breast masses in dedicated breast CT imaging: Radiomic feature stability between radiologists and artificial intelligence. Comput. Biol. Med. 2020, 118, 103629. [Google Scholar] [CrossRef]
  57. Losurdo, L.; Fanizzi, A.; Basile, T.M.A.; Bellotti, R.; Bottigli, U.; Dentamaro, R.; Didonna, V.; Lorusso, V.; Massafra, R.; Tamborra, P.; et al. Radiomics Analysis on Contrast-Enhanced Spectral Mammography Images for Breast Cancer Diagnosis: A Pilot Study. Entropy 2019, 21, 1110. [Google Scholar] [CrossRef] [Green Version]
  58. Wang, S.; Sun, Y.; Mao, N.; Duan, S.; Li, Q.; Li, R.; Jiang, T.; Wang, Z.; Xie, H.; Gu, Y. Incorporating the clinical and radiomics features of contrast-enhanced mammography to classify breast lesions: A retrospective study. Quant. Imaging Med. Surg. 2021, 11, 4418–4430. [Google Scholar] [CrossRef]
  59. Li, H.; Mendel, K.R.; Lan, L.; Sheth, D.; Giger, M.L. Digital Mammography in Breast Cancer: Additive Value of Radiomics of Breast Parenchyma. Radiology 2019, 291, 15–20. [Google Scholar] [CrossRef] [PubMed]
  60. Acciavatti, R.J.; Cohen, E.A.; Maghsoudi, O.H.; Gastounioti, A.; Pantalone, L.; Hsieh, M.-K.; Conant, E.F.; Scott, C.G.; Winham, S.J.; Kerlikowske, K.; et al. Robust radiomic feature selection in digital mammography: Understanding the effect of imaging acquisition physics using phantom and clinical data analysis. In Proceedings of the Medical Imaging 2020: Computer-Aided Diagnosis, Houston, TX, USA, 16–19 February 2020; Volume 11314, p. 113140W. [Google Scholar] [CrossRef]
Figure 1. Graphical representation of the study workflow with main methodological steps. (FO: first order; GLCM: grey-level co-occurrence matrix; GLDM: grey-level dependence matrix; GLRLM: grey-level run length matrix; GLSZM: grey-level size zone matrix; and NGTDM: neighbourhood grey tone difference matrix.)
Figure 1. Graphical representation of the study workflow with main methodological steps. (FO: first order; GLCM: grey-level co-occurrence matrix; GLDM: grey-level dependence matrix; GLRLM: grey-level run length matrix; GLSZM: grey-level size zone matrix; and NGTDM: neighbourhood grey tone difference matrix.)
Jpm 13 01104 g001
Figure 2. Illustration of original and preprocessed MG images in CC view.
Figure 2. Illustration of original and preprocessed MG images in CC view.
Jpm 13 01104 g002
Figure 3. Illustration of original and artificially generated ROI contours. Artificial ROI represents both overestimated and underestimated abnormality segmentations with respect to the original ROI.
Figure 3. Illustration of original and artificially generated ROI contours. Artificial ROI represents both overestimated and underestimated abnormality segmentations with respect to the original ROI.
Jpm 13 01104 g003
Figure 4. Visual comparison of original and artificially generated ROI masks and contours on (a) MLO and (b) CC image view, respectively.
Figure 4. Visual comparison of original and artificially generated ROI masks and contours on (a) MLO and (b) CC image view, respectively.
Jpm 13 01104 g004
Figure 5. Distribution of the overall robustness scores in the different radiomics feature classes calculated for: (a) calcifications, both MLO and CC view considered together; (b) masses, both MLO and CC view considered together; (c) calcifications in CC view only; (d) masses in CC view only; (e) calcifications in MLO view only; (f) masses in MLO view only. (FO: first order; GLCM: grey-level co-occurrence matrix; GLDM: grey-level dependence matrix; GLRLM: grey-level run length matrix; GLSZM: grey-level size zone matrix; and NGTDM: neighbourhood grey tone difference matrix).
Figure 5. Distribution of the overall robustness scores in the different radiomics feature classes calculated for: (a) calcifications, both MLO and CC view considered together; (b) masses, both MLO and CC view considered together; (c) calcifications in CC view only; (d) masses in CC view only; (e) calcifications in MLO view only; (f) masses in MLO view only. (FO: first order; GLCM: grey-level co-occurrence matrix; GLDM: grey-level dependence matrix; GLRLM: grey-level run length matrix; GLSZM: grey-level size zone matrix; and NGTDM: neighbourhood grey tone difference matrix).
Jpm 13 01104 g005
Figure 6. Distribution of the robustness scores in the different radiomics feature classes calculated for: (a) benign and malignant calcifications, both MLO and CC view considered together; (b) benign and malignant masses, both MLO and CC view considered together; (c) benign and malignant calcifications in CC view only; (d) benign and malignant masses in CC view only; (e) benign and malignant calcifications in MLO view only; (f) benign and malignant masses in MLO view only. (FO: first order; GLCM: grey-level Co-Occurrence matrix; GLDM: grey-level dependence matrix; GLRLM: grey-level run length matrix; GLSZM: grey-level size zone matrix; NGTDM: neighbourhood grey tone difference matrix).
Figure 6. Distribution of the robustness scores in the different radiomics feature classes calculated for: (a) benign and malignant calcifications, both MLO and CC view considered together; (b) benign and malignant masses, both MLO and CC view considered together; (c) benign and malignant calcifications in CC view only; (d) benign and malignant masses in CC view only; (e) benign and malignant calcifications in MLO view only; (f) benign and malignant masses in MLO view only. (FO: first order; GLCM: grey-level Co-Occurrence matrix; GLDM: grey-level dependence matrix; GLRLM: grey-level run length matrix; GLSZM: grey-level size zone matrix; NGTDM: neighbourhood grey tone difference matrix).
Jpm 13 01104 g006aJpm 13 01104 g006b
Figure 7. Representative examples of FO radiomics features with different robustness levels (high or low) and different discriminative power (high or low) for: (ad) calcifications; (eh) masses.
Figure 7. Representative examples of FO radiomics features with different robustness levels (high or low) and different discriminative power (high or low) for: (ad) calcifications; (eh) masses.
Jpm 13 01104 g007
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Ponsiglione, A.M.; Angelone, F.; Amato, F.; Sansone, M. A Statistical Approach to Assess the Robustness of Radiomics Features in the Discrimination of Mammographic Lesions. J. Pers. Med. 2023, 13, 1104. https://doi.org/10.3390/jpm13071104

AMA Style

Ponsiglione AM, Angelone F, Amato F, Sansone M. A Statistical Approach to Assess the Robustness of Radiomics Features in the Discrimination of Mammographic Lesions. Journal of Personalized Medicine. 2023; 13(7):1104. https://doi.org/10.3390/jpm13071104

Chicago/Turabian Style

Ponsiglione, Alfonso Maria, Francesca Angelone, Francesco Amato, and Mario Sansone. 2023. "A Statistical Approach to Assess the Robustness of Radiomics Features in the Discrimination of Mammographic Lesions" Journal of Personalized Medicine 13, no. 7: 1104. https://doi.org/10.3390/jpm13071104

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop