Next Article in Journal
Fixed-Point Theorems on Fuzzy Bipolar b-Metric Spaces
Previous Article in Journal
Semi-RainGAN: A Semisupervised Coarse-to-Fine Guided Generative Adversarial Network for Mixture of Rain Removal
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Radiomics and Its Feature Selection: A Review

School of Mathematical Science, Inner Mongolia University, Hohhot 010020, China
*
Author to whom correspondence should be addressed.
Symmetry 2023, 15(10), 1834; https://doi.org/10.3390/sym15101834
Submission received: 9 August 2023 / Revised: 2 September 2023 / Accepted: 6 September 2023 / Published: 27 September 2023
(This article belongs to the Special Issue Symmetry in Medical Image Processing)

Abstract

:
Medical imaging plays an indispensable role in evaluating, predicting, and monitoring a range of medical conditions. Radiomics, a specialized branch of medical imaging, utilizes quantitative features extracted from medical images to describe underlying pathologies, genetic information, and prognostic indicators. The integration of radiomics with artificial intelligence presents innovative avenues for cancer diagnosis, prognosis evaluation, and therapeutic choices. In the context of oncology, radiomics offers significant potential. Feature selection emerges as a pivotal step, enhancing the clinical utility and precision of radiomics. It achieves this by purging superfluous and unrelated features, thereby augmenting model performance and generalizability. The goal of this review is to assess the fundamental radiomics process and the progress of feature selection methods, explore their applications and challenges in cancer research, and provide theoretical and methodological support for future investigations. Through an extensive literature survey, articles pertinent to radiomics and feature selection were garnered, synthesized, and appraised. The paper provides detailed descriptions of how radiomics is applied and challenged in different cancer types and their various stages. The review also offers comparative insights into various feature selection strategies, including filtering, packing, and embedding methodologies. Conclusively, the paper broaches the limitations and prospective trajectories of radiomics.

1. Introduction

Cancer is the second leading cause of death worldwide. The burden of cancer continues to grow, exerting significant physical, emotional, and economic pressure on individuals, families, communities, and health systems [1]. Due to the high degree of tumor heterogeneity and recurrence rate, there is an urgent need for more effective strategies with which to detect and treat cancer. Medical imaging serves as a valuable tool in aiding the diagnosis, management, and monitoring of tumors. Techniques such as X-ray computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), and ultrasound (US) are pivotal in advancing personalized cancer diagnosis and treatment. These imaging modalities offer essential complementary information to genomic, proteomic, and metabolomic technologies, facilitating non-invasive assessments of human tissue properties [2]. Moreover, medical imaging can help identify biomarkers that predict treatment response and clinical outcomes [3]. However, the interpretation of images manually remains subjective, relying heavily on individual experience and qualitative evaluation. To enhance accuracy and reliability, identifying quantitative and reliable image biomarkers to support clinical diagnosis is essential. Radiomics, a methodology that integrates image segmentation, feature extraction, feature analysis, and data mining, has emerged as a promising approach in this domain.
Since its introduction in 2012 [4], the field of radiomics has experienced exponential growth. Although the term “radiomics” was formally coined in that year, its roots can be traced back to earlier methods in medical image texture analysis. These early techniques gradually evolved and became more sophisticated, leading to the formalization of radiomics. This evolution encompasses the integration of more complex algorithms and machine learning models, as well as broader applications in clinical decision-making and patient management. Understanding this technological progression not only showcases the maturity of the field but also sets the stage for future research directions. A keyword search for “radiomics” on PubMed (www.ncbi.nlm.nih.gov/pubmed accessed on 28 August 2023) revealed a rising number of annual publications, as depicted in Figure 1, indicating a growing interest in radiomics. These publications primarily concentrate on the field of radiomics, encompassing the extraction and quantitative analysis of features from medical images. They cover a wide range of scientific studies, including original research, reviews, conference proceedings, and related scholarly works related to radiomics. Collectively, these publications enhance the understanding and advancement of radiomics techniques, particularly in the context of medical imaging and diagnosis.
Radiomics is a fundamental medical technique used in clinical practice to aid in screening, diagnosis [5], treatment decision-making [6], and follow-up. It is a rapidly developing artificial intelligence (AI) technique in medical imaging, capable of extracting a large number of quantitative features from medical images in an objective, reproducible, and high-throughput manner. Using these extracted features, radiomics models, also known as signatures, are developed to interpret diverse clinical phenotypes, encompassing patient genotyping, treatment effectiveness, and clinical outcomes [7,8]. The core principle of radiomics is that medical images contain information relevant to the pathophysiology of certain diseases, which can be quantitatively analyzed to aid in diagnosis and treatment [9]. In oncology, the potential of radiomics lies in its capability to quantitatively evaluate intra-tumor heterogeneity, thereby unveiling phenotypes and microenvironments that might not be evident upon visual inspection [10,11]. Radiomics features, such as tumor shape, size, volume, intensity, and texture, provide information different from or complementary to clinical reports, laboratory tests, genomic, or proteomic analyses [12]. Radiomics features extracted from medical images can be categorized into two types: hand-crafted features obtained through traditional algorithms (such as intensity, shape, texture, and wavelets) and features generated through deep learning (DL) algorithms. Hand-crafted features offer specific information about the regions of interest (ROIs) within medical images, such as tumors or organs, and can be correlated with additional data sources, including clinical, treatment, or genomic data [13]. Meanwhile, DL features are acquired through the training of DL networks. In recent years, DL has gained popularity, primarily due to the exceptional performance of convolutional neural networks (CNNs) in tasks such as image classification and recognition in the field of medical imaging [14,15]. This trend has opened up new opportunities for research in radiomics. CNNs enable the extraction of DL features that capture intra-tumor heterogeneity, facilitating direct information extraction from medical images and providing a more accurate description of tumor characteristics compared to manually designed radiomics features [16,17]. The rationale behind this trend is well-founded, as CNNs have consistently demonstrated remarkable performance in medical imaging, offering strong support for their application in radiomics research. The advantage of DL is its ability to bypass the need for extensive data processing, making it more efficient. However, a notable limitation of DL is its “black box” nature; the resulting models and features often lack interpretability, which can impede their direct applicability in clinical scenarios. Additionally, DL methods are efficient when applied to large datasets. However, the sample size, especially in rare diseases, is not always sufficient to fully utilize these architectures [18]. Therefore, transfer learning is proposed to bridge this gap. Transfer learning involves using pre-trained models from images in other domains and fine-tuning the parameters of the model on the target dataset to make it applicable to the target task [19,20]. This process is widely used in the field of medical image analysis [21]. By combining both hand-crafted features and DL features with other patient data, such as pathology results, clinical records, and genomic data, more precise and robust predictive models of clinical outcomes can be developed.
Radiomics has gained significant attention in recent years for its ability to identify tumor genotypes and pathological imaging biomarkers, which have clinical potential [22,23]. Studies have indicated that radiomics has the potential to improve or surpass existing tumor diagnosis methods, as it can effectively evaluate genomic features [24,25], tumor subtypes [26,27,28], and lymph node metastasis [29,30,31,32] in various cancers. Radiomics has demonstrated the ability to predict the prognosis of various malignancies, including colorectal cancer [33,34,35,36,37,38], gastric cancer [39,40,41,42,43,44], cervical cancer [31,45,46,47,48], pancreatic cancer [49,50,51], nasopharyngeal cancer [52,53,54,55,56,57], breast cancer [58,59,60,61,62], and lung cancer [63,64,65,66,67,68]. To conduct a comprehensive literature review, we employed the PubMed search engine in combination with an online literature tracking tool named Stork (https://www.storkapp.me/ accessed on 8 March 2023). Stork serves as a researcher’s personalized scientific information assistant, designed to seamlessly provide the most recently published scientific literature based on the researcher’s preferences. This enables real-time engagement with the latest advancements. Additionally, Stork provides a wide range of advanced features. It dynamically acquires and analyzes a significant amount of scientific literature, then presents statistical insights using informative charts and graphs, significantly enhancing user productivity. In our study, we utilized this invaluable tool in conjunction with the PubMed search engine to curate relevant articles in the field of radiomics. Our search included relevant articles up to 8 March 2023, covering a broad spectrum of radiomics research, with a notable focus on lung and breast cancers (Figure 2).
The radiomics workflow comprises five distinct stages, each presenting specific challenges that researchers aim to overcome. Thee are as follows. Image acquisition and pre-processing: Ensuring the quality and consistency of the images before analysis. Image segmentation: Defining the boundaries of the ROIs using various methods, including manual, semi-automatic, and automatic methods. Feature extraction and selection: Extracting quantitative characteristics from the tumor area using specialized open-source or internally created software, followed by selecting a relevant subset of features. This step aims to reduce the number of predictors while retaining essential data for the study results. Model creation and evaluation: Building models involves various techniques, ranging from simple linear regression to complex machine learning methods such as random forests (RF), support vector machines, and neural networks. The evaluation of predictive models typically considers identification, calibration, and potential clinical value. Clinical application: Radiomics assists clinicians in making supplementary diagnoses and in guiding subsequent treatment decisions. In this review, we begin with an overview of each step in the radiomics workflow (Section 2). Subsequently, we comprehensively examine feature selection, which is one of the vital steps in radiomics studies, in Section 3. This section aims to aid researchers in constructing more effective radiomics models by offering insights into various feature selection methods. Section 4 discusses conventional and DL-based radiomics, highlighting the pros and cons of three feature selection methods commonly used in radiomics studies. In addition, it discusses the challenges facing radiomics and introduces the development of radiomics in a new perspective. Lastly, Section 5 presents the conclusions.

2. Radiomics Workflow

The radiomics workflow is divided into five key stages, each with its own specific challenges. Radiomics employs various feature extraction algorithms to transform imaging data from the ROIs into either first-order or higher-order features. This process aims to enhance clinical diagnostic accuracy and prognostic/predictive value by exploring and evaluating data relationships. Figure 3 depicts the fundamental radiomics process.

2.1. Image Acquisition and Pre-Processing

In radiomics, the workflow consists of five critical stages, each presenting unique challenges. The initial stage involves image acquisition using various modalities such as CT, MRI, PET, and US. One of the primary obstacles at this stage is the high variability in image acquisition procedures across different medical centers. This variability can compromise the reproducibility of quantitative features and, by extension, the external validity of radiomics models [69,70]. Given the complexities involved in image normalization, radiomics models must undergo internal and external testing in multiple validation cohorts to ensure accuracy. Therefore, one of the major challenges in radiomics is to reduce the influence of the imaging protocol on radiomics features. Many studies use multicenter data to assess the robustness of radiomics models under different image parameters [71,72,73].
Additionally, post-image acquisition pre-processing can mitigate the impact of certain image acquisition parameters on the model. The future of pre-processing depends on establishing standardized imaging protocols and image preprocessing procedures, including pixel spacing, gray intensity, and gray-level histogram bins. Nevertheless, the substantial variations among different imaging techniques necessitate additional quality control measures to ensure study reproducibility. Various initiatives have been proposed to establish acquisition and reconstruction criteria, aiming to advance quantitative imaging and ensure reliability. For instance, the Radiological Society of North America and the National Institute for Biomedical Imaging and Bioengineering have supported the Quantitative Imaging Biomarkers Alliance (QIBA) [74], and the European Society of Radiology has established the European Imaging Biomarkers Alliance subcommittee (EIBALL) [75].

2.2. Image Segmentation

After collecting a robust image dataset from the target patient cohort, the next crucial step is to accurately segment ROIs or Volumes of Interest (VOIs). Radiomics features are notably influenced not only by image acquisition but also by the variability in ROIs delineation. Precise ROIs segmentation is a pivotal step in radiomics investigations as it significantly affects the derived radiomics features. Segmentation methods include manual, semi-automatic, and fully automatic approaches. Typically, expert clinicians or researchers perform manual ROIs delineation, which serves as the benchmark. However, manual segmentation is resource-intensive and relies on trained practitioners to reduce inter-operator inconsistencies. In cases where lesion boundaries are unclear or multiple focal lesions are present, multiple manual segmentations may be needed in order to ensure accuracy and reproducibility. Despite being time-intensive and susceptible to bias, evaluating the reproducibility of reported features among different observers is essential. Radiomics attributes that lack repeatability should be discarded [76]. Feature consistency within and among observers can be quantified using the Intraclass Correlation Coefficient (ICC) [77]. In summary, following dataset collection, precise segmentation becomes the focal point, as it plays a significant role in the consistency and reproducibility of radiomics features.
Many studies indicate that employing automatic or semi-automatic segmentation techniques can mitigate human errors, enhancing efficiency. There are numerous software programs and segmentation algorithms available for performing semi-automatic and fully automatic segmentation of radiomics images. Examples of segmentation software widely used in medical research include 3D Slicer [78], ITK-SNAP [79], ImageJ [80], and MITK/3DMed [81,82]. Semi-automatic or fully automatic segmentation is appealing because it provides a more reproducible and efficient approach to observing robust features within a given ROIs [83]. While automated methods for medical image segmentation are becoming more accurate, they still depend on manual outlining as the gold standard. There is a growing belief that optimal and reliable segmentation can be achieved through a combination of computer-aided edge detection and manual refinement [84].
Computer-aided inspection systems commonly employ a variety of segmentation techniques, including active contour, level-set, area-based, and graph-based methods [85,86]. Parametric techniques, such as active contour and level-set models, require appropriate initialization and may have limitations when handling uncertain shapes or unknown ROIs locations. Region-based methods, which rely on pixel intensity homogeneity, may encounter challenges in scenarios involving complex anatomical structures or undefined ROIs. In contrast, graph-based methods effectively capture pixel interactions [87], making them suitable for handling complex anatomical structures, albeit with a potential increase in computational load.
Conversely, DL has emerged as a powerful paradigm for medical image segmentation, consistently achieving remarkable results across various applications in medical imaging [88]. Unlike traditional approaches that depend on hand-crafted features and algorithms, DL techniques learn directly from data, capturing intricate patterns and representations that are challenging to express through explicit rules. One key advantage of DL-based segmentation is its adaptability to a wide range of medical imaging tasks.
Prominent DL architectures for medical image segmentation include Unet [89], ST-Unet [90], Unet++ [91], Eres-UNet++ [92], CE-Net [93], and PIPO-FAN [94]. These foundational frameworks excel in segmenting structures and anomalies within medical images. Nevertheless, DL models are specific to particular tasks and necessitate customized training. Similar to the selection of different methods for different contexts, DL networks are specialized and may not perform effectively beyond their trained domain. In the field of medical imaging, misapplication of models can have serious consequences. Therefore, DL-based medical image segmentation systems require a well-considered combination of methodologies capable of adapting to diverse conditions. By understanding the strengths and limitations of models and incorporating them intelligently, medical professionals and researchers can effectively utilize DL for precise image segmentation in scenarios with intricate anatomies or undefined ROIs.

2.3. Feature Extraction and Feature Selection

Following tumor area segmentation, the subsequent critical step involves extracting features from the image. Automated radiomics feature extraction can be accomplished using various open-source software tools, including LIFEx (https://www.lifexsoft.org/index.php accessed on 8 August 2023), MITK (http://www.mitk.org/wiki/MITK accessed on 8 August 2023), CERR (https://github.com/cerr/CERR accessed on 8 August 2023), and others. Alternatively, open-source software packages such as pyRadiomics [95] can be used for feature extraction. In the realm of radiomics, data-driven methodologies eliminate prior assumptions concerning the clinical significance of particular features. Utilizing advanced open-source software tools, these algorithms autonomously identify, compute, and extract features, thereby revealing novel patterns pivotal for both diagnosis and prognosis. Importantly, the choice of software for feature extraction can introduce variability, potentially affecting the consistency and reliability of the resulting data.
In response to this challenge, Zwanenburg et al. introduced the Image Biomarker Standardization Initiative (IBSI) [96], which consists of a standardized set of 169 features. This initiative facilitates the validation and calibration of various radiomics software tools and addresses the variations and inconsistencies in radiological feature extraction methods employed across different studies and clinical settings. Consequently, it enhances the reliability and comparability of radiological image features and provides guidance for future research.
To ensure the reproducibility of radiomics-based models, it is crucial to extract radiomics features from pre-processed imaging data that have been optimized to reduce inconsistencies. Image normalization is a vital technique that ensures consistent distribution of different images within a similar value range. It is necessary for facilitating image comparisons and consistent analysis. Resampling images to the same size facilitates easier comparison and analysis. These strategies mitigate discrepancies caused by different acquisition devices, parameters, and factors. Radiomics features fall into three primary categories:–first-order statistical, shape-based, and texture features—as depicted in Table 1. First-order statistical features analyze the distribution of individual voxel values without considering spatial relationships. They reflect the symmetry, homogeneity, and local intensity distribution variations of the measured voxels.
These include median, mean, minimum, maximum, standard deviation, skewness, kurtosis, and more. Shape features describe the shape of the traced ROI and its geometric properties, including volume, maximum diameter along different orthogonal directions, maximum surface, tumor compactness, and sphericity. For instance, the surface-to-volume ratio of a spiculated tumor will exhibit higher values compared to that of a round tumor with similar volume [97]. Second-order and higher-order texture features can discern or gauge spatial variations in voxel intensity levels by evaluating the spatial dispersion among voxels [98]. Textural features can be categorized into several groups, as depicted in Figure 4.
  • The Gray Level Co-occurrence Matrix (GLCM) represents the probability of a voxel value occurring at a specific direction and distance [99,100];
  • The Gray Level Run Length Matrix (GLRLM) describes the length of consecutive voxels with the same gray level in a specified direction [101,102];
  • The Gray Level Size Zone Matrix (GLSZM) segments an image into regions with contiguous voxel values [103];
  • The Neighboring Gray Tone Difference Matrix (NGTDM) quantifies the gray value of a voxel by considering the difference between its average gray value and the gray value within a certain distance of the neighborhood [104];
  • The Gray Level Dependence Matrix (GLDM) calculates the difference between adjacent voxels based on their values [105].
Additionally, DL offers a meaningful approach to extract deep features. DL models consist of trainable nonlinear operations, referred to as layers, that transform input data into representations for effective pattern recognition. The addition of more layers to the model progressively abstracts the input data, resulting in a deep-feature representation [106]. An illustration of utilizing a DL network to extract features of occult peritoneal metastases from gastric cancer [107] is presented in Figure 5. DL algorithms possess the ability to automatically learn phenotypic features, exhibiting powerful characterization capabilities without requiring predefined characteristics or human intervention [108,109]. It is important to acknowledge that not all extracted features are beneficial for modeling. Therefore, data cleaning plays a crucial role in the process, encompassing error identification and correction in data files, validation of data consistency, and handling invalid and missing values.
Following feature extraction, it becomes essential to choose a subset of these features tailored to a specific disease process or research question. Radiomics features can be categorized based on their relevance to machine learning and their potential to enhance the performance of learning algorithms. The first category comprises relevant features that significantly benefit machine learning and enhance the efficiency of learning algorithms. The second category refers to irrelevant features that do not aid the algorithm and do not contribute to its learning process. Lastly, the third category consists of “redundant features”, which can be inferred from other available information and do not offer new insights to the learning algorithm.
Feature selection is a crucial step in the radiomics workflow, aimed at eliminating irrelevant or redundant features while retaining those that are relevant and useful. This process helps reduce the dimensionality of features, enabling machine learning models to learn more efficiently and reducing the risk of overfitting. The goal of feature selection is to identify a smaller subset of features that are most relevant and predictive for the specific task at hand. Additionally, feature dimensionality reduction involves reducing the dimensionality of the correlation feature matrix, which further enhances the efficiency and effectiveness of the radiomics analysis.

2.4. Model Creation and Evaluation

After selecting the radiomics features, they can be used in various models, such as diagnostic, prognostic, or predictive models [76]. Machine learning has become a prevalent method for radiomics-based predictions, owing to the rapid growth in high-dimensional data availability in recent years. The core concept of machine learning involves training models on input-output examples, allowing the system to learn desired relationships autonomously, without extensive human programming. To achieve this, numerous machine learning classifiers have been developed, employing a diverse range of candidate algorithms [110,111,112,113,114]. These classifiers aim to achieve optimal performance through repeated iterations of training and testing. Several popular classifiers include Extreme Gradient Boosting, RF, Naive Bayes, and Support Vector Machine (SVM). In radiomics studies, evaluating the predictive accuracy of multiple classifiers, feature selection methods, and their combinations is crucial for selecting the most suitable machine learning classifier for a given research problem or dataset [115]. Overfitting and underfitting can impact both statistical learning and machine learning models, resulting in unstable models that lack generalization ability. Employing strategies such as cross-validation, external validation, and multicenter validation during model training can lead to more stable models.
The receiver operating characteristic curve (ROC curve) is a common method used to evaluate the predictive accuracy of radiomics-based models. Moreover, other metrics, such as the area under the ROC curve (AUC), accuracy, precision, recall, and F1 score, are commonly used to assess the performance of the model. Independent validation, including external validation using datasets from different institutions, is a critical step in radiomics studies. Most existing radiomics studies are constrained by small sample sizes from a single institution, limiting the conclusions that can be drawn from them. To overcome this limitation, future radiomics studies should undergo validation and refinement through multiple large-scale, randomized controlled clinical trials. This approach will enhance the accuracy, reliability, and effectiveness of radiomics-based models.

2.5. Clinical Application

Radiomics has broad applications in medical research involving imaging techniques. It aids clinicians in reducing image analysis subjectivity, supporting supplementary diagnosis, and guiding appropriate follow-up treatment. Park et al. performed a meta-analysis to assess the scientific quality of radiomics studies using the Radiomics Quality Score (RQS) and Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD). The analysis showed that 91% of radiomics studies are focused on oncology, and 81% of these studies are utilized for diagnostic purposes [116].

3. Feature Selection Method

Radiomics entails the development of dependable predictive biomarkers, often referred to as features in the context of machine learning. This section offers a comprehensive introduction to the process of feature selection in radiomics and the associated methodology. In the realm of radiomics, we meticulously extract numerous features from medical images. Nonetheless, it is essential to recognize that certain features may not be relevant to the specific outcome of interest. It is worth noting, however, that an excessive number of features can potentially result in overfitting, which may compromise the model’s ability to make accurate predictions. Importantly, the presence of highly correlated or collinear features can have a detrimental impact on machine learning models. This stems from the foundational assumption of most models, which assumes the independence of input features. Faced with substantial correlation or collinearity, models may overly depend on these interconnected features, thereby sidelining other relevant information. This situation inevitably results in a reduction in the model’s ability to effectively generalize to new data. The correlations between features that the model learns from the training data may not apply to new datasets. To enhance the model’s resilience and generalization capabilities, it is often necessary to prune or consolidate closely correlated features during the training phase. This careful curation aims to reduce the model’s tendency to overfit to a specific dataset, ultimately improving its performance when confronted with previously unobserved data [117,118,119].
Feature selection is a vital component of radiomics, as it reduces data complexity by eliminating redundant and irrelevant features, thus enhancing model performance. Various feature selection methods have been proposed, and attempts have been made to compare their performance using data from different sources. However, there is no clear consensus on the best feature selection method for radiomics [120]. A wide range of feature selection methods exists; however, the absence of a uniform reference standard for removing redundant and irrelevant features may introduce a certain level of “arbitrariness” in the feature selection process. A strong understanding of feature selection techniques is essential when developing machine learning models; without it, creating effective and useful models can be challenging.

3.1. The Feature Selection Framework

The feature subset obtained through feature selection should be as small as possible while effectively identifying the target. Additionally, it should ideally improve the accuracy of the prediction model and not reduce it. Furthermore, the class distribution of the resulting subset should closely resemble that of the original data. Dash et al. [121] proposed a search-based framework, as depicted in Figure 6A, for achieving this goal of feature selection. The framework outlines four basic steps of feature selection: candidate feature subset generation (search strategy), subset evaluation, stopping criterion, and result validation. The search strategy involves searching for feature subsets and evaluating them for further analysis. Subset evaluation involves assessing the quality of a subset of features. The stopping criterion is met when the evaluation function reaches a specific benchmark. Finally, result validation involves assessing the accuracy of the selected feature subset on the validation dataset. The search strategy and subset evaluation are vital components of any feature selection method. The process of generating and evaluating subsets is iterated until the desired results are achieved.
Subset search in a search-based framework can be time-consuming, especially with large datasets. Yu et al. [122] proposed an efficient feature selection framework based on correlation and redundancy analysis (Figure 6B). It avoids subset search and quickly identifies the optimal subset. The original feature set can be divided into four subsets: uncorrelated, redundant, weakly correlated but non-redundant, and strongly correlated features [123]. The relevance-based framework identifies the most pertinent features by calculating their redundancy and relevance based on a specific metric. Feature selection algorithms under this framework include Minimum Redundancy Maximum Relevance (mRMR) [124], Correlation-based Feature Selection (CFS) [125], and Fast Correlation-Based Filter solution (FCBF) [122,126].

3.2. Classification of Feature Selection

Classification of feature selection methods can be based on their relationship with the learning algorithm, determining whether they are independent or dependent. Feature selection can be categorized into three distinct categories: filter (Figure 7A), wrapper (Figure 7B), and embedded (Figure 7C) [127,128].
The filter method relies on the intrinsic characteristics of the analyzed data and evaluates features independently of any learning algorithms. The method utilizes metric matrices obtained directly from the data, eliminating the need for feedback from learning algorithms to adjust feature selection. Feature selection is determined by the features’ dispersion or relevance, and unimportant ones are filtered out using a threshold or predetermined number. The growing availability of high-dimensional big data has led to increased interest in filter models, which are not biased towards a specific learner model, so its structure is always simple. Some common filter methods are listed in Table 2.
The wrapper method distinguishes itself from the filter method by utilizing a pre-determined learning algorithm to evaluate and identify a subset of features. Instead of relying on independent metrics, the wrapper method considers the feature selection algorithm as part of the learning process and utilizes the classification performance as an evaluation criterion for feature importance. Evaluating a subset of features with a machine learning algorithm enables the detection of interactions between features, leading to the selection of an optimal feature subset for the model. The wrapper method involves both feature subset search and evaluation metrics. The former generates candidate feature subsets, while the latter trains and evaluates a model with each feature subset using a validation set to identify the optimal one. Subset search is a vital step in wrapper methods, and it employs three types of search methods as depicted in Table 3. Common examples of wrapper methods are Recursive Feature Elimination (RFE) [140] and RF.
Filter methods exhibit computational efficiency, but they do not account for the bias of the learning algorithm. Conversely, wrapper methods offer a more accurate prediction estimate by considering the learning algorithm’s bias, yet they may incur computational expenses. Embedded methods merge the benefits of filter and wrapper methods by integrating feature selection into the model construction process. Embedded methods effectively utilize both filter and wrapper methods: (1) they are significantly less computationally intensive than wrapper methods as they do not necessitate multiple runs of the learning model to evaluate features; and (2) they interact with the learning model. The key distinction between the wrapper and embedded methods lies in the process of utilizing candidate features. The wrapper method initially trains the learning model using candidate features and then evaluates the features through feature selection using the learning model. In contrast, the embedded method selects features during feature model construction without the need for further evaluation through feature selection. Examples of embedded methods are Least Absolute Shrinkage and Selection Operator (LASSO) [152] and Gradient Boosted Decision Trees (GBDT) [153]. A summary of the advantages and disadvantages of the filter, wrapper, and embedded models is presented in Table 4.
Radiomics studies often employ five primary feature selection methods. These are as follows:
  • Distribution Analysis: The Mann–Whitney U-test measures the difference in the distribution of each feature within the positive and negative sample groups. The formula for U i is as follows:
    U i = R i n i ( n i + 1 ) 2 ,
    where i represents the positive or negative group, n i is the data size of group i, and R i is the sum of the ranks in group i. Smaller values of U i are consulted in significant tables to derive a p-value. A smaller p-value indicates that the corresponding feature can effectively distinguish positive and negative samples. Anderson–Darling (AD) test is also a widely used method. The AD test is primarily used for testing whether data conform to a specific distribution, such as the normal or exponential distribution. However, it can also be employed for feature selection in machine learning. When used for feature selection, the goal of the AD test is to assess how effectively the distribution of a given feature distinguishes different categories or outcomes within a dataset. A feature that produces significantly different distributions may be a good candidate for inclusion in a machine learning model. The mathematical formula for the Anderson–Darling test is as follows:
    A 2 = n 1 n i = 1 n ( 2 i 1 ) ln ( F ( x i ) ) + ln ( 1 F ( x n + 1 i ) ) ,
    where n is the sample size, x i represents the i-th ordered observation, and F ( x i ) is the theoretical cumulative distribution function value for x i . Higher values of A 2 indicate a worse fit between the sample data and the chosen distribution. Critical values of A 2 and corresponding p-values are commonly looked up in a distribution table or calculated using software. A smaller p-value, typically below a significance level of 0.05 or 0.01, indicates the rejection of the null hypothesis, suggesting that the data do not come from the specified distribution. A small p-value suggests that the feature can effectively differentiate between different categories.
  • Decorrelation: The Pearson linear correlation coefficient calculates the correlation between each pair of features:
    r = n = 1 N x n i x i ¯ x n j x j ¯ n = 1 N x n i x i ¯ 2 n = 1 N x n j x j ¯ 2 ,
    where x i and x j represent two different features of the patients in the training cohort, and N is the data size of the training cohort.
  • Minimum Redundancy Maximum Relevance (mRMR): The mRMR method selects features that are distant from each other while still being highly correlated with the predicted labels. The method is based on mutual information, defined as follows:
    I ( x ; y ) = p ( x , y ) log p ( x , y ) p ( x ) p ( y ) p ( x , y ) .
    Assuming a total of X features, m 1 of them are selected to create the feature set S ( m 1 ) . The m-th feature can be selected through a stepwise optimization process using the objective function.
    max x j X S m 1 I x j ; y 1 m 1 x i X S m 1 I x j ; x i
    In the equation, y represents the classification variables of the samples in the training cohort, while x i and x j represent distinct features of the patients in the same training cohort.
  • The Least Absolute Shrinkage and Selection Operator (LASSO) is a linear model that incorporates an L 1 -norm regularization to encourage sparse variable coefficients. It selects features with non-zero coefficients to form the final potential descriptor group for each specific task. The optimization objective for LASSO is represented as follows:
    min ω n = 1 N y n ω T x n 2 + λ | ω | 1 .
    Here, x n represents the feature vector of the n-th patient, y n is the classification variable, ω denotes the weight vector of the linear model, and λ > 0 is the normalization parameter [154].
It is important to note that once feature selection is completed, the model is trained. However, if the feature matrix becomes too large after the selection process, it can lead to prolonged computation times and extended training durations, potentially requiring more powerful hardware. Therefore, it becomes essential to reduce the dimensionality of the feature matrix, a process known as feature dimensionality reduction.
Feature dimensionality reduction can enhance the performance of radiomics models [155]. Feature selection and dimensionality reduction serve a common purpose: addressing high dimensionality in data processing, where the number of features in a sample tends to increase linearly with the amount of data to be processed. Feature selection and dimensionality reduction involve substituting high-dimensional features with more suitable and representative ones through specific algorithms. Nevertheless, these two methods differ in their approaches. Dimensionality reduction is mainly accomplished by analyzing the relationships between features, for example, by combining different features to create new ones, thereby altering the original feature space. In contrast, feature selection involves choosing a subset of the original feature dataset without altering the original feature space, and it operates based on an inclusion relationship. Common dimensionality reduction techniques comprise the L 1 -based penalty model (e.g., LASSO), as well as principal component analysis (PCA) [156,157] and linear discriminant analysis (LDA) [158,159], all of which map the original data to a lower-dimensional sample space.
Feature selection serves the purpose of assisting in the construction of radiomics models for disease diagnosis and prediction. The study conducted by Huang et al. [160] stands as an exemplary demonstration of effective feature selection in radiomics research, offering valuable insights into the selection and application of radiomics features in specific cancer cases. In this study, radiomics was applied to early-stage non-small-cell lung cancer (NSCLC) cases, aiming to construct a robust model for predicting disease-free survival (DFS). Feature selection was a vital step, meticulously executed to identify the most informative radiomics features associated with DFS. The study assembled a cohort consisting of early-stage NSCLC patients and created an extensive dataset encompassing CT and PET images. Using specialized image analysis software, a diverse range of radiomics features, encompassing shape, texture, and intensity attributes, were extracted from the images of each patient. The research employed the following strategies for feature selection: Initially, an exploration phase involved calculating Pearson correlation coefficients to assess the relationship between individual radiomic features and DFS. This analysis offered insights into the extent of correlation between features and DFS. Subsequently, further refinement of the feature set was accomplished through LASSO regression, which, owing to its capacity to induce sparsity in coefficients, identified the most predictive radiomics features. After feature selection, a predictive model for estimating early-stage NSCLC patient DFS was constructed. Rigorous validation procedures, including cross-validation, were employed to assess the performance of the model. The thorough application of the feature selection process ultimately defined a distinctive set of radiomics features known as the “radiomics signature”. This signature succinctly summarized radiomic features strongly correlated with the prediction of DFS in early-stage NSCLC patients. Significantly, during model validation, the “radiomics signature” exhibited strong predictive performance. Through the integration of correlation analysis and LASSO regression, researchers successfully identified a subset of radiomics features possessing significant prognostic value. The established radiomics signature has substantial potential for improving prognostic accuracy, consequently enhancing clinical decision-making and patient management capabilities in early-stage NSCLC cases.

4. Discussion

Traditional radiomics involves extracting numerous hand-crafted features from ROIs and then performing feature selection to identify those relevant to a specific task. This method, while somewhat effective in various applications, is criticized for introducing human bias into the feature computation process. In contrast, DL-based radiomics directly extracts features from images, eliminating the need for intermediate operations in feature computation and reducing the risk of information loss or additional errors. Various deep network architectures can be designed to meet specific requirements. The hierarchical structure of deep neural networks facilitates the discovery of deeper and higher-level features, in contrast to the shallow features typically identified in traditional machine learning algorithms [161]. Previous studies have shown that DL methods demonstrate high accuracy and offer promising clinical prospects [41,162,163]. However, Zhang et al. [164] have highlighted that black-box algorithms may potentially hinder the reliability of their clinical applicability.
Radiomics data are unique compared to other high-dimensional datasets since they arise from imaging data and frequently exhibit strong correlations [165]. Therefore, reducing the number of features in radiomics analysis is crucial. Feature selection methods can be categorized into three groups, each having its specific advantages and disadvantages. Features selected by wrapper-based and embedded techniques may not generalize well to other classifiers because they rely on specific learning algorithms for feature selection. Filter-based techniques have lower computational complexity compared to embedded and wrapper-based techniques. The filter method directly selects features and has lower computational complexity than the embedded and wrapper methods, leading to a more stable feature set selection. However, as it does not interact with the learning algorithm, its accuracy may not be as high as the other two methods [166,167]. Various feature selection methods are available, and selecting the appropriate algorithm depends on various considerations. Researchers must have a comprehensive understanding of feature selection methods. However, the runtime feasibility of these methods can be a concern when dealing with high-dimensional datasets, as highlighted by Guyon et al. [168]. To achieve high performance while avoiding excessive complexity, researchers are advised to consider feature selection methods such as ANOVA, LASSO, and mRMR. If these three methods fail to produce the desired results, researchers should explore additional approaches.
Radiomics has shown its versatility across various medical conditions, with a particular focus on tumor diseases, but its application varies between tumor diseases and other medical conditions. In the context of tumor diseases, radiomics plays a crucial role in characterizing tumors, aiding in diagnosis, and assessing treatment response. For example, in lung cancer, radiomic features extracted from CT or PET images can determine tumor characteristics, differentiation, and even predict treatment outcomes and patient prognosis. In contrast, when applied to non-tumor conditions such as tuberculosis, radiomics assists in identifying pulmonary abnormalities and supporting diagnostics. Similarly, in pancreatitis, radiomics contributes to evaluating pancreatic morphology and density, aiding precise disease classification. Additionally, in stroke cases, radiomics reveals the extent and severity of brain lesions, providing valuable insights for tailored treatment strategies and predicting potential recovery paths. While radiomics holds significance in both contexts, its focus shifts from comprehensive characterization and prediction in tumor diseases to enhancing diagnostic accuracy and providing insights into disease manifestation in non-tumor conditions. By addressing these differences, radiomics demonstrates its adaptability and potential to transform medical decision-making in diverse clinical scenarios.
The integration of imaging data with radiomics offers promise in advancing cancer research by enhancing screening, diagnosis, treatment decision-making, and follow-up [169]. However, despite the potential benefits of radiomics-based methods in cancer imaging, challenges related to their reproducibility and scalability persist. This paper aims to review the challenges associated with radiomics and provides an outlook to serve as a reference for future radiomics studies.
Radiomics faces various challenges, including data acquisition, repeatability, and reproducibility of radiomic features. Retrospective datasets are more accessible, making retrospective radiomics studies common [170]. However, utilizing retrospective data presents challenges regarding data accuracy for model training and validation. Pre-defined inclusion and exclusion criteria could introduce implicit biases in AI algorithms [171]. Furthermore, retrospective data are susceptible to non-harmonization issues due to the use of instruments from different providers, resulting in varying acquisition parameters, especially in multi-center studies. These differences in images may yield features influenced solely by technical discrepancies [172]. Sharing data and standardizing acquisition and reconstruction protocols can contribute to more reliable radiomics signatures, testable on external datasets, thereby addressing this concern [173]. To ensure result reproducibility, the IBSI was introduced to standardize the extraction of image biomarkers from acquired imaging. Nonetheless, conducting prospective studies is more demanding and time-consuming than retrospective studies, requiring long-term approval from pharmaceutical companies or cancer-collaborating organizations. Overall, combining data sharing with standardization of acquisition and reconstruction protocols may offer a potential solution to this issue and aid in identifying more robust radiomics features.
To achieve widespread acceptance, image biomarkers must address the challenge of interpretability, which poses an additional obstacle for radiomics. Radiomics models based on DL are often perceived as “black boxes” by clinicians, accurately predicting specific clinical outcomes but lacking interpretable explanations. To tackle this challenge, ongoing developments in radiomics techniques aim to merge the strengths of DL with the interpretability provided by hand-crafted methods. Interpretability is critical, particularly when evaluating image biomarkers for optimal therapy, as decisions guided by these biomarkers demand an interpretation rooted in pathophysiology [170]. Therefore, enhancing the interpretability of these models remains an ongoing area of research, with researchers exploring various tools [174].
The radiomics protocol, commonly used in cancer diagnosis, follows a signal-to-image-to-diagnosis approach. Accurately reconstructing visible images from the signal (raw data) obtained from medical devices is crucial for physicians to make accurate diagnoses. However, current image-based diagnostics often fail to fully utilize the abundant data in the signal, resulting in inadequate performance [175]. For instance, in CT imaging, the CT system collects raw data from the patient, which is then used to reconstruct images (signal-to-image). As a result, both AI-based and human-based diagnostic processes adopt a signal-to-image-to-knowledge approach. Medical images are prone to information distortion during both the acquisition and reconstruction processes [176]. Consequently, a novel approach to medical image analysis has emerged, emphasizing direct analysis of the raw data. Dong et al. [175] introduced an AI-based diagnostic scheme for pulmonary nodules that bypasses the reconstruction step and directly diagnoses from the signal. This groundbreaking research challenges conventional image-based diagnostics and may pave the way for a new era of AI-based diagnostics, offering a fresh direction for radiomics exploration.

5. Conclusions

Radiomics has showcased its diverse applications across various medical conditions, with a particular emphasis on tumor diseases. However, the application of radiomics significantly differs between tumor diseases and other medical conditions. In the context of tumor diseases, radiomics serves a pivotal role in characterizing tumors, aiding in diagnosis, and evaluating treatment response. For instance, in lung cancer, radiomics features extracted from CT or PET images facilitate the determination of tumor characteristics, differentiation, and even the prediction of treatment outcomes and patient prognosis. Conversely, the application of radiomics in non-tumor conditions diverges, primarily functioning as an assistive diagnostic tool. In diseases such as tuberculosis, radiomics aids in identifying pulmonary abnormalities and supporting diagnostic efforts. Similarly, in pancreatitis, radiomics contributes to assessing pancreatic morphology and density, aiding clinicians in precise disease classification. Moreover, in stroke cases, radiomics reveals the extent and severity of brain lesions, offering valuable insights for tailored treatment strategies and predicting potential recovery trajectories. While radiomics holds a profound influence in both contexts, its divergence lies in the specific clinical objectives. In tumor diseases, it serves as a comprehensive tool for characterization and prediction, whereas in non-tumor conditions, its emphasis shifts toward enhancing diagnostic accuracy and offering insights into disease manifestation. By bridging these differences, radiomics demonstrates its adaptability and potential to revolutionize medical decision-making in diverse clinical scenarios.
Radiomics, which encompasses intricate manual techniques and cutting-edge DL methods, represents a promising realm for the non-invasive identification of precise disease-related features. Its potential to support clinicians across tasks spanning from screening and diagnosis to treatment decision-making and post-treatment monitoring is indisputable. However, the truly groundbreaking facet of our study lies in the exhaustive exploration of the radiomics process. Commencing with the framework and categorization of feature selection methods, we unveil an array of techniques commonly employed in radiomics research. This panoramic perspective serves as a pivotal cornerstone for future advancements in the field. The central focus of this paper revolves around a pivotal aspect of radiomics: feature selection. This critical phase significantly enhances the performance and adaptability of radiomics models, thereby augmenting their clinical utility. Our study meticulously delves into diverse feature selection strategies, ranging from meticulous noise filtration to the strategic amalgamation of relevant features and even the introduction of features through embedding techniques. This comprehensive elucidation provides essential insights to guide researchers in making informed decisions amidst the intricate landscape of radiomics research. However, certain limitations within this study warrant consideration. First, despite our in-depth scrutiny of the radiomics process, there is room for further refinement and optimization of the application and effectiveness of feature selection methods. Second, while we discuss various feature selection strategies, the optimal selections across diverse disease types and clinical contexts require reinforcement through more empirical studies. Future endeavors should address these concerns to further amplify the applicability and advantages of radiomics within clinical practice. Notwithstanding the promising prospects of radiomics, researchers should remain attentive to potential obstacles lying ahead. Challenges pertaining to data quality, sample size, and the robustness of generalization capabilities demand attention. Furthermore, the urgency of developing standardized and validated radiomics studies cannot be underestimated. To surmount these challenges, a clear trajectory emerges: forthcoming studies must adhere to rigorous methodologies. This encompasses not only standardizing data acquisition and analysis procedures but also validating across diverse datasets. These collaborative efforts will assuredly fortify the position of radiomics within clinical practice and pave the way for enhanced patient care. In summation, situated at the crossroads of medical imaging and artificial intelligence, radiomics presents unprecedented opportunities within oncology research and clinical applications. As we overcome challenges, we are poised to propel radiomics to the forefront, ushering in a new chapter in patient well-being and the advancement of medical science.

Author Contributions

Conceptualization, W.Z., Y.G. and Q.J.; Data curation, W.Z.; Formal analysis, W.Z. and Q.J.; Funding acquisition, Q.J.; Investigation, W.Z.; Methodology, W.Z., Y.G. and Q.J.; Project administration, W.Z. and Q.J.; Resources, W.Z.; Software, W.Z. and Y.G.; Supervision, Q.J.; Validation, Q.J.; Visualization, W.Z.; Writing—original draft, W.Z., Y.G. and Q.J.; Writing—review & editing, W.Z., Y.G. and Q.J. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the National Natural Science Foundation of China (No. 12061052), the Young Talents of Science and Technology in Universities of Inner Mongolia Autonomous Region (No. NJYT22090), the Natural Science Fund of Inner Mongolia Autonomous Region (No. 2020MS01002), the Innovative Research Team in Universities of Inner Mongolia Autonomous Region (No. NMGIRT2207), Inner Mongolia University Independent Research Project (2022-ZZ004), and the Network Information Center of Inner Mongolia University.

Data Availability Statement

No new data were created or analyzed in this study. Data sharing is not applicable to this article.

Acknowledgments

The authors would like to express their sincere appreciation to Caiying Wu, Guoqing Chen, and Xuemei Wang for their invaluable contributions to this research. Special gratitude is extended to Xuemei Wang for the generous support from the Inner Mongolia Autonomous Region Major Special Project Fund and Chen Guoqing’s “111 Project” of Higher Education Talent Training in Inner Mongolia Autonomous Region program, which significantly contributed to the success of this work. Xuemei Wang’s expert insights and guidance have greatly enhanced the credibility and professionalism of our research. Additionally, we would like to acknowledge the editors and technical staff at MDPI for their understanding and unwavering support throughout the preparation of this paper. Our sincere thanks are also extended to the reviewers for their insightful and pertinent comments, which have enriched the quality of our work.

Conflicts of Interest

The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

References

  1. World Health Organization. Cancer—Key Facts. 2023. Available online: https://www.who.int/news-room/fact-sheets/detail/cancer (accessed on 8 March 2023).
  2. Lambin, P.; Van Stiphout, R.G.; Starmans, M.H.; Rios-Velazquez, E.; Nalbantov, G.; Aerts, H.J.; Roelofs, E.; Van Elmpt, W.; Boutros, P.C.; Granone, P.; et al. Predicting outcomes in radiation oncology—Multifactorial decision support systems. Nat. Rev. Clin. Oncol. 2013, 10, 27–40. [Google Scholar] [CrossRef] [PubMed]
  3. Chen, Q.; Zhang, L.; Liu, S.; You, J.; Chen, L.; Jin, Z.; Zhang, S.; Zhang, B. Radiomics in precision medicine for gastric cancer: Opportunities and challenges. Eur. Radiol. 2022, 32, 5852–5868. [Google Scholar] [CrossRef] [PubMed]
  4. Lambin, P.; Rios-Velazquez, E.; Leijenaar, R.; Carvalho, S.; Van Stiphout, R.G.; Granton, P.; Zegers, C.M.; Gillies, R.; Boellard, R.; Dekker, A.; et al. Radiomics: Extracting more information from medical images using advanced feature analysis. Eur. J. Cancer 2012, 48, 441–446. [Google Scholar] [CrossRef]
  5. Aerts, H.J.; Velazquez, E.R.; Leijenaar, R.T.; Parmar, C.; Grossmann, P.; Carvalho, S.; Bussink, J.; Monshouwer, R.; Haibe-Kains, B.; Rietveld, D.; et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat. Commun. 2014, 5, 4006. [Google Scholar] [CrossRef] [PubMed]
  6. Hood, L.; Friend, S.H. Predictive, personalized, preventive, participatory (P4) cancer medicine. Nat. Rev. Clin. Oncol. 2011, 8, 184–187. [Google Scholar] [CrossRef]
  7. Shen, C.; Liu, Z.; Guan, M.; Song, J.; Lian, Y.; Wang, S.; Tang, Z.; Dong, D.; Kong, L.; Wang, M.; et al. 2D and 3D CT radiomics features prognostic performance comparison in non-small cell lung cancer. Transl. Oncol. 2017, 10, 886–894. [Google Scholar] [CrossRef]
  8. Peng, H.; Dong, D.; Fang, M.J.; Li, L.; Tang, L.L.; Chen, L.; Li, W.F.; Mao, Y.P.; Fan, W.; Liu, L.Z.; et al. Prognostic value of deep learning PET/CT-based radiomics: Potential role for future individual induction chemotherapy in advanced nasopharyngeal carcinoma. Clin. Cancer Res. 2019, 25, 4271–4279. [Google Scholar] [CrossRef]
  9. Gillies, R.J.; Kinahan, P.E.; Hricak, H. Radiomics: Images are more than pictures, they are data. Radiology 2016, 278, 563–577. [Google Scholar] [CrossRef]
  10. Chen, Q.; Xia, T.; Zhang, M.; Xia, N.; Liu, J.; Yang, Y. Radiomics in stroke neuroimaging: Techniques, applications, and challenges. Aging Dis. 2021, 12, 143. [Google Scholar] [CrossRef]
  11. Liu, Z.; Wang, S.; Dong, D.; Wei, J.; Fang, C.; Zhou, X.; Sun, K.; Li, L.; Li, B.; Wang, M.; et al. The applications of radiomics in precision diagnosis and treatment of oncology: Opportunities and challenges. Theranostics 2019, 9, 1303. [Google Scholar] [CrossRef]
  12. Wang, Y.; Zhang, L.; Qi, L.; Yi, X.; Li, M.; Zhou, M.; Chen, D.; Xiao, Q.; Wang, C.; Pang, Y.; et al. Machine learning: Applications and advanced progresses of radiomics in endocrine neoplasms. J. Oncol. 2021, 2021, 8615450. [Google Scholar] [CrossRef] [PubMed]
  13. Gatenby, R.A.; Grove, O.; Gillies, R.J. Quantitative imaging in cancer evolution and ecology. Radiology 2013, 269, 8–14. [Google Scholar] [CrossRef] [PubMed]
  14. Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. Commun. ACM 2017, 60, 84–90. [Google Scholar] [CrossRef]
  15. Soffer, S.; Ben-Cohen, A.; Shimon, O.; Amitai, M.M.; Greenspan, H.; Klang, E. Convolutional neural networks for radiologic images: A radiologist’s guide. Radiology 2019, 290, 590–606. [Google Scholar] [CrossRef] [PubMed]
  16. Zhao, X.; Liang, Y.J.; Zhang, X.; Wen, D.X.; Fan, W.; Tang, L.Q.; Dong, D.; Tian, J.; Mai, H.Q. Deep learning signatures reveal multiscale intratumor heterogeneity associated with biological functions and survival in recurrent nasopharyngeal carcinoma. Eur. J. Nucl. Med. Mol. Imaging 2022, 49, 2972–2982. [Google Scholar] [CrossRef]
  17. Lambin, P.; Zindler, J.; Vanneste, B.G.; Van De Voorde, L.; Eekers, D.; Compter, I.; Panth, K.M.; Peerlings, J.; Larue, R.T.; Deist, T.M.; et al. Decision support systems for personalized and participative radiation oncology. Adv. Drug Deliv. Rev. 2017, 109, 131–153. [Google Scholar] [CrossRef]
  18. Guiot, J.; Vaidyanathan, A.; Deprez, L.; Zerka, F.; Danthine, D.; Frix, A.N.; Lambin, P.; Bottari, F.; Tsoutzidis, N.; Miraglio, B.; et al. A review in radiomics: Making personalized medicine a reality via routine imaging. Med. Res. Rev. 2022, 42, 426–440. [Google Scholar] [CrossRef]
  19. Yosinski, J.; Clune, J.; Bengio, Y.; Lipson, H. How transferable are features in deep neural networks? Adv. Neural Inf. Process. Syst. 2014, 27, 33203328. [Google Scholar]
  20. Sheng, J.; Wumaier, A.; Li, Z. POISE: Efficient Cross-Domain Chinese Named Entity Recognization via Transfer Learning. Symmetry 2020, 12, 1673. [Google Scholar] [CrossRef]
  21. Christodoulidis, S.; Anthimopoulos, M.; Ebner, L.; Christe, A.; Mougiakakou, S. Multisource transfer learning with convolutional neural networks for lung pattern analysis. IEEE J. Biomed. Health Inform. 2016, 21, 76–84. [Google Scholar] [CrossRef]
  22. Lambin, P.; Leijenaar, R.T.; Deist, T.M.; Peerlings, J.; De Jong, E.E.; Van Timmeren, J.; Sanduleanu, S.; Larue, R.T.; Even, A.J.; Jochems, A.; et al. Radiomics: The bridge between medical imaging and personalized medicine. Nat. Rev. Clin. Oncol. 2017, 14, 749–762. [Google Scholar] [CrossRef] [PubMed]
  23. Dong, D.; Zhang, F.; Zhong, L.Z.; Fang, M.J.; Huang, C.L.; Yao, J.J.; Sun, Y.; Tian, J.; Ma, J.; Tang, L.L. Development and validation of a novel MR imaging predictor of response to induction chemotherapy in locoregionally advanced nasopharyngeal cancer: A randomized controlled trial substudy (NCT01245959). BMC Med. 2019, 17, 190. [Google Scholar] [CrossRef] [PubMed]
  24. Wang, S.; Shi, J.; Ye, Z.; Dong, D.; Yu, D.; Zhou, M.; Liu, Y.; Gevaert, O.; Wang, K.; Zhu, Y.; et al. Predicting EGFR mutation status in lung adenocarcinoma on computed tomography image using deep learning. Eur. Respir. J. 2019, 53. [Google Scholar] [CrossRef]
  25. Tian, P.; He, B.; Mu, W.; Liu, K.; Liu, L.; Zeng, H.; Liu, Y.; Jiang, L.; Zhou, P.; Huang, Z.; et al. Assessing PD-L1 expression in non-small cell lung cancer and predicting responses to immune checkpoint inhibitors using deep learning on computed tomography images. Theranostics 2021, 11, 2098. [Google Scholar] [CrossRef] [PubMed]
  26. Zhang, S.; Song, G.; Zang, Y.; Jia, J.; Wang, C.; Li, C.; Tian, J.; Dong, D.; Zhang, Y. Non-invasive radiomics approach potentially predicts non-functioning pituitary adenomas subtypes before surgery. Eur. Radiol. 2018, 28, 3692–3701. [Google Scholar] [CrossRef]
  27. Zhu, X.; Dong, D.; Chen, Z.; Fang, M.; Zhang, L.; Song, J.; Yu, D.; Zang, Y.; Liu, Z.; Shi, J.; et al. Radiomic signature as a diagnostic factor for histologic subtype classification of non-small cell lung cancer. Eur. Radiol. 2018, 28, 2772–2778. [Google Scholar] [CrossRef]
  28. Gong, L.; Xu, M.; Fang, M.; Zou, J.; Yang, S.; Yu, X.; Xu, D.; Zhou, L.; Li, H.; He, B.; et al. Noninvasive prediction of high-grade prostate cancer via biparametric MRI radiomics. J. Magn. Reson. Imaging 2020, 52, 1102–1109. [Google Scholar] [CrossRef]
  29. Li, J.; Dong, D.; Fang, M.; Wang, R.; Tian, J.; Li, H.; Gao, J. Dual-energy CT–based deep learning radiomics can improve lymph node metastasis risk prediction for gastric cancer. Eur. Radiol. 2020, 30, 2324–2333. [Google Scholar] [CrossRef]
  30. Kan, Y.; Dong, D.; Zhang, Y.; Jiang, W.; Zhao, N.; Han, L.; Fang, M.; Zang, Y.; Hu, C.; Tian, J.; et al. Radiomic signature as a predictive factor for lymph node metastasis in early-stage cervical cancer. J. Magn. Reson. Imaging 2019, 49, 304–310. [Google Scholar] [CrossRef]
  31. Liu, Y.; Fan, H.; Dong, D.; Liu, P.; He, B.; Meng, L.; Chen, J.; Chen, C.; Lang, J.; Tian, J. Computed tomography-based radiomic model at node level for the prediction of normal-sized lymph node metastasis in cervical cancer. Transl. Oncol. 2021, 14, 101113. [Google Scholar] [CrossRef]
  32. Li, L.; Zhang, J.; Zhe, X.; Tang, M.; Zhang, X.; Lei, X.; Zhang, L. A meta-analysis of MRI-based radiomic features for predicting lymph node metastasis in patients with cervical cancer. Eur. J. Radiol. 2022, 151, 110243. [Google Scholar] [CrossRef]
  33. Huang, Y.q.; Liang, C.h.; He, L.; Tian, J.; Liang, C.s.; Chen, X.; Ma, Z.l.; Liu, Z.y. Development and validation of a radiomics nomogram for preoperative prediction of lymph node metastasis in colorectal cancer. J. Clin. Oncol. 2016, 34, 2157–2164. [Google Scholar] [CrossRef]
  34. Meng, Y.; Zhang, Y.; Dong, D.; Li, C.; Liang, X.; Zhang, C.; Wan, L.; Zhao, X.; Xu, K.; Zhou, C.; et al. Novel radiomic signature as a prognostic biomarker for locally advanced rectal cancer. J. Magn. Reson. Imaging 2018, 48, 605–614. [Google Scholar] [CrossRef] [PubMed]
  35. Yang, L.; Dong, D.; Fang, M.; Zhu, Y.; Zang, Y.; Liu, Z.; Zhang, H.; Ying, J.; Zhao, X.; Tian, J. Can CT-based radiomics signature predict KRAS/NRAS/BRAF mutations in colorectal cancer? Eur. Radiol. 2018, 28, 2058–2067. [Google Scholar] [CrossRef]
  36. Liang, C.; Huang, Y.; He, L.; Chen, X.; Ma, Z.; Dong, D.; Tian, J.; Liang, C.; Liu, Z. The development and validation of a CT-based radiomics signature for the preoperative discrimination of stage I-II and stage III-IV colorectal cancer. Oncotarget 2016, 7, 31401. [Google Scholar] [CrossRef] [PubMed]
  37. Bedrikovetski, S.; Dudi-Venkata, N.N.; Kroon, H.M.; Seow, W.; Vather, R.; Carneiro, G.; Moore, J.W.; Sammour, T. Artificial intelligence for pre-operative lymph node staging in colorectal cancer: A systematic review and meta-analysis. BMC Cancer 2021, 21, 1058. [Google Scholar] [CrossRef]
  38. Jia, L.L.; Zhao, J.X.; Zhao, L.P.; Tian, J.H.; Huang, G. Current status and quality of radiomic studies for predicting KRAS mutations in colorectal cancer patients: A systematic review and meta-analysis. Eur. J. Radiol. 2023, 158, 110640. [Google Scholar] [CrossRef]
  39. Dong, D.; Fang, M.J.; Tang, L.; Shan, X.H.; Gao, J.B.; Giganti, F.; Wang, R.P.; Chen, X.; Wang, X.X.; Palumbo, D.; et al. Deep learning radiomic nomogram can predict the number of lymph node metastasis in locally advanced gastric cancer: An international multicenter study. Ann. Oncol. 2020, 31, 912–920. [Google Scholar] [CrossRef] [PubMed]
  40. Zhang, L.; Dong, D.; Zhong, L.; Li, C.; Hu, C.; Yang, X.; Liu, Z.; Wang, R.; Zhou, J.; Tian, J. Multi-focus network to decode imaging phenotype for overall survival prediction of Gastric cancer patients. IEEE J. Biomed. Health Inform. 2021, 25, 3933–3942. [Google Scholar] [CrossRef]
  41. Li, C.; Qin, Y.; Zhang, W.H.; Jiang, H.; Song, B.; Bashir, M.R.; Xu, H.; Duan, T.; Fang, M.; Zhong, L.; et al. Deep learning-based AI model for signet-ring cell carcinoma diagnosis and chemotherapy response prediction in gastric cancer. Med. Phys. 2022, 49, 1535–1546. [Google Scholar] [CrossRef]
  42. Wang, S.; Feng, C.; Dong, D.; Li, H.; Zhou, J.; Ye, Y.; Liu, Z.; Tian, J.; Wang, Y. Preoperative computed tomography-guided disease-free survival prediction in gastric cancer: A multicenter radiomics study. Med. Phys. 2020, 47, 4862–4871. [Google Scholar] [CrossRef] [PubMed]
  43. Dong, D.; Tang, L.; Li, Z.Y.; Fang, M.J.; Gao, J.B.; Shan, X.H.; Ying, X.J.; Sun, Y.S.; Fu, J.; Wang, X.X.; et al. Development and validation of an individualized nomogram to identify occult peritoneal metastasis in patients with advanced gastric cancer. Ann. Oncol. 2019, 30, 431–438. [Google Scholar] [CrossRef] [PubMed]
  44. Zhang, F.; Wu, G.; Chen, N.; Li, R. The predictive value of radiomics-based machine learning for peritoneal metastasis in gastric cancer patients: A systematic review and meta-analysis. Front. Oncol. 2023, 13, 1196053. [Google Scholar] [CrossRef] [PubMed]
  45. Li, Z.; Li, H.; Wang, S.; Dong, D.; Yin, F.; Chen, A.; Wang, S.; Zhao, G.; Fang, M.; Tian, J.; et al. MR-based radiomics nomogram of cervical cancer in prediction of the lymph-vascular space invasion preoperatively. J. Magn. Reson. Imaging 2019, 49, 1420–1426. [Google Scholar] [CrossRef]
  46. Fang, M.; Kan, Y.; Dong, D.; Yu, T.; Zhao, N.; Jiang, W.; Zhong, L.; Hu, C.; Luo, Y.; Tian, J. Multi-habitat based radiomics for the prediction of treatment response to concurrent chemotherapy and radiation therapy in locally advanced cervical cancer. Front. Oncol. 2020, 10, 563. [Google Scholar] [CrossRef] [PubMed]
  47. Chen, J.; He, B.; Dong, D.; Liu, P.; Duan, H.; Li, W.; Li, P.; Wang, L.; Fan, H.; Wang, S.; et al. Noninvasive CT radiomic model for preoperative prediction of lymph node metastasis in early cervical carcinoma. Br. J. Radiol. 2020, 93, 20190558. [Google Scholar] [CrossRef]
  48. Ren, J.; Li, Y.; Liu, X.Y.; Zhao, J.; He, Y.L.; Jin, Z.Y.; Xue, H.D. Diagnostic performance of ADC values and MRI-based radiomics analysis for detecting lymph node metastasis in patients with cervical cancer: A systematic review and meta-analysis. Eur. J. Radiol. 2022, 156, 110504. [Google Scholar] [CrossRef]
  49. Xie, T.; Wang, X.; Li, M.; Tong, T.; Yu, X.; Zhou, Z. Pancreatic ductal adenocarcinoma: A radiomics nomogram outperforms clinical model and TNM staging for survival estimation after curative resection. Eur. Radiol. 2020, 30, 2513–2524. [Google Scholar] [CrossRef]
  50. Gao, Y.; Cheng, S.; Zhu, L.; Wang, Q.; Deng, W.; Sun, Z.; Wang, S.; Xue, H. A systematic review of prognosis predictive role of radiomics in pancreatic cancer: Heterogeneity markers or statistical tricks? Eur. Radiol. 2022, 32, 8443–8452. [Google Scholar] [CrossRef]
  51. Zhong, J.; Hu, Y.; Xing, Y.; Ge, X.; Ding, D.; Zhang, H.; Yao, W. A systematic review of radiomics in pancreatitis: Applying the evidence level rating tool for promoting clinical transferability. Insights Imaging 2022, 13, 139. [Google Scholar] [CrossRef]
  52. Zhong, L.; Dong, D.; Fang, X.; Zhang, F.; Zhang, N.; Zhang, L.; Fang, M.; Jiang, W.; Liang, S.; Li, C.; et al. A deep learning-based radiomic nomogram for prognosis and treatment decision in advanced nasopharyngeal carcinoma: A multicentre study. EBioMedicine 2021, 70, 103522. [Google Scholar] [CrossRef] [PubMed]
  53. Zhang, L.; Zhou, H.; Gu, D.; Tian, J.; Zhang, B.; Dong, D.; Mo, X.; Liu, J.; Luo, X.; Pei, S.; et al. Radiomic nomogram: Pretreatment evaluation of local recurrence in nasopharyngeal carcinoma based on MR imaging. J. Cancer 2019, 10, 4217. [Google Scholar] [CrossRef] [PubMed]
  54. Zhang, B.; Tian, J.; Dong, D.; Gu, D.; Dong, Y.; Zhang, L.; Lian, Z.; Liu, J.; Luo, X.; Pei, S.; et al. Radiomics Features of Multiparametric MRI as Novel Prognostic Factors in Advanced Nasopharyngeal CarcinomaPretreatment Radiomics for Nasopharyngeal Carcinoma. Clin. Cancer Res. 2017, 23, 4259–4269. [Google Scholar] [CrossRef] [PubMed]
  55. Zhong, L.Z.; Fang, X.L.; Dong, D.; Peng, H.; Fang, M.J.; Huang, C.L.; He, B.X.; Lin, L.; Ma, J.; Tang, L.L.; et al. A deep learning MR-based radiomic nomogram may predict survival for nasopharyngeal carcinoma patients with stage T3N1M0. Radiother. Oncol. 2020, 151, 1–9. [Google Scholar] [CrossRef]
  56. Lee, S.; Choi, Y.; Seo, M.K.; Jang, J.; Shin, N.Y.; Ahn, K.J.; Kim, B.S. Magnetic Resonance Imaging-Based Radiomics for the Prediction of Progression-Free Survival in Patients with Nasopharyngeal Carcinoma: A Systematic Review and Meta-Analysis. Cancers 2022, 14, 653. [Google Scholar] [CrossRef]
  57. Yang, C.; Jiang, Z.; Cheng, T.; Zhou, R.; Wang, G.; Jing, D.; Bo, L.; Huang, P.; Wang, J.; Zhang, D.; et al. Radiomics for Predicting Response of Neoadjuvant Chemotherapy in Nasopharyngeal Carcinoma: A Systematic Review and Meta-Analysis. Front. Oncol. 2022, 12, 893103. [Google Scholar] [CrossRef]
  58. Park, H.; Lim, Y.; Ko, E.S.; Cho, H.h.; Lee, J.E.; Han, B.K.; Ko, E.Y.; Choi, J.S.; Park, K.W. Radiomics Signature on Magnetic Resonance Imaging: Association with Disease-Free Survival in Patients with Invasive Breast CancerRadiomics Signature on MRI for DFS in Invasive Breast Cancer. Clin. Cancer Res. 2018, 24, 4705–4714. [Google Scholar] [CrossRef]
  59. Han, L.; Zhu, Y.; Liu, Z.; Yu, T.; He, C.; Jiang, W.; Kan, Y.; Dong, D.; Tian, J.; Luo, Y. Radiomic nomogram for prediction of axillary lymph node metastasis in breast cancer. Eur. Radiol. 2019, 29, 3820–3829. [Google Scholar] [CrossRef]
  60. Jiang, M.; Li, C.L.; Luo, X.M.; Chuan, Z.R.; Lv, W.Z.; Li, X.; Cui, X.W.; Dietrich, C.F. Ultrasound-based deep learning radiomics in the assessment of pathological complete response to neoadjuvant chemotherapy in locally advanced breast cancer. Eur. J. Cancer 2021, 147, 95–105. [Google Scholar] [CrossRef]
  61. Pesapane, F.; Agazzi, G.M.; Rotili, A.; Ferrari, F.; Cardillo, A.; Penco, S.; Dominelli, V.; D’Ecclesiis, O.; Vignati, S.; Raimondi, S.; et al. Prediction of the Pathological Response to Neoadjuvant Chemotherapy in Breast Cancer Patients with MRI-Radiomics: A Systematic Review and Meta-analysis. Curr. Probl. Cancer 2022, 46, 100883. [Google Scholar] [CrossRef]
  62. Liang, X.; Yu, X.; Gao, T. Machine learning with magnetic resonance imaging for prediction of response to neoadjuvant chemotherapy in breast cancer: A systematic review and meta-analysis. Eur. J. Radiol. 2022, 150, 110247. [Google Scholar] [CrossRef] [PubMed]
  63. Zhou, H.; Dong, D.; Chen, B.; Fang, M.; Cheng, Y.; Gan, Y.; Zhang, R.; Zhang, L.; Zang, Y.; Liu, Z.; et al. Diagnosis of distant metastasis of lung cancer: Based on clinical and radiomic features. Transl. Oncol. 2018, 11, 31–36. [Google Scholar] [CrossRef] [PubMed]
  64. Chen, J.; Zeng, H.; Zhang, C.; Shi, Z.; Dekker, A.; Wee, L.; Bermejo, I. Lung cancer diagnosis using deep attention-based multiple instance learning and radiomics. Med. Phys. 2022, 49, 3134–3143. [Google Scholar] [CrossRef] [PubMed]
  65. He, B.; Dong, D.; She, Y.; Zhou, C.; Fang, M.; Zhu, Y.; Zhang, H.; Huang, Z.; Jiang, T.; Tian, J.; et al. Predicting response to immunotherapy in advanced non-small-cell lung cancer using tumor mutational burden radiomic biomarker. J. Immunother. Cancer 2020, 8, e000550. [Google Scholar] [CrossRef]
  66. Zhang, L.; Chen, B.; Liu, X.; Song, J.; Fang, M.; Hu, C.; Dong, D.; Li, W.; Tian, J. Quantitative biomarkers for prediction of epidermal growth factor receptor mutation in non-small cell lung cancer. Transl. Oncol. 2018, 11, 94–101. [Google Scholar] [CrossRef] [PubMed]
  67. Zheng, X.; He, B.; Hu, Y.; Ren, M.; Chen, Z.; Zhang, Z.; Ma, J.; Ouyang, L.; Chu, H.; Gao, H.; et al. Diagnostic Accuracy of Deep Learning and Radiomics in Lung Cancer Staging: A Systematic Review and Meta-Analysis. Front. Public Health 2022, 10, 938113. [Google Scholar] [CrossRef]
  68. Gabelloni, M.; Faggioni, L.; Fusco, R.; Simonetti, I.; De Muzio, F.; Giacobbe, G.; Borgheresi, A.; Bruno, F.; Cozzi, D.; Grassi, F.; et al. Radiomics in Lung Metastases: A Systematic Review. J. Pers. Med. 2023, 13, 225. [Google Scholar] [CrossRef]
  69. Gu, H.; Zhang, X.; Di Russo, P.; Zhao, X.; Xu, T. The current state of radiomics for meningiomas: Promises and challenges. Front. Oncol. 2020, 10, 567736. [Google Scholar] [CrossRef]
  70. Reiazi, R.; Abbas, E.; Famiyeh, P.; Rezaie, A.; Kwan, J.Y.; Patel, T.; Bratman, S.V.; Tadic, T.; Liu, F.F.; Haibe-Kains, B. The impact of the variation of imaging parameters on the robustness of computed tomography radiomic features: A review. Comput. Biol. Med. 2021, 133, 104400. [Google Scholar] [CrossRef]
  71. Cui, Y.; Zhang, J.; Li, Z.; Wei, K.; Lei, Y.; Ren, J.; Wu, L.; Shi, Z.; Meng, X.; Yang, X.; et al. A CT-based deep learning radiomics nomogram for predicting the response to neoadjuvant chemotherapy in patients with locally advanced gastric cancer: A multicenter cohort study. EClinicalMedicine 2022, 46, 101348. [Google Scholar] [CrossRef]
  72. Zhang, L.; Dong, D.; Zhang, W.; Hao, X.; Fang, M.; Wang, S.; Li, W.; Liu, Z.; Wang, R.; Zhou, J.; et al. A deep learning risk prediction model for overall survival in patients with gastric cancer: A multicenter study. Radiother. Oncol. 2020, 150, 73–80. [Google Scholar] [CrossRef] [PubMed]
  73. Wu, X.; Dong, D.; Zhang, L.; Fang, M.; Zhu, Y.; He, B.; Ye, Z.; Zhang, M.; Zhang, S.; Tian, J. Exploring the predictive value of additional peritumoral regions based on deep learning and radiomics: A multicenter study. Med. Phys. 2021, 48, 2374–2385. [Google Scholar] [CrossRef] [PubMed]
  74. Sullivan, D.C.; Obuchowski, N.A.; Kessler, L.G.; Raunig, D.L.; Gatsonis, C.; Huang, E.P.; Kondratovich, M.; McShane, L.M.; Reeves, A.P.; Barboriak, D.P.; et al. Metrology standards for quantitative imaging biomarkers. Radiology 2015, 277, 813–825. [Google Scholar] [CrossRef]
  75. deSouza, N.M.; Achten, E.; Alberich-Bayarri, A.; Bamberg, F.; Boellaard, R.; Clément, O.; Fournier, L.; Gallagher, F.; Golay, X.; Heussel, C.P.; et al. Validated imaging biomarkers as decision-making tools in clinical trials and routine practice: Current status and recommendations from the EIBALL* subcommittee of the European Society of Radiology (ESR). Insights Imaging 2019, 10, 87. [Google Scholar] [CrossRef] [PubMed]
  76. Avery, E.; Sanelli, P.C.; Aboian, M.; Payabvash, S. Radiomics: A Primer on Processing Workflow and Analysis. Semin. Ultrasound CT MRI 2022, 43, 142–146. [Google Scholar] [CrossRef] [PubMed]
  77. Bartko, J.J. The intraclass correlation coefficient as a measure of reliability. Psychol. Rep. 1966, 19, 3–11. [Google Scholar] [CrossRef] [PubMed]
  78. Fedorov, A.; Beichel, R.; Kalpathy-Cramer, J.; Finet, J.; Fillion-Robin, J.C.; Pujol, S.; Bauer, C.; Jennings, D.; Fennessy, F.; Sonka, M.; et al. 3D Slicer as an image computing platform for the Quantitative Imaging Network. Magn. Reson. Imaging 2012, 30, 1323–1341. [Google Scholar] [CrossRef]
  79. Yushkevich, P.A.; Gao, Y.; Gerig, G. ITK-SNAP: An interactive tool for semi-automatic segmentation of multi-modality biomedical images. In Proceedings of the 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA, 16–20 August 2016; IEEE: Piscataway, NJ, USA, 2016; pp. 3342–3345. [Google Scholar]
  80. Girish, V.; Vijayalakshmi, A. Affordable image analysis using NIH Image/ImageJ. Indian J. Cancer 2004, 41, 47. [Google Scholar]
  81. Tian, J.; Xue, J.; Dai, Y.; Chen, J.; Zheng, J. A novel software platform for medical image processing and analyzing. IEEE Trans. Inf. Technol. Biomed. 2008, 12, 800–812. [Google Scholar] [CrossRef]
  82. Dong, D.; Tian, J.; Dai, Y.; Yan, G.; Yang, F.; Wu, P. Unified reconstruction framework for multi-modal medical imaging. J. X-ray Sci. Technol. 2011, 19, 111–126. [Google Scholar] [CrossRef]
  83. Wang, S.; Zhou, M.; Liu, Z.; Liu, Z.; Gu, D.; Zang, Y.; Dong, D.; Gevaert, O.; Tian, J. Central focused convolutional neural networks: Developing a data-driven model for lung nodule segmentation. Med. Image Anal. 2017, 40, 172–183. [Google Scholar] [CrossRef]
  84. Yang, S.; Kweon, J.; Roh, J.H.; Lee, J.H.; Kang, H.; Park, L.J.; Kim, D.J.; Yang, H.; Hur, J.; Kang, D.Y.; et al. Deep learning segmentation of major vessels in X-ray coronary angiography. Sci. Rep. 2019, 9, 16897. [Google Scholar] [CrossRef] [PubMed]
  85. Yu, S.; Wang, Q.; Ru, C.; Pang, M. Location detection of key areas in medical images based on Haar-like fusion contour feature learning. Technol. Health Care 2020, 28, 391–399. [Google Scholar] [CrossRef] [PubMed]
  86. Chen, X.; Pan, L. A survey of graph cuts/graph search based medical image segmentation. IEEE Rev. Biomed. Eng. 2018, 11, 112–124. [Google Scholar] [CrossRef] [PubMed]
  87. Zanaty, E.; Afifi, A. A watershed approach for improving medical image segmentation. Comput. Methods Biomech. Biomed. Eng. 2013, 16, 1262–1272. [Google Scholar] [CrossRef] [PubMed]
  88. Hesamian, M.H.; Jia, W.; He, X.; Kennedy, P. Deep Learning Techniques for Medical Image Segmentation: Achievements and Challenges. J. Digit. Imaging 2019, 32, 582–596. [Google Scholar] [CrossRef]
  89. Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of the Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, 5–9 October 2015; Proceedings, Part III 18. Springer: Cham, Switzerland, 2015; pp. 234–241. [Google Scholar]
  90. Zhang, J.; Qin, Q.; Ye, Q.; Ruan, T. ST-Unet: Swin Transformer boosted U-Net with Cross-Layer Feature Enhancement for medical image segmentation. Comput. Biol. Med. 2023, 153, 106516. [Google Scholar] [CrossRef] [PubMed]
  91. Zhou, Z.; Rahman Siddiquee, M.M.; Tajbakhsh, N.; Liang, J. Unet++: A nested u-net architecture for medical image segmentation. In Proceedings of the Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, 20 September 2018; Proceedings 4. Springer: Cham, Switzerland, 2018; pp. 3–11. [Google Scholar]
  92. Li, J.; Liu, K.; Hu, Y.; Zhang, H.; Heidari, A.A.; Chen, H.; Zhang, W.; Algarni, A.D.; Elmannai, H. Eres-UNet++: Liver CT image segmentation based on high-efficiency channel attention and Res-UNet++. Comput. Biol. Med. 2023, 158, 106501. [Google Scholar] [CrossRef] [PubMed]
  93. Gu, Z.; Cheng, J.; Fu, H.; Zhou, K.; Hao, H.; Zhao, Y.; Zhang, T.; Gao, S.; Liu, J. Ce-net: Context encoder network for 2d medical image segmentation. IEEE Trans. Med. Imaging 2019, 38, 2281–2292. [Google Scholar] [CrossRef] [PubMed]
  94. Fang, X.; Yan, P. Multi-organ segmentation over partially labeled datasets with multi-scale feature abstraction. IEEE Trans. Med. Imaging 2020, 39, 3619–3629. [Google Scholar] [CrossRef]
  95. Van Griethuysen, J.J.; Fedorov, A.; Parmar, C.; Hosny, A.; Aucoin, N.; Narayan, V.; Beets-Tan, R.G.; Fillion-Robin, J.C.; Pieper, S.; Aerts, H.J. Computational radiomics system to decode the radiographic phenotype. Cancer Res. 2017, 77, e104–e107. [Google Scholar] [CrossRef] [PubMed]
  96. Zwanenburg, A.; Vallières, M.; Abdalah, M.A.; Aerts, H.J.; Andrearczyk, V.; Apte, A.; Ashrafinia, S.; Bakas, S.; Beukinga, R.J.; Boellaard, R.; et al. The image biomarker standardization initiative: Standardized quantitative radiomics for high-throughput image-based phenotyping. Radiology 2020, 295, 328–338. [Google Scholar] [CrossRef]
  97. Rizzo, S.; Botta, F.; Raimondi, S.; Origgi, D.; Fanciullo, C.; Morganti, A.G.; Bellomi, M. Radiomics: The facts and the challenges of image analysis. Eur. Radiol. Exp. 2018, 2, 36. [Google Scholar] [CrossRef] [PubMed]
  98. Ergen, B.; Baykara, M. Texture based feature extraction methods for content based medical image retrieval systems. Bio-Med. Mater. Eng. 2014, 24, 3055–3062. [Google Scholar] [CrossRef]
  99. Haralick, R.M.; Shanmugam, K.; Dinstein, I.H. Textural features for image classification. IEEE Trans. Syst. Man Cybern. 1973, SMC-3, 610–621. [Google Scholar] [CrossRef]
  100. Oliver, J.A.; Budzevich, M.; Zhang, G.G.; Dilling, T.J.; Latifi, K.; Moros, E.G. Variability of Image Features Computed from Conventional and Respiratory-Gated PET/CT Images of Lung Cancer. Transl. Oncol. 2015, 8, 524–534. [Google Scholar] [CrossRef]
  101. Chu, A.; Sehgal, C.M.; Greenleaf, J.F. Use of gray value distribution of run lengths for texture analysis. Pattern Recognit. Lett. 1990, 11, 415–419. [Google Scholar] [CrossRef]
  102. Galloway, M.M. Texture analysis using gray level run lengths. Comput. Graph. Image Process. 1975, 4, 172–179. [Google Scholar] [CrossRef]
  103. Thibault, G.; Fertil, B.; Navarro, C.; Pereira, S.; Cau, P.; Levy, N.; Sequeira, J.; Mari, J.-L. Shape and Texture Indexes Application to Cell Nuclei Classification. Int. J. Pattern Recognit. Artif. Intell. 2013, 27, 1357002. [Google Scholar] [CrossRef]
  104. Amadasun, M.; King, R. Textural features corresponding to textural properties. IEEE Trans. Syst. Man Cybern. 1989, 19, 1264–1274. [Google Scholar] [CrossRef]
  105. Haralick, R.M. Statistical and structural approaches to texture. Proc. IEEE 1979, 67, 786–804. [Google Scholar] [CrossRef]
  106. Bera, K.; Braman, N.; Gupta, A.; Velcheti, V.; Madabhushi, A. Predicting cancer outcomes with radiomics and artificial intelligence in radiology. Nat. Rev. Clin. Oncol. 2022, 19, 132–146. [Google Scholar] [CrossRef] [PubMed]
  107. Jiang, Y.; Liang, X.; Wang, W.; Chen, C.; Yuan, Q.; Zhang, X.; Li, N.; Chen, H.; Yu, J.; Xie, Y.; et al. Noninvasive prediction of occult peritoneal metastasis in gastric cancer using deep learning. JAMA Netw. Open 2021, 4, e2032269. [Google Scholar] [CrossRef]
  108. Li, Z.; Zhang, J.; Tan, T.; Teng, X.; Sun, X.; Zhao, H.; Liu, L.; Xiao, Y.; Lee, B.; Li, Y.; et al. Deep learning methods for lung cancer segmentation in whole-slide histopathology images—The acdc@ lunghp challenge 2019. IEEE J. Biomed. Health Inform. 2020, 25, 429–440. [Google Scholar] [CrossRef]
  109. Liang, B.; Li, H.; Su, M.; Li, X.; Shi, W.; Wang, X. Detecting adversarial image examples in deep neural networks with adaptive noise reduction. IEEE Trans. Dependable Secur. Comput. 2018, 18, 72–85. [Google Scholar] [CrossRef]
  110. Chen, T.; Guestrin, C. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13–17 August 2016; pp. 785–794. [Google Scholar]
  111. Belgiu, M.; Drăguţ, L. Random forest in remote sensing: A review of applications and future directions. ISPRS J. Photogramm. Remote Sens. 2016, 114, 24–31. [Google Scholar] [CrossRef]
  112. Breiman, L. Random forests. Mach. Learn. 2001, 45, 5–32. [Google Scholar] [CrossRef]
  113. Murphy, K.P. Naive bayes classifiers. Univ. Br. Columbia 2006, 18, 1–8. [Google Scholar]
  114. Suthaharan, S.; Suthaharan, S. Support vector machine. In Machine Learning Models and Algorithms for Big Data Classification: Thinking with Examples for Effective Learning; Springer: New York, NY, USA, 2016; pp. 207–235. [Google Scholar]
  115. Shiri, I.; Salimi, Y.; Pakbin, M.; Hajianfar, G.; Avval, A.H.; Sanaat, A.; Mostafaei, S.; Akhavanallaf, A.; Saberi, A.; Mansouri, Z.; et al. COVID-19 prognostic modeling using CT radiomic features and machine learning algorithms: Analysis of a multi-institutional dataset of 14,339 patients. Comput. Biol. Med. 2022, 145, 105467. [Google Scholar] [CrossRef]
  116. Park, J.E.; Kim, D.; Kim, H.S.; Park, S.Y.; Kim, J.Y.; Cho, S.J.; Shin, J.H.; Kim, J.H. Quality of science and reporting of radiomics in oncologic studies: Room for improvement according to radiomics quality score and TRIPOD statement. Eur. Radiol. 2020, 30, 523–536. [Google Scholar] [CrossRef]
  117. Baeßler, B.; Weiss, K.; Dos Santos, D.P. Robustness and reproducibility of radiomics in magnetic resonance imaging: A phantom study. Investig. Radiol. 2019, 54, 221–228. [Google Scholar] [CrossRef] [PubMed]
  118. Bernatz, S.; Zhdanovich, Y.; Ackermann, J.; Koch, I.; Wild, P.J.; Dos Santos, D.P.; Vogl, T.J.; Kaltenbach, B.; Rosbach, N. Impact of rescanning and repositioning on radiomic features employing a multi-object phantom in magnetic resonance imaging. Sci. Rep. 2021, 11, 14248. [Google Scholar] [CrossRef]
  119. Pinto dos Santos, D.; Dietzel, M.; Baessler, B. A decade of radiomics research: Are images really data or just patterns in the noise? Eur. Radiol. 2021, 31, 1–4. [Google Scholar] [CrossRef]
  120. Demircioğlu, A. Benchmarking feature selection methods in radiomics. Investig. Radiol. 2022, 57, 433–443. [Google Scholar] [CrossRef] [PubMed]
  121. Dash, M.; Liu, H. Feature selection for classification. Intell. Data Anal. 1997, 1, 131–156. [Google Scholar] [CrossRef]
  122. Yu, L.; Liu, H. Efficient feature selection via analysis of relevance and redundancy. J. Mach. Learn. Res. 2004, 5, 1205–1224. [Google Scholar]
  123. Li, Y.; Li, T.; Liu, H. Recent advances in feature selection and its applications. Knowl. Inf. Syst. 2017, 53, 551–577. [Google Scholar] [CrossRef]
  124. Peng, H.; Long, F.; Ding, C. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 2005, 27, 1226–1238. [Google Scholar] [CrossRef]
  125. Hall, M.A. Correlation-based Feature Selection for Discrete and Numeric Class Machine Learning. In Proceedings of the International Conference on Machine Learning, Stanford, CA, USA, 29 June–2 July 1999; pp. 1–8. [Google Scholar]
  126. Yu, L.; Liu, H. Feature selection for high-dimensional data: A fast correlation-based filter solution. In Proceedings of the 20th International Conference on Machine Learning (ICML-03), Washington, DC, USA, 21–24 August 2003; pp. 856–863. [Google Scholar]
  127. Anukrishna, P.; Paul, V. A review on feature selection for high dimensional data. In Proceedings of the 2017 International Conference on Inventive Systems and Control (ICISC), Coimbatore, India, 19–20 January 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 1–4. [Google Scholar]
  128. Alduailij, M.; Khan, Q.W.; Tahir, M.; Sardaraz, M.; Alduailij, M.; Malik, F. Machine-Learning-Based DDoS Attack Detection Using Mutual Information and Random Forest Feature Importance Method. Symmetry 2022, 14, 1095. [Google Scholar] [CrossRef]
  129. Cohen, I.; Huang, Y.; Chen, J.; Benesty, J.; Benesty, J.; Chen, J.; Huang, Y.; Cohen, I. Pearson correlation coefficient. Noise Reduct. Speech Process. 2009, 2, 1–4. [Google Scholar]
  130. Yu, H.; Meng, X.; Chen, H.; Han, X.; Fan, J.; Gao, W.; Du, L.; Chen, Y.; Wang, Y.; Liu, X.; et al. Correlation between mammographic radiomics features and the level of tumor-infiltrating lymphocytes in patients with triple-negative breast cancer. Front. Oncol. 2020, 10, 412. [Google Scholar] [CrossRef] [PubMed]
  131. Spearman, C. The proof and measurement of association between two things. Amer. J. Psychol. 1904, 15, 72–101. [Google Scholar] [CrossRef]
  132. Chelvan, P.M.; Perumal, K. A comparative analysis of feature selection stability measures. In Proceedings of the 2017 International Conference on Trends in Electronics and Informatics (ICEI), Tirunelveli, India, 11–12 May 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 124–128. [Google Scholar]
  133. Larson, M.G. Analysis of variance. Circulation 2008, 117, 115–121. [Google Scholar] [CrossRef] [PubMed]
  134. Wang, Z.; Yang, G.; Wang, X.; Cao, Y.; Jiao, W.; Niu, H. A combined model based on CT radiomics and clinical variables to predict uric acid calculi which have a good accuracy. Urolithiasis 2023, 51, 37. [Google Scholar] [CrossRef] [PubMed]
  135. Liu, H.; Setiono, R. Chi2: Feature selection and discretization of numeric attributes. In Proceedings of the 7th IEEE International Conference on Tools with Artificial Intelligence, Herndon, VA, USA, 5–8 November 1995; IEEE: Piscataway, NJ, USA, 1995; pp. 388–391. [Google Scholar]
  136. Plackett, R.L. Karl Pearson and the chi-squared test. Int. Stat. Rev. Int. De Statistique 1983, 51, 59–72. [Google Scholar] [CrossRef]
  137. Vergara, J.R.; Estévez, P.A. A review of feature selection methods based on mutual information. Neural Comput. Appl. 2014, 24, 175–186. [Google Scholar] [CrossRef]
  138. Wu, Y.; Liu, B.; Wu, W.; Lin, Y.; Yang, C.; Wang, M. Grading glioma by radiomics with feature selection based on mutual information. J. Ambient. Intell. Humaniz. Comput. 2018, 9, 1671–1682. [Google Scholar] [CrossRef]
  139. Gan, M.; Zhang, L. Iteratively local fisher score for feature selection. Appl. Intell. 2021, 51, 6167–6181. [Google Scholar] [CrossRef]
  140. Zeng, X.; Chen, Y.W.; Tao, C. Feature selection using recursive feature elimination for handwritten digit recognition. In Proceedings of the 2009 Fifth International Conference on Intelligent Information Hiding and Multimedia Signal Processing, Kyoto, Japan, 12–14 September 2009; IEEE: Piscataway, NJ, USA, 2009; pp. 1205–1208. [Google Scholar]
  141. Bundy, A.; Wallen, L. Breadth-first search. In Catalogue of Artificial Intelligence Tools; Springer: Berlin/Heidelberg, Germany, 1984; p. 13. [Google Scholar]
  142. Dechter, R.; Pearl, J. Generalized best-first search strategies and the optimality of A. J. ACM (JACM) 1985, 32, 505–536. [Google Scholar] [CrossRef]
  143. Morris, R. Tapered floating point: A new floating-point representation. IEEE Trans. Comput. 1971, 100, 1578–1579. [Google Scholar] [CrossRef]
  144. Theodoridis, S.; Pikrakis, A.; Koutroumbas, K.; Cavouras, D. Introduction to Pattern Recognition: A Matlab Approach; Academic Press: Cambridge, MA, USA, 2010. [Google Scholar]
  145. Mustafa, S. Feature selection using sequential backward method in melanoma recognition. In Proceedings of the 2017 13th International Conference on Electronics, Computer and Computation (ICECCO), Abuja, Nigeria, 28–29 November 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 1–4. [Google Scholar]
  146. Doak, J. An Evaluation of Feature Selection Methods and Their Application to Computer Security; Technical Report CSE-92-18; University of California, Department of Computer Science: Berkeley, CA, USA, 1992. [Google Scholar]
  147. Webb, A.R. Statistical Pattern Recognition; John Wiley & Sons: Hoboken, NJ, USA, 2003. [Google Scholar]
  148. Pudil, P.; Novovičová, J.; Kittler, J. Floating search methods in feature selection. Pattern Recognit. Lett. 1994, 15, 1119–1125. [Google Scholar] [CrossRef]
  149. Grabczewski, K.; Jankowski, N. Feature selection with decision tree criterion. In Proceedings of the Fifth International Conference on Hybrid Intelligent Systems (HIS’05), Rio de Janeiro, Brazil, 6–9 November 2005; IEEE: Piscataway, NJ, USA, 2005; p. 6. [Google Scholar]
  150. Bertsimas, D.; Tsitsiklis, J. Simulated annealing. Stat. Sci. 1993, 8, 10–15. [Google Scholar] [CrossRef]
  151. Goldberg, D.; Sastry, K. Genetic Algorithms: The Design of Innovation; Springer: Berlin/Heidelberg, Germany, 2007. [Google Scholar]
  152. Tibshirani, R. Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B (Methodol.) 1996, 58, 267–288. [Google Scholar] [CrossRef]
  153. Friedman, J.H. Greedy function approximation: A gradient boosting machine. Ann. Stat. 2001, 29, 1189–1232. [Google Scholar] [CrossRef]
  154. Meng, L.; Dong, D.; Chen, X.; Fang, M.; Wang, R.; Li, J.; Liu, Z.; Tian, J. 2D and 3D CT radiomic features performance comparison in characterization of gastric cancer: A multi-center study. IEEE J. Biomed. Health Inform. 2020, 25, 755–763. [Google Scholar] [CrossRef] [PubMed]
  155. Gong, C.S.A.; Su, C.H.S.; Chao, K.W.; Chao, Y.C.; Su, C.K.; Chiu, W.H. Exploiting deep neural network and long short-term memory method-ologies in bioacoustic classification of LPC-based features. PLoS ONE 2021, 16, e0259140. [Google Scholar] [CrossRef]
  156. Jolliffe, I.T.; Cadima, J. Principal component analysis: A review and recent developments. Philos. Trans. R. Soc. A Math. Eng. Sci. 2016, 374, 20150202. [Google Scholar] [CrossRef] [PubMed]
  157. Wiltgen, T.; Fleischmann, D.F.; Kaiser, L.; Holzgreve, A.; Corradini, S.; Landry, G.; Ingrisch, M.; Popp, I.; Grosu, A.L.; Unterrainer, M.; et al. 18F-FET PET radiomics-based survival prediction in glioblastoma patients receiving radio (chemo) therapy. Radiat. Oncol. 2022, 17, 198. [Google Scholar] [CrossRef]
  158. Balakrishnama, S.; Ganapathiraju, A. Linear discriminant analysis—A brief tutorial. Inst. Signal Inf. Process. 1998, 18, 1–8. [Google Scholar]
  159. Zhao, H.; Wang, Z.; Nie, F. A new formulation of linear discriminant analysis for robust dimensionality reduction. IEEE Trans. Knowl. Data Eng. 2018, 31, 629–640. [Google Scholar] [CrossRef]
  160. Huang, Y.; Liu, Z.; He, L.; Chen, X.; Pan, D.; Ma, Z.; Liang, C.; Tian, J.; Liang, C. Radiomics Signature: A Potential Biomarker for the Prediction of Disease-Free Survival in Early-Stage (I or II) Non-Small Cell Lung Cancer. Radiology 2016, 281, 947–957. [Google Scholar] [CrossRef]
  161. Zhao, Z. -Q.; Zheng, P.; Xu, S. -T.; Wu, X. Object Detection With Deep Learning: A Review. IEEE Trans. Neural Netw. Learn. Syst. 2019, 30, 3212–3232. [Google Scholar] [CrossRef]
  162. Li, C.; Dong, D.; Li, L.; Gong, W.; Li, X.; Bai, Y.; Wang, M.; Hu, Z.; Zha, Y.; Tian, J. Classification of severe and critical covid-19 using deep learning and radiomics. IEEE J. Biomed. Health Inform. 2020, 24, 3585–3594. [Google Scholar] [CrossRef] [PubMed]
  163. Tomaszewski, M.R.; Gillies, R.J. The biological meaning of radiomic features. Radiology 2021, 298, 505–516. [Google Scholar] [CrossRef] [PubMed]
  164. Zhang, L.; Wang, M.; Liu, M.; Zhang, D. A Survey on Deep Learning for Neuroimaging-Based Brain Disorder Analysis. Front. Neurosci. 2020, 14, 779. [Google Scholar] [CrossRef]
  165. Tang, J.; Alelyani, S.; Liu, H. Feature selection for classification: A review. In Data Classification: Algorithms and Applications; CRC Press: Boca Raton, FL, USA, 2014; p. 37. [Google Scholar]
  166. Li, J.; Cheng, K.; Wang, S.; Morstatter, F.; Trevino, R.P.; Tang, J.; Liu, H. Feature Selection: A Data Perspective. ACM Comput. Surv. 2017, 50, 94. [Google Scholar] [CrossRef]
  167. Cai, J.; Luo, J.; Wang, S.; Yang, S. Feature selection in machine learning: A new perspective. Neurocomputing 2018, 300, 70–79. [Google Scholar] [CrossRef]
  168. Guyon, I.; Gunn, S.; Ben-Hur, A.; Dror, G. Result analysis of the NIPS 2003 feature selection challenge. Adv. Neural Inf. Process. Syst. 2004, 17. Available online: https://api.semanticscholar.org/CorpusID:2803126 (accessed on 8 August 2023).
  169. Porcu, M.; Solinas, C.; Mannelli, L.; Micheletti, G.; Lambertini, M.; Willard-Gallo, K.; Neri, E.; Flanders, A.E.; Saba, L. Radiomics and “radi-… omics” in cancer immunotherapy: A guide for clinicians. Crit. Rev. Oncol./Hematol. 2020, 154, 103068. [Google Scholar] [CrossRef]
  170. Oliva, J.T.; Lee, H.D.; Spolaôr, N.; Coy, C.S.R.; Wu, F.C. Prototype system for feature extraction, classification and study of medical images. Expert Syst. Appl. 2016, 63, 267–283. [Google Scholar] [CrossRef]
  171. Park, S.H.; Han, K. Methodologic guide for evaluating clinical performance and effect of artificial intelligence technology for medical diagnosis and prediction. Radiology 2018, 286, 800–809. [Google Scholar] [CrossRef] [PubMed]
  172. Schwier, M.; Van Griethuysen, J.; Vangel, M.G.; Pieper, S.; Peled, S.; Tempany, C.; Aerts, H.J.; Kikinis, R.; Fennessy, F.M.; Fedorov, A. Repeatability of multiparametric prostate MRI radiomics features. Sci. Rep. 2019, 9, 9441. [Google Scholar] [CrossRef]
  173. Stanzione, A.; Gambardella, M.; Cuocolo, R.; Ponsiglione, A.; Romeo, V.; Imbriaco, M. Prostate MRI radiomics: A systematic review and radiomic quality score assessment. Eur. J. Radiol. 2020, 129, 109095. [Google Scholar] [CrossRef] [PubMed]
  174. Ibrahim, A.; Primakov, S.; Beuque, M.; Woodruff, H.; Halilaj, I.; Wu, G.; Refaee, T.; Granzier, R.; Widaatalla, Y.; Hustinx, R.; et al. Radiomics for precision medicine: Current challenges, future prospects, and the proposal of a new framework. Methods 2021, 188, 20–29. [Google Scholar] [CrossRef]
  175. Dong, D.; He, B.; Kong, B.; Zhang, L.; Tong, L.; Huang, F.; Han, D.; Huang, Z.; Zhang, H.; Tian, J. Abstract CT274: Diagnosis based on signal: The first time break the routinely used circle of signal-to-image-to-diagnose. Cancer Res. 2020, 80, CT274. [Google Scholar] [CrossRef]
  176. He, B.; Guo, Y.; Zhu, Y.; Tong, L.; Kong, B.; Wang, K.; Sun, C.; Li, H.; Huang, F.; Wu, L.; et al. From signal to knowledge: The diagnostic value of rawdata in artificial intelligence prediction of human data for the first time. Engineering 2023. [Google Scholar] [CrossRef]
Figure 1. Number of publications related to radiation medicine in the PubMed database from 2012 to the present.
Figure 1. Number of publications related to radiation medicine in the PubMed database from 2012 to the present.
Symmetry 15 01834 g001
Figure 2. Radiomics techniques have been applied to variety of diseases.
Figure 2. Radiomics techniques have been applied to variety of diseases.
Symmetry 15 01834 g002
Figure 3. Radiomics Workflow.
Figure 3. Radiomics Workflow.
Symmetry 15 01834 g003
Figure 4. Examples of specific matrices that allocate information about the spatial distribution of pixel values in the image.
Figure 4. Examples of specific matrices that allocate information about the spatial distribution of pixel values in the image.
Symmetry 15 01834 g004
Figure 5. DL network to extract radiomics features.
Figure 5. DL network to extract radiomics features.
Symmetry 15 01834 g005
Figure 6. Image (A) indicates search-based framework; image (B) indicates relevance-based framework.
Figure 6. Image (A) indicates search-based framework; image (B) indicates relevance-based framework.
Symmetry 15 01834 g006
Figure 7. Subfigure (A) depicts the filter method, while Subfigure (B) illustrates the wrapper method, and Subfigure (C) shows the embedded method.
Figure 7. Subfigure (A) depicts the filter method, while Subfigure (B) illustrates the wrapper method, and Subfigure (C) shows the embedded method.
Symmetry 15 01834 g007
Table 1. Commonly extracted first-order, shape-based, and texture features.
Table 1. Commonly extracted first-order, shape-based, and texture features.
First-order
features
Shape-based
features
Texture features
Gray Level
Co-occurrence
Matrix (GLCM)
Gray Level
Size Zone
Matrix (GLSZM)
Gray Level
Dependence
Matrix (GLDM)
Gray Level
Run Length
Matrix (GLRLM)
Neighboring Gray
Tone Difference
Matrix (NGTDM)
EnergyMesh VolumeAutocorrelationSmall Area EmphasisSmall Dependence
Emphasis
Short Run
Emphasis
Coarseness
Total EnergyVoxel VolumeJoint AverageLarge Area EmphasisLarge Dependence
Emphasis
Long Run
Emphasis
Contrast
EntropySurface AreaCluster ProminenceGray Level
Non-Uniformity
Gray Level
Non-Uniformity
Gray Level
Non-Uniformity
Busyness
MinimumSurface Area
to Volume ratio
Cluster ShadeGray Level
Non-Uniformity Normalized
Dependence
Non-Uniformity
Gray Level
Non-Uniformity
Normalized
Complexity
10th percentileSphericityCluster TendencySize-Zone
Non-Uniformity
Dependence
Non-Uniformity Normalized
Run Length
Non-Uniformity
Strength
90th percentileMaximum 3D
diameter
CorrelationSize-Zone
Non-Uniformity Normalized
Gray Level VarianceRun Length
Non-Uniformity
Normalized
MaximumMaximum 2D
diameter (Slice)
Difference AverageZone PercentageDependence VarianceRun Percentage
MeanMaximum 2D
diameter (Column)
Difference EntropyZone VarianceDependence EntropyGray Level Variance
MedianMaximum 2D
diameter(Row)
Difference VarianceZone EntropyLow Gray Level
Emphasis
Run Variance
Interquartile
Range
Major Axis
Length
Joint EnergyLow Gray
Level Zone Emphasis
High Gray
Level Emphasis
Run Entropy
RangeMinor Axis
Length
Joint EntropyHigh Gray
Level Zone Emphasis
Small Dependence Low
Gray Level Emphasis
Low Gray Level
Run Emphasis
Mean Absolute
Deviation
Least Axis

Length
Informational Measure
of Correlation
Small Area Low Gray
Level Emphasis
Small Dependence High
Gray Level Emphasis
High Gray Level
Run Emphasis
Robust Mean
Absolute Deviation
ElongationInverse Difference
Moment
Small Area High
Gray Level Emphasis
Large Dependence Low
Gray Level Emphasis
Short Run Low
Gray Level Emphasis
Root Mean SquaredFlatnessMaximal Correlation
Coefficient
Large Area Low
Gray Level Emphasis
Large Dependence High
Gray Level Emphasis
Short Run High
Gray Level Emphasis
Skewness Inverse Difference
Moment Normalized
Large Area High
Gray Level Emphasis
Long Run Low
Gray Level Emphasis
Kurtosis Inverse Difference Long Run High
Gray Level Emphasis
Variance Inverse Difference
Normalized
Uniformity Invers
Variance
Maximum
Probability
Sum
Average
Sum
Entropy
Sum of
Squares
Table 2. Frequently used filtered feature selection methods.
Table 2. Frequently used filtered feature selection methods.
Filter MethodsSelection Rules
Missing PercentageFeatures with a disproportionate share of missing samples and difficult to fill were removed.
VarianceFeatures with variance close to or equal to 0 were excluded.
FrequencyFeatures that are excessively concentrated in one category of values are removed.
Pearson Correlation Coefficient [129,130]Features with correlation coefficients close to or equal to 0 were excluded.
Spearman’s Rank Correlation Coefficient [131,132]Features with correlation coefficients close to or equal to 0 were excluded.
Kendall’s tau Rank Correlation CoefficientFeatures with correlation coefficients close to or equal to 0 were excluded.
Analysis of variance (ANOVA) [133,134]Exclude features with too low an F-value, or exclude features with a p-value < 0.05.
Chi-squared Test [135,136]Features with too low a chi-squared value were excluded, or features with a p-value < 0.05 were excluded.
Mutual Information [137,138]Features with mutual information close to or equal to 0 were removed.
mRMR [124]The features with the minimum correlation and maximum redundancy were removed.
Fisher Score [139]Features with large intraclass distances and small interclass distances were excluded.
Table 3. Feature subset search algorithms.
Table 3. Feature subset search algorithms.
Classification of Search Strategies
(Subset Generation Process)
Algorithm FeaturesSubset Search Algorithm
Complete searchIterate through all possible combinations
of feature subsets, then select the feature
subset with the best model score.
Breadth First Search [141]
Best First Search [142]
Heuristic searchThe search is evaluated for each location
searched, the best position is obtained,
and the search is carried out from this
position until the target is reached.
Sequential Forward Selection (SFS) [143,144]
Sequential Backward Selection (SBS) [145]
Bidirectional Search (BDS) [146]
Plus-L Minus-R Selection (LRS) [147]
Sequential Floating Selection (SFS) [148]
Decision Tree Method (DTM) [149]
Random searchA random subset of features is generated
and then these feature subsets are given
an evaluation.
Random Generation plus Sequential Selection (RGSS) [146]
Simulated Annealing (SA) [150]
Genetic Algorithms (GA) [151]
Table 4. Advantages and disadvantages of filters, wrappers and embedded methods.
Table 4. Advantages and disadvantages of filters, wrappers and embedded methods.
CategoryAdvantageDisadvantages
FilterMore efficient calculation
Effectively avoid over-fitting
Independent of the learning algorithms
Ignores interaction with the learning algorithms
Weakeed learer fitting ability
WrapperSimple
Interacts with the learning algorithms
Models feature dependencies
Risk of overfitting
Learning algorithms-dependent selection
A large number of calculations
EmbeddedInteracts with the learning algorithms
Less complexity than Wrapper
More efficient calculation
Learning algorithms-dependent selection
Increases the model training burden
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Zhang, W.; Guo, Y.; Jin, Q. Radiomics and Its Feature Selection: A Review. Symmetry 2023, 15, 1834. https://doi.org/10.3390/sym15101834

AMA Style

Zhang W, Guo Y, Jin Q. Radiomics and Its Feature Selection: A Review. Symmetry. 2023; 15(10):1834. https://doi.org/10.3390/sym15101834

Chicago/Turabian Style

Zhang, Wenchao, Yu Guo, and Qiyu Jin. 2023. "Radiomics and Its Feature Selection: A Review" Symmetry 15, no. 10: 1834. https://doi.org/10.3390/sym15101834

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop