Preoperative Prediction of Microsatellite Instability in Rectal Cancer Using Five Machine Learning Algorithms Based on Multiparametric MRI Radiomics

Objectives: To establish and verify radiomics models based on multiparametric MRI for preoperatively identifying the microsatellite instability (MSI) status of rectal cancer (RC) by comparing different machine learning algorithms. Methods: This retrospective study enrolled 383 (training set, 268; test set, 115) RC patients between January 2017 and June 2022. A total of 4148 radiomics features were extracted from multiparametric MRI, including T2-weighted imaging, T1-weighted imaging, apparent diffusion coefficient, and contrast-enhanced T1-weighted imaging. The analysis of variance, correlation test, univariate logistic analysis, and a gradient-boosting decision tree were used for the dimension reduction. Logistic regression, Bayes, support vector machine (SVM), K-nearest neighbor (KNN), and tree machine learning algorithms were used to build different radiomics models. The relative standard deviation (RSD) and bootstrap method were used to quantify the stability of these five algorithms. Then, predictive performances of different models were assessed using area under curves (AUCs). The performance of the best radiomics model was evaluated using calibration and discrimination. Results: Among these 383 patients, the prevalence of MSI was 14.62% (56/383). The RSD value of logistic regression algorithm was the lowest (4.64%), followed by Bayes (5.44%) and KNN (5.45%), which was significantly better than that of SVM (19.11%) and tree (11.94%) algorithms. The radiomics model based on logistic regression algorithm performed best, with AUCs of 0.827 and 0.739 in the training and test sets, respectively. Conclusions: We developed a radiomics model based on the logistic regression algorithm, which could potentially be used to facilitate the individualized prediction of MSI status in RC patients.


Introduction
Rectal cancer (RC) is one of the leading causes of cancer-related death worldwide, and it occurs with a series of genetic and protein abnormalities [1]. Of these, approximately 10% to 20% of RCs are caused by microsatellite instability (MSI), which manifests as the loss of one or more mismatch repair (MMR) proteins. In contrast, RC patients with microsatellite stability (MSS) have intact MMR proteins [2]. Research has shown that RC patients with MSI have unique biological behaviors and distinct responses to treatment, which may be resistant to 5-FU-based chemotherapy and more likely to benefit from immunotherapy [3][4][5]. Therefore, the MSI status of RC patients is a key predictor of treatment strategy and prognosis [6][7][8].
National Comprehensive Cancer Network (NCCN) and European Society for Medical Oncology (ESMO) guidelines both recommend detecting MSI status in RC patients [9,10]. It is worth noting that preoperative MSI assessment can only be performed by endoscopic biopsy [11,12]. However, the results of MSI detection may vary depending on insufficient Diagnostics 2023, 13, 269 2 of 12 samples or sampling techniques due to tumor heterogeneity [13,14]. The DNA extracted from the sample may not meet the minimum quality/quantity criteria for the genetic assay, thus resulting in unknown MSI status. In addition, the risks and complications of invasive biopsy limit its application in the real-time monitoring of disease progression and biological behaviors [15]. Therefore, it is valuable to develop a non-invasive, repeatable, and costeffective MSI prediction method to guide clinicians to choose the next treatment strategy.
Radiomics can transform microscopic heterogeneity into quantitative features to capture the deep information of tumors [16][17][18]. A few scholars reported that radiomics based on enhanced CT have certain value in predicting MSI of colorectal cancer [19][20][21]. However, these studies were all based on CT and the subjects were colorectal cancer patients, while the incidence of MSI varies with the location of colorectal cancer [22][23][24]. In addition, two studies have found that CT-based tumor and peritumoral radiomic features can be used as important biomarkers for the preoperative prediction of MSI status [25,26]. With the development of imaging technology, MRI has gradually become the mainstream of preoperative tumor evaluation [27]. A recent study enrolled 199 RC patients found that the radiomics model based on multiparametric MRI have better predictive performance than those based on single unenhanced sequence images, with AUCs of 0.78 and 0.78 in the training and validation sets, respectively [28]. In addition, highly accurate and stable prediction model can be constructed by comparing different machine learning algorithms in order to improve the decision-making process in clinical practice [29]. Accordingly, the predictive value of MRI-based radiomics in evaluating the MSI of RC patients still deserves further attention.
Therefore, the purpose of this study was to extract radiomic features based on multiparametric MRI and construct a best noninvasive radiomic model by comparing different machine learning algorithms in order to better preoperatively predict the MSI status of RC patients. We believed that this predictive information will help stratify patients based on MSI status and help optimize decisions for personalized cancer treatment.

Patients
This retrospective study was approved by our institute review board, and written informed consent was waived. A total of 1274 patients with suspected RCs were included between January 2017 and June 2022. Inclusion criteria included: (1) pathologically proven RC; (2) received rectal MRI examinations one month before surgery; (3) no history of other malignant tumors. Exclusion criteria included: (1) preoperative anti-tumor treatments; (2) lack of complete clinicopathological data; (3) poor image quality caused by metal or motion artifact. Finally, 383 patients were enrolled and divided into training (n = 268) and test (n = 115) sets at a ratio of 7:3. The patient recruitment process is shown in Figure 1.

MRI Examinations
All MRI examinations were performed using a 3.0 T MRI scanner (Skyra; Siemens Healthineers, Erlangen, Germany) equipped with an 8-channel phased-array coil in supine position. The MRI protocol included the following sequences: (1)

MRI Examinations
All MRI examinations were performed using a 3.0 T MRI scanner (Skyra; Siemens Healthineers, Erlangen, Germany) equipped with an 8-channel phased-array coil in supine position. The MRI protocol included the following sequences:

Clinical and Radiological Data
Clinical data from our picture archiving and communication system, including age, gender, carcinoembryonic antigen (CEA; levels greater than 5 ng/mL as abnormal), and carbohydrate antigens 19-9 (CA19-9; levels greater than 37 U/mL as abnormal), were retrospectively analyzed.
Radiological data were obtained from the structured report of rectum MRI, which included tumor size (maximum diameter of the tumor on the sagittal section), distance (DIS; distance from the end of the convex edge of the tumor to the edge of the anus), radiological tumor (T) stage, lymph node (N) stage, MRI-based extramural venous invasion (mrEMVI) status, circumferential resection margin (CRM), and anal canal invasion (ACI). These features were independently assessed by two experienced radiologists. For qualitative data, agreement was reached by negotiation when there was disagreement between the two radiologists. For quantitative data, measurements from these two radiologists were averaged.

Clinical and Radiological Data
Clinical data from our picture archiving and communication system, including age, gender, carcinoembryonic antigen (CEA; levels greater than 5 ng/mL as abnormal), and carbohydrate antigens 19-9 (CA19-9; levels greater than 37 U/mL as abnormal), were retrospectively analyzed.
Radiological data were obtained from the structured report of rectum MRI, which included tumor size (maximum diameter of the tumor on the sagittal section), distance (DIS; distance from the end of the convex edge of the tumor to the edge of the anus), radiological tumor (T) stage, lymph node (N) stage, MRI-based extramural venous invasion (mrEMVI) status, circumferential resection margin (CRM), and anal canal invasion (ACI). These features were independently assessed by two experienced radiologists. For qualitative data, agreement was reached by negotiation when there was disagreement between the two radiologists. For quantitative data, measurements from these two radiologists were averaged.

Pathological Data
The MSI status of MMR proteins (MLH1, MSH2, PMS2, and MSH6) was evaluated by immunohistochemistry staining. RC patients were divided into MSI group and MSS group based on whether they were deficient in one or more MMR proteins [2]. Other pathological data included differentiation, pathological tumor (pT) stage, lymph node (pN) stage, and EMVI.

Tumor Segmentation
Before tumor segmentation, A.K. software (Analysis Kit, GE Healthcare, Hangzhou, China) was used to adopt T 2 WI as the template for the rigid registration of T 1 WI, ADC, and +C sequences to ensure that the four sequences contained the same resolution, spacing, and origin. The standardized T 2 WI images were imported into open-source ITK-SNAP software, and the whole rectal tumor was segmented slice-by-slice to determine the volume of interest (VOI) for each patient by a radiologist with 5 years of experience in rectum MRI.
According to the registration of different sequences, T 1 WI, ADC, and +C can share the same VOI obtained from T 2 WI. Then, the segmentation results were validated by another radiologist with more than 10 years of experience using intraclass correlation coefficient (ICC) on a cohort of 30 randomly selected patients. The tumor segmentation procedure is shown in Figure 2.
Before tumor segmentation, A.K. software (Analysis Kit, GE Healthcare, Hangzhou, China) was used to adopt T2WI as the template for the rigid registration of T1WI, ADC, and +C sequences to ensure that the four sequences contained the same resolution, spacing, and origin. The standardized T2WI images were imported into open-source ITK-SNAP software, and the whole rectal tumor was segmented slice-by-slice to determine the volume of interest (VOI) for each patient by a radiologist with 5 years of experience in rectum MRI. According to the registration of different sequences, T1WI, ADC, and +C can share the same VOI obtained from T2WI. Then, the segmentation results were validated by another radiologist with more than 10 years of experience using intraclass correlation coefficient (ICC) on a cohort of 30 randomly selected patients. The tumor segmentation procedure is shown in Figure 2.  Figure 2 shows the representative results of the whole tumor on T2WI, T1WI, ADC, and +C sequences using ITK software. Three-dimensional volumetric reconstruction of segmented lesion is shown at the bottom right.

Radiomics Features Extraction and Selection
All segmented VOIs were imported into the Pyradiomics-based PHIgo software (GE Healthcare, V1.2.0, Hangzhou, China) for feature extraction. A total of 1037 radiomics features were extracted from each sequence, including four groups: (1) Figure 2 shows the representative results of the whole tumor on T 2 WI, T 1 WI, ADC, and +C sequences using ITK software. Three-dimensional volumetric reconstruction of segmented lesion is shown at the bottom right.
The ICCs of the measurements from the two radiologists were applied to evaluate inter-observer reliability and reproducibility. Features with ICCs > 0.80 were considered robust features. Then, dimension reduction was performed using analysis of variance, correlation test, univariate logistic analysis, and a gradient-boosting decision tree (GBDT) to reduce data redundancy and to further select the best significant radiomics features. Among them, gradient boosting sequentially combines weak learners in such a way that each new learner fits to the residuals from the previous step. The final features aggregate the results from each step and achieve powerful radiomics feature selection.

Model Construction and Evaluation
Five machine learning algorithms, including logistic regression, Bayes, support vector machine (SVM), K-nearest neighbor (KNN), and tree algorithms were used to construct radiomics models. The area under the receiver operating characteristic (ROC) curve (AUC) and DeLong test were used to evaluate the performance of different models. The 500 bootstrap method and its relative standard deviation (RSD) were taken to quantify the stability of these five algorithms. RSD = (the standard deviation of the 500 AUCs of each algorithm)/(the corresponding mean value of the 500 AUCs) × 100% [30]. The lowest RSD represented the best stability of the algorithm. Radiomics score (rad-score) was calculated via a linear combination of remaining features that were weighted by their respective coefficients to quantify the discriminability of the best radiomics model. The Hosmer-Lemeshow test were used to assess the goodness-of-fit of the best model. Then, patients were classified into high-risk and low-risk groups according to the best model to evaluate the predictive performance.

Statistical Analysis
Statistical analyses were performed with SPSS software (version 24.0, Chicago, IL, USA) and R software (version 3.4.1, Vienna, Austria). The two-sample t test or Mann-Whitney U test if not normally distributed was used for continuous variables, and results were expressed as mean ± standard deviation or median (interquartile range). Chi-squared test or Fisher's exact test was used for categorical variables, and the results were expressed as numbers (percentages). Statistical significance was set at two-sided p < 0.05.

Patients' Characteristics
Among the 383 patients, the prevalence of MSI was 14.62% (56/383). In terms of MSS and MSI groups, there were no significant differences in any of the clinical, radiological, and pathological variables between the two groups in the whole, the training, and the test sets (p > 0.05, Tables 1 and 2).

Radiomics Features Selection
A total of 4148 radiomics features were extracted from each patient. Then, 2816 robust features with ICCs > 0.80 were obtained and used for dimension reduction. Firstly, analysis of variance on the 2816 robust features was performed. The variance of each feature was calculated, and then the features greater than the threshold 1 were retained. In this study, analysis of variance selected 1752 features. Secondly, dimensionality reduction of the selected 1752 features was performed using correlation test and univariate logistic analysis, and 234 and 22 features were selected successively. Following GBDT, 11 features were ultimately retained from the four sequences-T 2 WI (n = 1), T 1 WI (n = 1), ADC (n = 2), and + C (n = 7)-to construct the radiomics models. Among these 11 features, there were 4 LoG transform features and 7 wavelet transform features, as shown in Figure 3.

Model Construction and Comparison
The RSD value of the radiomics model based on logistic regression was the lowest (4.64%), followed by Bayes (5.44%) and KNN (5.45%), which was significantly better than that of the SVM (19.11%) and tree (11.94%) algorithms, as shown in Figure 4.
Among the five different radiomics models, the logistic model performed best with AUCs of 0.827 and 0.739 in the training and test sets, respectively, followed by the Bayes model with AUCs of 0.817 and 0.713, respectively, although there were no statistical differences (p > 0.05, Table 3 and Figure 5). In addition, the DeLong test showed that the prediction performance of the logistic model performed better than that of the SVM model (AUC = 0.783, p = 0.013) and the tree model (AUC = 0.590, p < 0.001) in the training set. Furthermore, the logistic model performed better than that of the KNN model (AUC = 0.606, p = 0.047) and the tree model (AUC = 0.520, p < 0.001) in the test set (Table 3 and Figure 5). Additional performance metrics of the logistic model (F1 score, 0.417; Matthews correlation coefficient, 0.337; G-mean, 0.723) were all higher than those of the other four models in the test set. 2), and + C (n = 7)-to construct the radiomics models. Among these 11 features, there were 4 LoG transform features and 7 wavelet transform features, as shown in Figure 3.

Model Construction and Comparison
The RSD value of the radiomics model based on logistic regression was the lowest (4.64%), followed by Bayes (5.44%) and KNN (5.45%), which was significantly better than that of the SVM (19.11%) and tree (11.94%) algorithms, as shown in Figure 4.  2), and + C (n = 7)-to construct the radiomics models. Among these 11 features, there were 4 LoG transform features and 7 wavelet transform features, as shown in Figure 3.

Model Construction and Comparison
The RSD value of the radiomics model based on logistic regression was the lowest (4.64%), followed by Bayes (5.44%) and KNN (5.45%), which was significantly better than that of the SVM (19.11%) and tree (11.94%) algorithms, as shown in Figure 4.
The Rad-scores in the MSI group were significantly higher than in the MSS group in both the training and test sets (p < 0.05, Figure 6). Red represents MSS, and blue represents MSI in the rad-score plot (Figure 7a). Patients with rad-scores greater than −2.260 were stratified into the high-risk group, and the others were stratified into the low-risk group. There were significant differences in the number of patients whose predicted MSI were between the low-risk and high-risk groups in both the training and test sets (p < 0.001), indicating the clinical applicability of the logistic model (Figure 7b).  The Rad-scores in the MSI group were significantly higher than in the MSS group in both the training and test sets (p < 0.05, Figure 6). Red represents MSS, and blue represents MSI in the rad-score plot (Figure 7a). Patients with rad-scores greater than −2.260 were stratified into the high-risk group, and the others were stratified into the low-risk group. There were significant differences in the number of patients whose predicted MSI were between the low-risk and high-risk groups in both the training and test sets (p < 0.001), indicating the clinical applicability of the logistic model (Figure 7b).

Discussion
The preoperative prediction of MSI is of great significance for clinical decision making and prognosis. In this study, five different machine learning algorithms were compared, and the results showed that the logistic regression algorithm had the best stability. The logistic radiomics model based on multiparametric MRI can effectively predict MSI status and proved that it had great potential in the noninvasive preoperative prediction of MSI in RC patients.
Imaging can better capture the overall heterogeneity of the tumor and is superior to invasive tissue biopsy with sampling error due to insufficient samples or sampling techniques. Several studies have reported the correlation between CT-based radiomics and MSI status in colorectal cancer patients [19][20][21][22][23]. However, CT scan results in more radiation in patients receiving preoperative therapy and follow-up. Multiparametric MRI can provide more useful information and is recommended as the preferred examination for RC patients. Moreover, compared with MSS, colorectal cancer patients with MSI have distinct clinical and pathological features, including proximal colonic dominance and poor tumor differentiation [22,23]. In our study, the prevalence of MSI was 14.62% (56/383) in RC cases, which was consistent with the incidence of 10% to 20% in previous studies [26,31,32]. There were no significant differences in clinical, radiological, and pathological features between MSI and MSS in our study. Therefore, it is urgent to mine more in-depth quantitative radiomics features based on multiparametric MRI to predict MSI in RC patients.
Currently, only a few recently published studies have developed MRI-based radiomics for the preoperative prediction of MSI in RC patients [31][32][33]. However, the radiomics features extracted from these studies were all first-order, shape-based, and texture features. In addition to the above features, LoG and wavelet transform features were extracted in our study, in accordance with the Image Biomarker Standardization Initiative (IBSI) [34]. Surprisingly, the 11 features retained in our study were all LoG (n = 4) and wavelet (n = 7) transform features, indicating that transform features can capture more valuable information related to MSI in RC patients and better reflect the biological characteristics and heterogeneity of tumors [35]. Furthermore, the rad-score calculated based on these 11 features was significantly higher in MSI than in MSS, which was consistent with a previous study on CT-based radiomics for predicting MSI in colorectal cancer [22]. These findings indicate that quantitative radiomics features have certain value in predicting MSI in RC patients, which deserves further research and exploration.
In addition, radiomics models based on machine learning algorithms have attracted great attention to overfitting. Previous studies have used different algorithms to construct models [31][32][33]. The most valuable part of our study was the comparison of five different commonly used machine learning algorithms. The results showed that logistic regression algorithm had the lowest RSD (4.64%), followed by Bayes (5.44%) and KNN (5.45%), which was significantly better than that of SVM (19.11%) and tree (11.94%) algorithms. Therefore, the logistic regression algorithm with the best stability had great potential in predicting the MSI status of RC patients. Besides, the radiomics model based on the logistic regression algorithm performed best in predicting MSI, with AUCs of 0.827 and 0.739 in the training and test sets, respectively. Encouragingly, the logistic model can also afford the good classification and recognition of MSI status in RC patients, further demonstrating its superiority in clinical application. These confirmed that radiomics based on multiparametric MRI can noninvasively extract deeper quantitative image information and timely capture and reflect the biological characteristics of tumors. These also realized the preoperative individualized prediction of MSI status in RC patients, which was in line with the current trend of personalized and precise medicine. Some limitations should be noted. Firstly, this retrospective study may lead to selection bias. Secondly, this study was a single-center study with a limited sample size. Therefore, further studies using large-scale multicenter prospective study are needed to reduce the impact of selection bias and to verify our findings. Finally, the manual segmentation may be affected by subjective evaluation, which may not be suitable for data processing in large samples. It is necessary to further find a suitable algorithm for automatic segmentation.

Conclusions
In conclusion, we compared and selected the optimal logistic regression machine learning algorithm to construct a radiomics model based on multiparametric MRI in this study. The logistic model was an effective and noninvasive approach for predicting the MSI status of RC patients and demonstrating better predictive performance, which could potentially be used to facilitate the individualized prediction of MSI status. Our study also provided important evidence for the potential use of the radiomics model for individualized treatment and improve the long-term survival outcomes of RC patients in the future. Informed Consent Statement: This retrospective study was approved by our institutional review board, which waived the requirement for patients' informed consent.

Data Availability Statement:
The data presented in this study are available on request from the corresponding author.