A Fast and Efficient Semi-Unsupervised Segmentation and Feature-Extraction Methodology for Artificial Intelligence and Radiomics Applications: A Preliminary Study Applied to Glioblastoma

Abstract

Brain tumors are pathologies characterized by a high degree of mortality. An early diagnosis of these pathologies could reduce mortality and limit the adverse effects of brain surgery. Computer-aided tomography (CT), and magnetic resonance imaging (MRI) are fundamental diagnostic methods. They offer lots of helpful information that help medical operators to make an early and effective diagnosis. However, a human operator must analyze and classify the enormous amount of data provided. This process is time-consuming, and sometimes the information is not directly visible to the human eye, leading to lost essential information that could be useful for obtaining a correct and early diagnosis. In such a scenario, the development of suitable tools aimed at helping the human operator is essential. In particular, artificial intelligence (AI) methodologies could help the clinical operator correctly classify different tumoral pathologies, suggest more appropriate therapy, and support the surgeon in reducing invasiveness. All AI systems require a so-called training phase and suitable feature identification to work properly. In this work, we propose a tool to speed up brain tumor segmentation and feature extraction. In particular, we focus on Glioblastoma (GBM), a brain tumor characterized by high tissue heterogeneity and difficult segmentation. The method has been assessed by considering an experimental dataset belonging to the Radiomic Laboratory of the University of Trento. The obtained results are encouraging and demonstrate that the proposed method can be very useful to speed up the pathologies segmentation and features extraction compared to other well-known methods.

1. Introduction

Brain tumors are pathologies characterized by a high mortality and disability rate. The analysis of histopathological specimens is the gold standard method in the evaluation of brain tumors. However, these methods are invasive, and standard biopsies may lead to incorrect results due to intratumoral heterogeneity. Moreover, some tumors cannot be resectable, since they are in a critical area. In such a critical scenario, an early detection of the pathology is mandatory to limit invasive surgery or to make more effective treatments aimed at reducing or destroying the tumor. On the other hand, medical imaging procedures can evaluate the entire tumor in a non-invasive and reproducible way. In particular, a non-invasive method that provides an accurate pre-surgical diagnosis has the potential to improve patient treatment planning from the initial presentation. Many non-invasive imaging techniques have been successfully employed for the diagnosis and the study of brain tumors, such as computed tomography (CT), magnetic resonance imaging (MRI) and positron emission tomography (PET). In particular, MRI represents, to date, the reference point in the study and characterization of brain tumors since it is a multi-parametric, non-invasive method that provides biological information of the sample. Moreover, MRI provides good soft tissue contrasts using different sequences. Among them, the most used are T1-weighted (T1w), T1-weighted with contrast enhancement (T1w+c), T2-weighted (T2w), and Fluid-Attenuated Inversion Recovery (FlAIR) [1,2]. However, although conventional MRI can evaluate the entire tumor in a non-invasive way, the information detected by the naked eye of a physician is limited. In this context, artificial intelligence (AI)-based automatic segmentation methods have shown great advantages in medical image analysis [3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20]. These methods are able to extract features from the tumor tissues that are useful for classifying from the original data healthy from malignant tissues, or identifying meaningful patterns, imaging biomarkers [21] or other analogies that can help provide a correct diagnosis for a given pathology. In particular, features are fundamental for a recent diagnostic approach called radiomics [22,23]. Radiomics is a low-cost, fast and non-invasive methodology based only on multimodality medical images. Radiomics has been successfully applied to different pathologies with very good results [24,25,26,27,28]. After the detection of pathologic tissues, it is of fundamental importance to separate tumor tissue, necrosis and edema from normal and healthy brain tissues. This procedure, called segmentation [29], is fundamental not only for a correct medical diagnosis, but also to plan surgical and radio treatments. Tumor segmentation is usually hand-made, and it is a time-consuming task and must be performed by expert radiologists. If the operator only delineates the region of interest (ROI), a poor segmentation result is usually obtained since necrosis, edema and other tissues could belong to the ROI. Therefore, automatic segmentation or semi-supervised segmentation methodologies are an important topic that has attracted the attention of different research groups involved in the development of effective diagnostic tools. In a previous article [30], we proposed a semi-automatic methodology aimed at helping the identification and segmentation of tumoral pathologies, using a combination of binary texture recognition, the growing-area algorithm, and machine-learning techniques. In particular, the method proposed in [30] not only helps to better identify pathologic tissues better but also permits quick analysis of the huge amount of data, in Dicom format, provided by MRI. This tool was very effective in detecting and segmenting gliomas tumors, since it was demonstrated to outperform a 3D segmentation technique slicer. Unfortunately, when we tested the methodology with tumors characterized by a heterogeneous structure such as glioblastoma and astrocytomas that are considered to be highly invasive and malignant tumors, the method failed. In particular, the pattern recognition was ineffective, since the heterogeneous structure of these pathologies confuses the pattern recognition algorithm that interprets healthy brain tissue as malignant tissues. This happens because glioblastoma and astrocytomas are very heterogeneous and characterized by different tissues such as necrotic tissues, edema, blood, and vessels. The previous published methodology completely fails and is unable to correctly identify the malignant tissue. The segmentation must be done by hand with expert neuroradiologists, so it requires a lot of time and human resources. In this work, we’ll try to overcome this problem. We propose a fast semi-automatic methodology for the segmentation of malignant tissues and feature extraction. The new methodology proposed in this work can manage heterogeneous structures and it is particularly suitable for glioblastoma pathologies, providing a fast and accurate method for the segmentation of such pathologies. Additionally, inexpert student operators can be easily trained to use the tool in a very limited amount of time. The methods make use of a combination of image-processing techniques, segmentation algorithms, and graphical tools. In particular, the proposed methodology supports human operators to identify pathologic tissues in a fast and easy way, and to immediately extract the features mandatory for any AI algorithms and radiomics. It permits quick analysis of the huge amount of data, in Dicom format, provided by MRI. The proposed method is an experimental assessment, considering a real medical MRI database of glioblastoma brain tumors patients. In particular, a medical dataset of MRI data in Dicom format, composed of 136 patients with GBM, has been collected. The dataset construction was supported by a qualified neuroradiologist with more than 15 years experience who also provided the correct diagnosis of the pathologies and the correct segmentation used to assess the proposed methodology. The obtained results were compared with a widely used segmentation software tool. The results are very promising in terms of time reduction, accuracy, and efficacy. It is worth noting that the proposed methodology not only provides a fast and accurate segmentation tool for neuroradiologists, but make them able to formulate fast, correct and accurate diagnosis. The possibility to analyze a high volume of data with high accuracy and in a reduced amount of time contributes to the earlier diagnosis of tumors. Moreover, the new friendly graphical interface permits the simplification of the training phase of inexpert operators. The manuscript is organized as follows: Section 2 describes the new segmentation methodology and its graphical interface. Section 3 reports a detailed description of the considered medical dataset related to real patients with glioblastoma pathologies. In Section 4, the new segmentation methodology is experimentally assessed and compared with other state-of-the-art segmentation tools. Finally, Section 5 concludes.

2. Method Description

Figure 1 reports an example of the heterogeneous structure of GBM pathology. In particular, the example reported in Figure 1 refers to Patient 1 and protocol Axial T1+C. The semi-unsupervised segmentation procedure proposed in our previous work cannot be applied to these pathologies, since GBMs are too heterogeneous. The development of a suitable system to manage glioblastoma pathologies is mandatory and is reported in this section. The procedure workflow is summarized in Figure 2. The first step of the new procedure consists of acquiring MRI data in Dicom format; different MRI protocols can be considered. The tool requires a patient identifier and the protocol we want to analyze. A 2D representation has been considered for 3D visualization, since this kind of representation is preferred by medical operators. The second phase is aimed at manually selecting the slices where the operator suspects the presence of pathological tissues. After the slice selection procedure, a red flag is placed on each selected slice, as reported in Figure 3a. After the selection procedure, at each marked slice, a seed must be manually associated as reported in Figure 3b. Then, the proposed tool permits the fast testing of the efficacy of a selected seed by applying a set of image-processing algorithms. In particular, the first step considers the application of a Gaussian filter, with a moving window of dimension $3\times 3$ pixels, aimed at reducing the noise. Then, artifacts of low dimensions (one or two pixels) are removed using an erosion filter [31], and the image is binarized using the algorithm based on an adaptive threshold proposed in [32]. Small object removal is conducted with a $3\times 3$ pixel structural mask. The structural mask is moved from left to right and from top to bottom pixels of the original image. When the mask completely overlaps the image pixels, the central pixel of the image is set to 0. The whole process is summarized in Figure 4. The last step consists of the application of the area-growing algorithm [33,34,35] aimed at selecting the malignant tissue characterized with a voxel intensity similar to the selected seed. The results of these operations are reported in Figure 3c. If the operator considers the segmentation acceptable, the seed is stored and the slice is marked with a blue asterisk, indicating the seed position as shown in Figure 3d. If the user chooses a seed for each slice, the tools will extract the mask and show the results as reported in Figure 5. If seeds for some slices are missing, the tool automatically associates the nearest seed to provide each slice with a suitable seed. This is particularly useful when the pathology is symmetrical. In this scenario, the operator will choose only one seed the tool will provide to associate the same seed to the other marked slices obtaining a strong reduction of the segmentation time. Then, the tool permits the manual modification of the masks, by adding or removing marked tissues areas with a right and left mouse click, respectively. The removed/added area can be manually selected by the operator. The adjusting phase window is shown in Figure 6. The following steps are aimed at saving the obtained masks in nrrd format in order to keep the compatibility with other state-of-the-art software tools commonly adopted by clinicians, namely Slicer 3D, Horos and Pyradiomics. Then, the tool estimates the tumor volume and shows its 3D representation with the correct position in the head. The 3D visualization could be very useful to plan a surgical treatment, since the pathology can be printed with additive 3D printers and positioned in a virtual head to allow medicine students and surgeons to practice. An example of 3D visualization is reported in the following Figure 7. The tool is directly interfaced with Pyradiomics, a GNU software aimed at extracting 144 different features mandatory for the learning phase of every artificial intelligence (AI) algorithm. The last phase of the procedure is the estimation and storage of all features. The proposed segmentation procedure is quite simple with respect to the method proposed in [30]; however, the results are quite efficient for the segmentation of heterogeneous pathologies such as glioblastoma tumors. In particular, the new approach schema has been strongly simplified, and the pattern recognition algorithm, aimed at identifying the malignant tissues, has been removed since it was ineffective when dealing with heterogeneous pathologies. This simplification makes the application of area-growing algorithm, aimed at identifying the malignant area, very fast. In the next sections, the methodology will be assessed by considering an experimental dataset and compared with a well-known segmentation tool commonly used by neuroradiologists, namely Slicer 3D.

3. Database Description

Another important goal of this work is to provide a large database of GBM pathologies for training neuroradiology students to learn more about this aggressive disease and provide them software tools aimed at formulating fast and correct diagnosis. With the development of a medical archive, it is very important to understand the evolution of different kinds of pathologies [36,37,38]. In particular, a database of 136 patients affected by glioblastoma pathology is provided. The medical images of the tumor were acquired by a heterogeneous scanner of MR; specifically, the data were collected from both 1.5T and 3T MRI scanners (GE Optima MR450w 1.5T, Waukesha, WI, USA; Ingenia, 1.5T, Philps, The Netherlands; Signa Excite, MAGNETOM Skyra, 3T Siemens Healthcare, Erlangen, Germany). The following

Table 1. Considered MRI scanners and related spatial resolutions.

Scanner	Magnetic Field Strength [T]	Section Thickness [mm]	Pixels Spacing [mm]
Sigma HDxt	1.5	4.0	$0.44\times 0.44$
Sigma Explorer	1.5	3.0	$0.49\times 0.49$
Sigma HDxt	3.0	4.0	$0.47\times 0.47$
Skyra	3.0	4.0	$0.47\times 0.47$
Skyra	3.0	4.0	$0.45\times 0.45$

summarizes the MRI models considered and the related spatial resolution.

The following conventional MRI protocols were acquired: axial T1-weighted (T1w) fast spin-echo (FSE), axial T2-weighted (T2w) fast relaxation and spin-echo propeller sequences (FRFSE-propeller), and axial T2w fluid-attenuated inversion recovery imaging (FlAIR). The spatial resolution of the acquired images are thickness between 4 and 5 mm, intersection gap (1 mm) and FOV (240 × 240 mm). After intravenous contrast-agent injection, a 3D fast-spoiled gradient-echo (FSPGR) sequence was acquired (1 mm isotropic voxel). DWI was performed in the axial plane with an echo-planar sequence before the injection of the contrast agent (gadobutrol, 0.1 mmol/kg) with a section thickness of 4 mm, intersection gap 1 mm, and FOV 240 × 240 mm. Other protocols are present, such as GRE, in the database even if they are not named. It is worth noting that the protocols have different names for different scanners; the one reported refers to the GE scanner. The above database has been used to assess the proposed semi-unsupervised software tool. Each patient belonging to the database has been diagnosed by an expert neuroradiologist, who provided the correct manual segmentation considered real truth. The obtained results were compared with a well-known segmentation tool, namely Slicer 3D.

4. Experimental Assessment

This section is aimed at the experimental validation of the proposed segmentation methodology. Only the T1 axial sequence and a selected set of 20 subjects extracted from the GBM database are considered in this assessment. The patient subset and the related data are reported in the following

Table 2. Description of the considered patients subset and related data.

Patient	Sex	Age	MRI Scanner
1	M	70	Sigma HDxt
2	F	47	Sigma HDxt
3	F	69	Skyra
6	M	61	Sigma HDxt
7	M	51	Sigma HDxt
9	M	72	Sigma HDxt
10	M	56	Sigma HDxt
15	F	56	Skyra
18	F	76	Sigma HDxt
24	M	52	Sigma HDxt
26	M	66	Sigma HDxt
28	M	46	Skyra
29	M	71	Skyra
30	M	77	Sigma HDxt
37	F	71	Sigma Explorer
38	M	57	Sigma HDxt
40	M	75	Sigma HDxt
44	M	65	Sigma HDxt
45	M	75	Sigma HDxt
51	M	70	Skyra

. The segmentations have been performed with the proposed methodology and then compared with those obtained with a 3D Slicer, one of the best-known pieces of segmentation software and commonly used by neuroradiologists. The first scenario is related to Patient 1, with a GBM located in the right parietal lobe. An expert neuroradiologist provided a complete manual segmentation with Slicer 3D software, and the results are reported in Figure 8. It is considered the absolute reference truth. Then, to assess the proposed methodology in realistic conditions, the same segmentation was performed by an operator with low clinical experience in brain anatomy and pathology. Then, the operator made the segmentation with Slicer 3D software, using a multi-seed growing-area semi-unsupervised procedure. The obtained results are reported in Figure 8. The segmentation with Slicer 3D required about 25 minutes, while the one obtained with the new tool only needed 8 minutes. Concerning the pathology volume of interest (VOI), as can be observed, Slicer 3D provided a satisfactory segmentation even if an extra slice was wrongly identified. The segmentation was then performed with the new tool following the guidelines reported in Section 2. Only half an hour of training was enough to train the operator thoroughly. Results of the further segmentation are reported in Figure 9; as can be observed, the selected slices are the same as the reference segmentation. Figure 10 shows the results obtained with the new tool; as can be observed, the obtained segmentation is very good and similar to the reference real truth, as confirmed by the following error figures. Reference VOI was 13.61 cm${}^3$, and VOI estimated with Slicer 3D and the new tool were 8.42 cm${}^3$ and 11.69 cm${}^3$ with errors of $38.15\%$ for Slicer 3D and $14.12\%$ for the new tool. The error figure has been estimated with the following formula:

$$\epsilon \left(VO{I}_{\left(est\right)}\right)=\frac{\left|VO{I}_{\left(ref\right)}-VO{I}_{\left(est\right)}\right|}{VO{I}_{\left(ref\right)}}$$

(1)

In this first simple scenario, the advantages of the new tool are indisputable. The next experiment considers a more complex design related to Patient 10, and is characterized by an extended pathology in two different brain areas. Moreover, the tumor section localized in the left parietal lobe is characterized by an extended necrosis that makes segmentation difficult to perform automatically. The reference, Slicer 3D and new tool segmentations of Patient 10 are reported in Figure 11, Figure 12 and Figure 13, respectively. The segmentations have been performed with a single slice seed and without manual adjustment. The pathology volume is 29.15 cm${}^3$, while the estimation with Slicer 3D and the new tool are 16.40 cm${}^3$ and 19.19 cm${}^3$, respectively, with an error of $\epsilon =43.7\%$ for slicer and $\epsilon =34.1\%$. The obtained errors are unacceptable for clinical applications. Still, it is worth noticing that the results have been obtained only with the semi-unsupervised procedure and that a manual calibration phase will report errors below an acceptable threshold. Concerning the segmentation required time, they are comparable: 35 minutes for Slicer 3D and about 25 minutes with the new tool. For completeness, the 3D visualizations with both methodologies are reported in Figure 14 and Figure 15, respectively. Additionally, in this experiment, the new tool provided indisputable advantages to Slicer 3D. The third considered scenario is related to a GBM of very extended dimension localized in the right parietal lobe; the patient identifier is 40. The pathology presents a volume of 43.91 cm${}^3$, extended necrosis, and also, in this scenario, it is quite challenging to automatically perform the segmentation. Additionally, for this scenario, segmentation has been achieved with the semi-unsupervised growing-area algorithm, with only one seed for each slice. The volume estimation error for Slicer 3D and the proposed method are $42.62\%$, and $15.31\%$. The slices related to the segmentation of Patient 40 obtained with the new methodology are reported in Figure 16. The segmentation time with the same operator was 25 minutes with Slicer 3D and 20 minutes with the new tool. The segmentation time is comparable, but the accuracy of pathologic volume estimation provided with the new tool strongly outperforms that obtained with Slicer 3D. Indeed, both segmentations are not acceptable, and a manual tuning phase is mandatory. However, the segmentation obtained with the new tool requires only a few adjustments to be sufficient, while segmentation using Slicer 3D requires a long manual cleaning phase due to the high error. All the patients belonging to the subset of Table 1 were considered in order to complete this preliminary experimental assessment. The segmentation was performed considering both segmentation methodologies, and compared with the reference segmentations performed by an expert neuroradiologist and inserted in the database. Features of each pathology were extracted and could be used for training AI algorithms. One of the most crucial parameters was the volume occupied by the pathological tissue [39]. Patients 10, 15, and 30 were characterized by multiple invasive pathologies, while Patients 40, 44, and 29 were characterized by very extended pathologies with a volume greater than 40 cm${}^3$. The results of pathology volume estimation with the considered segmentation methodologies are summarized in

Table 3. Pathological volume of interest estimation (VOI). Comparisons between actual VOI and VOIs estimated with Slicer 3D and the new methodology.

Patient	Actual VOI	Slicer 3D VOI	New Method VOI
1	13.61	8.41	11.69
2	1.41	3.41	1.67
3	13.57	10.75	10.73
6	30.85	15.74	28.81
7	20.82	21.77	23.41
9	23.74	34.47	51.79
10	29.15	16.40	19.10
15	31.50	22.00	24.00
18	29.15	28.13	22.49
24	15.50	18.99	12.91
26	22.15	12.72	23.36
28	26.52	23.02	29.08
29	40.09	43.92	45.14
30	29.52	2.07	28.45
37	32.65	16.11	36.30
38	35.82	20.77	56.08
40	43.91	25.20	50.64
44	44.81	65.09	54.51
45	21.68	19.27	27.41
51	7.29	8.29	8.03

. As can be observed from the data reported in Table 3, the proposed method strongly outperforms Slicer 3D. The performances for Patients 3, 15, 28, and 29 are comparable. Slicer 3D exceeds the new methodology only for Patient 18. For Patients 2 and 30, Slicer 3D provided an error more significant than $100\%$, while the new methodology provided an error of less than $18\%$ for Patient 2 and less than $5\%$ for Patient 30. This is confirmed by the error figure reported in Figure 17 and estimated with relationship (1). Concerning the segmentation time, it is comparable to complex scenarios characterized by multiple or extended pathologies, such as Patients 10, 15, 29, 30, 40, and 44. For simple scenarios characterized by small and regular contour pathologies, the segmentation with the new tool requires only a few minutes, while for Slicer 3D, the segmentation time remains 20 minutes.

5. Conclusions

This work presents a simple and effective approach for the semi-unsupervised segmentation and feature extraction of brain tumors characterized by high tissue heterogeneity. The method is particularly suitable for AI approaches such as SVM classifiers or radiomics applications. In particular, the method has been experimentally assessed with MRI, in DICOM format, provided by a dataset of 136 actual patients suffering from GBM. The considered medical data concern real patients. Expert neuroradiologists, as well as medicine students, have been trained to use the tool and used it to provide diagnoses that have been checked by an expert neuroradiologist who formulated the diagnosis reported in the dataset. The conclusion was that the new methodology offers a good trade-off between high accuracy and applicability of equipment. In particular, the proposed tool permits the segmentation of the pathology using a combination of filtering and growing-area algorithms. Then, the features can be extracted and used to provide the training phase of AI algorithms, such as classifiers and regressors, based on different AI algorithms. They can be very useful in giving a correct diagnosis, helping expert operators to process more patients, and assisting inexperienced ones in providing an accurate diagnosis. The preliminary results are encouraging and have been compared with a widely used segmentation technique, Slicer 3D. Concerning the obtained performances, they are quite satisfactory. Specifically, the new methodology outperforms the state-of-the-art reference tool (namely Slicer 3D) in almost all considered scenarios. Moreover, thanks to the new friendly graphical interface, the time required for the training phase of inexpert operators, students or clinicians is strongly reduced. Despite the proposed tool being assessed only on brain tumors, characterized by high heterogeneity (the glioblastomas), future works will be devoted to extending/customizing the proposed methodology to other tumors located in other sites of the body, or in deep tissues. We also plan to organize and acquire other medical datasets related to other tumors.