Automatic Multiorgan Segmentation in Pelvic Region with Convolutional Neural Networks on 0.35 T MR-Linac Images

Abstract

MR-Linac is a recent device combining a linear accelerator with an MRI scanner. The improved soft tissue contrast of MR images is used for optimum delineation of tumors or organs at risk (OARs) and precise treatment delivery. Automatic segmentation of OARs can contribute to alleviating the time-consuming process for radiation oncologists and improving the accuracy of radiation delivery by providing faster, more consistent, and more accurate delineation of target structures and organs at risk. It can also help reduce inter-observer variability and improve the consistency of contouring while reducing the time required for treatment planning. In this work, state-of-the-art deep learning techniques were evaluated based on 2D and 2.5D training strategies to develop a comprehensive tool for the accurate segmentation of pelvic OARs dedicated to 0.35 T MR-Linac. In total, 103 cases with 0.35 T MR images of the pelvic region were investigated. Experts considered and contoured the bladder, rectum, and femoral heads as OARs and the prostate as the target volume. For the training of the neural network, 85 patients were randomly selected, and 18 were used for testing. Multiple U-Net-based architectures were considered, and the best model was compared using both 2D and 2.5D training strategies. The evaluation of the models was performed based on two metrics: the Dice similarity coefficient (DSC) and the Hausdorff distance (HD). In the 2D training strategy, Residual Attention U-Net (ResAttU-Net) had the highest scores among the other deep neural networks. Due to the additional contextual information, the configured 2.5D ResAttU-Net performed better. The overall DSC were 0.88 ± 0.09 and 0.86 ± 0.10, and the overall HD was 1.78 ± 3.02 mm and 5.90 ± 7.58 mm for 2.5D and 2D ResAttU-Net, respectively. The 2.5D ResAttU-Net provides accurate segmentation of OARs without affecting the computational cost. The developed end-to-end pipeline will be merged with the treatment planning system for in-time automatic segmentation.

1. Introduction

External radiation therapy is a widely used curative treatment option for patients with prostate cancer [1]. In radiotherapy, the optimization of the tumor local control and the minimization of the radiation toxicities are highly linked to the accuracy of the definition of the target volume and the OARs [2]. The treatment planning procedure is conventionally based on 3D anatomical information obtained via computed tomography (CT) scans. Nevertheless, because of its superior soft tissue contrast, MR imaging has been used in radiotherapy treatment planning for several years [3]. The better quality of MR images improves the organs’ delineation and reduces the contouring uncertainty [4,5]. Especially for prostate cancer, an MR-only workflow has become more and more common [6,7,8]. In addition, the recent integration of magnetic resonance imager (MRI) on linear accelerators (LINAC) for magnetic resonance-guided radiotherapy (MRgRT) allows the direct use of MR images for the planning and the guidance of treatment. Indeed, because it is a non-ionizing imaging device, MR images can be acquired continuously during the treatment to monitor the radiation activation (gating process), and the on-board MR image quality enables the online replanning of the treatment by considering the daily position and shape of the organs (adaptive radiotherapy process). MR-Linacs technology represents a higher cost than classical Linacs; nevertheless, it promises improving the clinical outcomes of prostate cancer [9,10]. First, reduced toxicity has been pointed out [11]. In this context, the MRIdian (ViewRay Inc., Oakwood, OH, USA) MR-Linac system has been clinically used in our institution since June 2019. It combines a 6 MV Flattening Filter-Free (FFF) LINAC with a 0.35 T MR imaging system [12]. Nevertheless, with this device, only manual segmentation is possible on the MR planning images. Manual delineation has been described as a tedious process, increasing the workload of radiation oncologists [13]. The segmentations are also subject to intra- and inter-operator variabilities due to different delineation expertise and subjective evaluation from radiation oncologists. Automatic segmentation techniques, on the other hand, have been developed during recent decades in medical imaging for an accurate and efficient analysis of images. It can produce reliable and precise results in a shorter time frame, improving the speed and accuracy of diagnosis and treatment planning [14,15,16,17]. These issues have been addressed, and a plethora of automatic segmentation techniques have been proposed in the literature in both CT and MR imaging modalities [13,18,19]. Atlas-based, deep learning, and other image-processing techniques have been developed in the last two decades [20,21]. Atlas-based methods have shown acceptable accuracy for the segmentation of OARs. However, most of the results require a significant amount of time from the radiation oncologist to correct regions that contain small areas, imaging artifacts, deformable organs, or organs near the target volume [22]. Due to the issues with atlas-based methods, researchers focused on multi-atlas segmentation (MAS), where multiple atlases are used as prior knowledge instead of individual atlases [23]. However, even multi-atlas methods have limitations, the most notable being the need for considerable computational resources [24].

Currently, a common trend in healthcare is the development of deep neural networks for automatic detection, classification, and segmentation. Deep learning techniques are capable of learning task-specific features without the need for hand-engineered features. Convolutional neural networks (CNNs) are a branch of deep learning that gain complex information from training samples, which is essential for successfully delineating anatomical structures. Specifically in MR imaging, researchers have shown great interest in alleviating MRgRT workflow [19]. For segmentation, a fully convolutional network (FCN) [25] was proposed to leverage the computational limitation by implementing symmetrical encoding and decoding paths to learn low- and high-level features from the input image. Later, Ronneberger et al. [26] proposed a successful encoder–decoder architecture for medical image segmentation called U-Net. Nowadays, a wide variety of implementations rely on the U-Net architecture [27,28,29,30]. Another aspect of neural network models for medical image analysis is the training strategy 2D, 2.5D, or 3D. Each strategy learns features from the training data in a different manner. The most common strategy is 2D, in which a single image (or slice) is passed as an input. Similarly, 2.5D considers stacks of adjacent slices as input, collecting pseudo-3D information using 2D operations [31]. The 3D training strategy is significantly more demanding than the other strategies since it performs 3D operations. Indeed, the 2.5D strategy is regarded as computationally efficient since the memory requirements are reduced [32].

Based on the MR modality, there is considerable literature on semantic segmentation. Maji et al. [33] proposed a modified 2D version of Attention U-Net [26,30,34] with an attention-guided decoder for semantic segmentation of brain tumors for MR-only treatment planning. Aldoj et al. [35] proposed densely connected blocks instead of plain convolutional blocks [36] in a U-Net-based architecture for the automatic segmentation of prostate parts. Elguindi et al. [37] developed a multiorgan automatic segmentation model for prostate radiotherapy using transfer learning with the fully convolutional network (FCN) architecture [38]. All the proposed architectures used a combination of Dice and cross-entropy loss functions. In the 2.5D strategy, Alkadi et al. [39] implemented a deep FCN based on SegNet for prostate cancer detection and segmentation using T2-weighted MR images. The input with their method was a single slice replicated into three channels. Similarly, Huang et al. [40] proposed a modified version of Fuse U-Net with an attention mechanism to accurately segment multiple OARs (femoral heads, bladder, rectum, and anal canal) through multi-sequence MR images to facilitate MR-only treatment planning preparation for prostate and cervical cancer. Nevertheless, no study has been found focusing on 0.35 T MR Linac images.

In this context, we intend to develop a deep-learning-based automated multiorgan segmentation approach dedicated to the images from a 0.35 T MR-Linac for the pelvic area. A challenge of this study is the evaluation of the 2.5D strategy to mitigate the lack of contextual information in 2D-based networks. State-of-the-art U-Net variations were configured and compared with a 2D training method for multi-class segmentation in the male pelvis. Then, the best-performing model was trained and tested with 2.5D training. The organs of interest are the pelvic OARs: femoral heads, bladder, and rectum. The prostate was considered as the target volume. MRI studies were acquired from the same machine (0.35 T MRIdian, ViewRay Inc., Oakwood, OH, USA) at two institutions to create a large enough dataset. After evaluating deep learning architectures, an automated tool was developed for clinical use.

This work should reduce the radiation oncologists’ workload and inter-operator variability and consequently reinforce the use of MR images for treatment planning. To our knowledge, it is the first time that the development of a deep-learning-based automatic segmentation tool has been performed on 0.35 T MR-Linac images.

2. Materials and Methods

2.1. Dataset

Patients in this study were diagnosed with prostate cancer and underwent external radiotherapy between June 2019 and February 2022 on a 0.35 T MR-Linac MRIdian (ViewRay Inc., Oakwood, OH, USA) [9]. In total, 103 cases were collected. More specifically, 93 cases were acquired from the Centre Georges-François Leclerc (CGFL, Dijon, France) and 10 cases from the University Hospital of Pitié Salpêtrière (Paris, France). This study was approved by the human subject ethics board of the Centre Georges-François Leclerc and was conducted in accordance with the Helsinki Declaration of 1975, as revised in 2013. A balanced steady-state free precession pulse sequence (bSSFP) was used for MR image acquisition in 3D with a raw image reconstruction in axial orientation, yielding T2/T1-weighted image contrast. The 0.35 T low magnetic field and the original split gradient coil design of the MRIdian is unusual in comparison to diagnostic MR imaging systems. These aspects make the images acquired with this system unique [12], with possible systemic imaging artifacts [41]. Pixel spacing was 1.5 × 1.5 mm², and the slice thickness was 1.5 mm. In addition to the MR images, a radiation therapy (RT) structure set file was associated with each MRI exam. The RT structure file contains the contextual information of the targets and OARs delineated by an expert with an integrated treatment planning system (TPS) of the MRIdian system. In the case of prostate cancer, the whole prostate or the prostate with the seminal vesicle is generally defined as the target volume that must receive the prescribed dose. All the involved data were acquired without restrictions (i.e., artifacts, malignancy level, etc.). The OARs that the model focused on were the femoral heads, bladder, and rectum; the target volume was the prostate. For the proposed approach, only the axial view of the images was used in both training strategies, considering the raw image orientation. Figure 1 shows a sample of the dataset.

2.2. Preprocessing

Firstly, image normalization was performed, changing the pixels’ value range to [0–1] to stabilize the gradient descent step during training. Secondly, centre cropping or padding was performed to homogenize the shape of the images to 256 × 256 according to the model’s input requirements. The last pre-processing step is the minimum filtering used to discard the potential noise in the background (pixels close to zero). MR images often exhibit regions with pixels that are near or reach values close to zero. The aim is to isolate and preserve these pixels inside the body, as they represent regions with minimal signal and are potentially crucial for maintaining the overall structure of the image. This threshold determination is grounded in a histogram-based examination and utilization of the anatomical information from MRI images. The visualization of the minimum filtering is depicted in Figure 2. Data augmentation was also performed for the training process, thereby contributing to overfitting deterrence by increasing the number of samples. More specifically, the data augmentation was random vertical flipping, random rotation with a limit of ±15° and arbitrary scaling with a full scale of 10% of the original image size.

2.3. Residual Attention U-Net Network

The U-Net structure [26] was adopted to consider high-resolution local textures and low-resolution contextual information using encoding and decoding paths. Additional operations were implemented, such as attention gates (AGs) [30] and residual blocks [34]. The integration of the AG into the model provides superior attention to important regions in our case (OARs and prostate) rather than unlabeled regions (background). Also, residual blocks were implemented for in-depth feature information collection in each convolution, increasing the model’s performance and reducing the model’s parameters. The input layer requires a fixed number of slices to perform the 2.5D training strategy. In our study, this fixed number is defined as 3, where the second slice serves as the primary one, encompassing the segmented OARs. The first and third slices, designated as neighboring slices, are also input into the model to gather contextual information. These neighboring slices provide additional features that help the model in considering the broader context in which the regions of interest exist. Then, the input shape of the model is 256 × 256 × 3, representing the height, width, and number of slices, respectively.

The encoding and decoding paths contain four layers in depth; each layer includes a residual block. Between each layer, both encoding and decoding layers are connected through concatenation connections. In the encoder, each residual block is a pair of convolutional blocks with a fixed sequence of 2D convolutions having a 3 × 3 filter size, stride one, padding one, batch normalization, and ReLU operations followed by a 2D max-pooling operation with a kernel size of 2 × 2 and stride 2. After the encoding path, the bottleneck of the model provides a 16 × 16 × 1024 feature map. Then, the attention mechanism is triggered by inputting the feature map from the previous layer and the feature map from the same layer of the encoding path. In the last phase, concatenation is performed between the output of the AG and the up-sampled feature map of the previous layer, followed by the same residual block as in the encoding path. The schematic of Residual Attention U-Net (ResAttU-Net) architecture is presented in Figure 3 and Figure 4.

Multiple basic loss functions were tested during the implementation related to the semantic segmentation task. Among them, Focal Loss (FL) [42] is a modified version of the cross-entropy loss, including an additional term to focus on hard misclassified examples, dealing with the class imbalance issue [43]. Focal loss is defined in Equation (1), where $p_t$ is the estimated probability for the class, and γ is a tunable parameter with γ ∈ [0,5].

$$\mathrm{FL}\left(p_t\right)=-{\mathsf{\alpha }}_t{\left(1-p_t\right)}^{\mathsf{\gamma }}\log \left(p_t\right)$$

(1)

The adaptive moment estimation (ADAM) algorithm was used due to its efficiency [44]. ADAM is an optimization algorithm that updates the network weights iteratively, remaining controlled and unbiased. The learning rate was 0.001, and the batch size was 4 for all the experiments. Additional call-back functions were introduced in the experiments to smooth the learning procedure and decrease the training time. Reduced learning on the plateau was used to reduce the learning rate when there was no update throughout the learning iterations. Additionally, early stopping was implemented to stop the learning process when the model could not learn different features. The patience factor of the call-backs was 15 and 25 epochs, respectively. Apart from the ResAttU-Net, two additional deep learning models were trained to compare the performance of the proposed model using the same hyperparameters: U-Net [26] and Attention U-Net [30].

2.4. Post-Processing

The output of each class in our model is a 2D mask depicting the target’s predicted location, or OAR, in the same view as the input image. Sometimes, the predicted classes are not segmented properly, creating gaps inside the segmented regions. Thus, an automatic algorithm with a combination of post-processing techniques and rules was implemented. Firstly, the predicted mask is binarized with a pixels’ threshold of 0.5. The optimal threshold was determined after an extensive analysis of binarized predicted outcomes. A hole-filling process then eliminates the faulty gaps inside the segmentation. However, the hole-filling method is not performed for the bladder because neighbouring organs are commonly in the inner area (i.e., the prostate).

A series of rules is also helpful in eliminating misclassified pixels. The rules are set according to each organ’s original position and hierarchy. For example, the bladder is localized at the top of the image, followed by the rectum. Similarly, the left femoral head could not be on the right side, and vice versa.

Artifacts are a common issue in medical imaging [45]. The most common artifacts for the acquisitions with the SSFP sequence occur at extreme slices where the magnetic field is less homogeneous.

In whole-volume segmentation, it is a common problem that the model predicts inaccurate segments because of these artifacts. Hence, the classified pixels were eliminated if any organ was shown there according to slice position. In the last step, interpolation proposed by Schenk et al. [46] was used to homogenize the whole volume in three dimensions. Figure 5 illustrates the prediction without post-processing and the results with post-processing.

2.5. Evaluation of the Segmentation Algorithm

The network was trained and evaluated on our dataset, which has 7866 slices containing at least one class per slice. The dataset was divided into two parts: 80% (83 patients) for training and validation, while the remaining 20% (20 patients) was set aside as a distinct test set. The training process further employed k-fold cross-validation with k = 5, enabling a comprehensive evaluation of the model’s performance across different subsets of the training data. Once the cross-validation was completed, all the models were evaluated using the test data. This approach provides an unbiased estimate of the model’s generalization capability. Also, it is worth noting that patient-wise splitting was performed to prevent any slice overlap among the sets. The data division is detailed in Figure 6.

The geometric evaluation of the organs’ delineation was assessed using traditional metrics such as the Dice coefficient (DSC), which is responsible for the global evaluation of the segmentation Equation (2), and the Hausdorff distance (HD), where it is relevant to highlight the outliers Equation (3). These two metrics are then complementary and expressed as the mean ± standard deviation (STD). The Hausdorff distance (HD) is the maximum distance between two contour point sets, A and B, which are the predicted and the ground truth.

$$H\left(A,B\right)=\max \left(\hspace{0.17em}\underset{a\in A}{\sup }\underset{b\in B}{\inf }|a-b|,\hspace{0.17em}\underset{b\in B}{\sup }\underset{a\in A}{\inf }|a-b|\hspace{0.17em}\right)$$

(2)

where $H\left(A,B\right)$ represents the maximum distance between two sets (A and B). The distance between these points is expressed in millimeters (mm). The smaller the HD value, the higher the segmentation accuracy.

The DSC calculates the degree of similarity between the manually segmented and the automatically segmented areas. Both A and B represent the point sets contained in the two regions. The calculation formula is shown in Equation (3).

$$\mathrm{Dice}\left(A,B\right)=\frac{2\left|A\cap B\right|}{\left|A\right|+\left|B\right|}$$

(3)

where |A∩B| represents the intersection of A and B. The value range of DSC is [0–1]. The higher the DCS value is, the better the segmentation results are.

The evaluation was conducted through two experiments. The 2D strategy was carried out in the first experiment to investigate the optimal model, comparing the ResAttU-Net with conventional neural networks such as U-Net and Attention U-Net [26,30]. The DSC and the HD were compared between ResAttU-Net and the other networks using the paired t-test after training. A p-value smaller than 0.05 was considered to represent statistical significance. Then, the best model candidate was retrained with a 2.5D training strategy.

2.6. Computing Environment

The computed environment consisted of the following components: Ubuntu 18.04, CPU: Intel(R) Xeon (R) W-2145 CPU @ 3.70 GHz, memory: 32 GB, GPU: TITAN V 12 GB + Quadro P400 2 GB (NVIDIA Corp., Santa Clara, CA, USA). Mask construction was performed using Python 3.9.10, NumPy 1.22.3, OpenCV 4.5.5, PyTorch 1.11.0, and NVIDIA Compute Unified Device Architecture (CUDA) Deep Learning Network library (cuDNN) 10.2.

3. Results

To demonstrate the efficacy of the residual attention network, we compared the result of the modified ResAttU-Net against the plain U-Net and Attention U-Net architectures. These networks were trained on our MRI dataset. The mean slice number per sample was 77 slices containing the prostate and the OARs (femoral heads, bladder, and rectum). All the architectures were evaluated using k-fold cross-validation, with approximately 8.5 h of training per fold. Firstly, the U-Net, Attention U-Net and ResAttU-Net were trained with the 2D training strategy. Regarding the evaluation metrics, ResAttU-Net performed slightly better than the other architectures (

Table 1. Evaluation of 3 different architectures in 2D strategy. Best results in bold.

Architectures	Dice ± STD	HD ± STD (mm)
U-Net	0.83 ± 0.14	7.95 ± 6.03
Attention U-Net	0.84 ± 0.12	7.50 ± 6.18
ResAttU-Net	0.85 ± 0.11	7.49 ± 6.54

). Concerning the DSC, the differences were significant between U-Net and ResAttU-Net (p = 0.01) but not between Attention U-Net and ResAttU-Net (p = 0.10). On the other hand, for the HD, the differences were always not significant (p = 0.11 between U-Net and ResAttU-Net and p = 0.49 between Attention U-Net and ResAttU-Net). Even if the differences were not always significant, we decided to keep the model that provided the best results.

The second task was the evaluation of the ResAttU-Net in the 2.5D strategy.

Table 2. Evaluation of the 2.5D ResAttU-Net.

Organs	2D		2.5D
	Dice ± STD	HD ± STD (mm)	Dice ± STD	HD ± STD (mm)
Rectum	0.84 ± 0.12	5.20 ± 4.86	0.84 ± 0.10	5.29 ± 5.04
Bladder	0.89 ± 0.12	9.73 ± 7.78	0.92 ± 0.09	6.13 ± 5.46
Femoral head right	0.87 ± 0.08	6.83 ± 6.26	0.88 ± 0.08	7.72 ± 5.27
Femoral head left	0.88 ± 0.08	6.64 ± 5.40	0.88 ± 0.08	7.02 ± 6.15
Prostate	0.79 ± 0.18	9.03 ± 8.40	0.80 ± 0.15	7.08 ± 5.81
Overall Score	0.85 ± 0.11	7.49 ± 6.54	0.87 ± 0.10	6.65 ± 5.33

shows the results for each organ and the overall performance between the 2D and the 2.5D training strategies. It is worth noting that the bladder showed the best performance, yielding 0.92 DSC. The prostate underperformed the other organs with the lowest DSC value of 0.80. The presence of seminal vesicles is the cause of the low DSC on prostate segmentation. Moreover, the comparison between the different training strategies shows improvements with the 2.5D strategy in all the organs apart from the rectum and the femoral head (right). However, the results are close between the two strategies for these organs. Qualitative results are shown in Figure 7 and Figure 8. Considering the performance, the 2.5D strategy is 0.001 and 0.03 seconds (s) per slice slower than 2D using GPU and CPU, respectively (

Table 3. Inference time estimation with 2D and 2.5D training strategies. The times shown are seconds per slice.

Strategies	CPU (s)	GPU (s)
2D	1.45	0.113
2.5D	1.48	0.114

).

4. Discussion

Up to now, the most common screening tool for radiation therapy has been CT imaging [47]. Through the rapid advancements of the MR-Linac radiation therapy systems, MR imaging can screen anatomical structures with superior soft-tissue contrast compared to CT scans. In this context, an automatic multiorgan segmentation tool for the pelvic region dedicated to a 0.35 T MR-Linac system was proposed in our study. A modified version of the U-Net [33] combined with AGs and residual blocks using a 2.5D training strategy was used, utilizing multi-slice input and deep connections. The use of a 2.5D strategy allows for the obtainment of pseudo-3D information of the patient’s volume without the excessive computational complexity of 3D-based methods [48,49,50]. The 3D modeling allows the physicians to access a global view of the region of interest in question, with any kind of view or angle. It has been used in a lot of medical applications [51,52,53], and this display could complement the advanced imaging capabilities of the MR-Linac device. Minema et al. [32] reported shorter computational times in both strategies than in 3D without significant performance drawbacks. Clearly, the 3D training strategy is not always the optimal solution for robust results [54].

For the ResAttU-Net, we conducted experiments on two training strategies as the results of the proposed approach exhibited higher DSC than the U-Net and Attention U-Net in the 2D strategy. Following the optimal network from 2D, the 2.5D ResAttU-Net strategy surpassed the 2D strategy. More specifically, per organ comparison showed a significant increase in the accuracy for the segmentation of the bladder and prostate despite a slight decrease for the rectum. The results show the effectiveness of the 2.5D method, exploiting pseudo-3D information through adjacent segments with insignificant time savings. The segmentation of the bladder obtained the best results among all the contours, having mean DSC: 0.92 ± 0.09, HD: 6.13 ± 5.46 mm. Both femoral heads obtained stable results between both strategies without affecting the overall accuracy. On the contrary, the segmentation of the prostate showed lower performance with the 2.5D strategy. The observer variability in the prostate segmentation was subject-specific, with target shape and disease staging. This issue is particularly noticeable in cases where patients have intermediate-to-high risk. Similarly, the delineated prostate contains the base of seminal vesicles that can be delineated differently [55]. Additionally, the prostate base is in continuity with seminal vesicles and bladder, which may cause unexpected artifacts throughout the inference. There are not many studies related to multiorgan segmentation in the male pelvis dedicated to 0.35 T MRI. Specifically, the related works are MRI-based but with different sequence types or different magnetic fields. Our work demonstrated better or similar performance compared with the existing studies on T2-weighted images and on the multisequence approach (T1-weighted, T2-weighted, and enhanced Dixon T1-weighted images) [56]. To our knowledge, there is no related work based on the SSFP sequence for multiorgan segmentation in the pelvic region. A comparison shows that our work has comparable results regarding the related approaches (

Table 4. Comparison with recent works related to the pelvic region.

Organs	Our Work	Elguindi et al. [37]	Huang et al. [40]
	Dice ± STD	Dice ± STD	Dice ± STD
Rectum	0.84 ± 0.10	0.82 ± 0.05	0.78 ± 0.07
Bladder	0.92 ± 0.09	0.93 ± 0.04	0.90 ± 0.09
Femoral head right	0.88 ± 0.08	-	0.90 ± 0.02
Femoral head left	0.88 ± 0.08	-	0.89 ± 0.03
Prostate	0.80 ± 0.15	0.85 ± 0.07	-
Sequence Protocol	SSFP	T2-weighted	Multisequence

). However, the prostate remains lower compared to the results of Elguindy et al. The work of Huang et al. has no available prostate results since the work was based on cervical cancer. It is worth noting that it is not possible for the model to work in the same manner with MR imaging systems that have different technical specifications and magnetic field strength due to the differences in image resolution, contrast, and noise levels. Therefore, it is important to train deep learning models on MRIs of the same magnetic strength as those that will be used in clinical practice to ensure optimal performance.

The optimal strategy for the time-efficient segmentation of OARs is both the use of an automatic segmentation tool and the manual correction of the predicted contours from expert physicians if necessary. In addition to the barriers of accurate automatic segmentation, data collection and acquisition is also challenging as fully supervised learning and model performance are dependent on high-quality datasets. In the present study, the model was evaluated using a clinical case provided by an experienced radiologist with over five years of expertise. The automatic segmentation, along with manual modifications by an expert, required only 10 min, in contrast to the fully manual delineation process that typically takes 20 min or more.

While this study presents promising results, further work is needed to evaluate the approach in a clinical setting. Specifically, a dosimetry analysis will be performed to assess the clinical impact and correlation between automatic segmentation and manual segmentation. Furthermore, we will undertake an extensive examination of model enhancements, with a primary focus on the incorporation of the slices in the 2.5D strategy employing varied input modalities. Notably, introducing additional slices as input to the model holds promise for enhancing segmentation performance. Similarly, the substitution of randomly oriented 2D cross-sections for the existing axial slices as input into the model will be considered since they could exploit further spatial information. On the other hand, even if the 3D deep networks require many training parameters, the use of 3D volume contains important information [56]. An evaluation framework between the 2.5D and 3D strategies will be further investigated.

5. Conclusions

To conclude, the main objective of this study was the automatic segmentation of OARs and prostate in the pelvic region and the construction of a TPS-readable RT structure set. The presented work compared different training strategies with an in-house dataset of 103 cases. The 2.5D ResAttU-Net was the optimal model compared to plain U-Net and Attention U-Net. The mean DSC for each organ in the test dataset was 0.84, 0.92, 0.88, 0.88, and 0.80 for the rectum, bladder, femoral head (right), femoral head (left), and prostate, respectively. Problems arose mainly related to the class imbalance, where the organs were not shown in each slice. The proposed model shows great potential to be adapted to different anatomical regions or additional organs in the pelvic region for radiation therapy planning, which requires a considerable amount of training samples. For future work, dosimetry analysis will be performed. The final tool for the pelvic region is clinically used in our institution and is available for any ViewRay Inc. MRIdian user. The source code for the project is available on GitHub. (https://github.com/manoskout/automatic_segmentation) (accessed on 25 July 2023).