A Comparison Between Unimodal and Multimodal Segmentation Models for Deep Brain Structures from T1- and T2-Weighted MRI

Abstract

Accurate segmentation of deep brain structures is critical for preoperative planning in such neurosurgical procedures as Deep Brain Stimulation (DBS). Previous research has showcased successful pipelines for segmentation from T1-weighted (T1w) Magnetic Resonance Imaging (MRI) data. Nevertheless, the role of T2-weighted (T2w) MRI data has been underexploited so far. This study proposes and evaluates a fully automated deep learning pipeline based on nnU-Net for the segmentation of eight clinically relevant deep brain structures. A heterogeneous dataset has been prepared by gathering 325 paired T1w and T2w MRI scans from eight publicly available sources, which have been annotated by means of an atlas-based registration approach. Three 3D nnU-Net models—unimodal T1w, unimodal T2w, and multimodal (encompassing both T1w and T2w)—have been trained and compared by using 5-fold cross-validation and a separate test set. The outcomes prove that the multimodal model consistently outperforms the T2w unimodal model and achieves comparable performance with the T1w unimodal model. On our dataset, all proposed models significantly exceed the performance of the state-of-the-art DBSegment tool. These findings underscore the value of multimodal MRI in enhancing deep brain segmentation and offer a robust framework for accurate delineation of subcortical targets in both research and clinical settings.

1. Introduction

The human thalamus, which is located centrally in the brain, copes with the processing and transmission of motor and sensory signals between the cerebral cortex and various subcortical regions [1]. It is possible to identify different neuronal clusters, each of which can be subdivided into sub-nuclei, such as the subthalamic nucleus (STN), which is a biconvex glutamatergic structure located within the subcortex [2]. These sub-nuclei have been associated with such diseases as epilepsy [3] and, mainly, Parkinson’s disease (PD) [4,5,6,7,8].

The symptoms of such pathologies can be treated with Deep Brain Stimulation (DBS) [6], a neurosurgical procedure performed in the event of no response to drug treatment [2,9]. Since its success is derived from the electrode placement in subcortical regions [7], the STN may be accurately identified by means of microelectrode recording. However, it potentially increases both the surgical operation time [10] and the risk for the patient due to infection or bleeding [6]. Therefore, an alternative method for segmenting the STN is crucial for both optimizing the intervention by limiting the area of interest [1] and ensuring the patient’s safety by minimizing side effects [6].

Magnetic Resonance Imaging (MRI) offers the possibility to visualize the STN and its boundaries in advance [8]. Hence, MRI has been a valid tool to support preoperative planning for DBS by the automatic segmentation of the STN and its surrounding structures. Nonetheless, this annotation mode demands both extreme domain knowledge and a high amount of time for its accomplishment [4], which is higher and higher for each new segmentation [10]. Moreover, such identification of the STN is complicated by its variability in shape and orientation, together with its small size [6].

Standardized anatomical atlases have been created to help visualize and identify deep brain structures consistently and efficiently. Baniasadi et al. [3] have proposed an automatic approach for the automatic segmentation of 30 selected brain structures, combining information obtained from three anatomic atlases for the brain: CIT168 [11] (California Institute of Technology), DISTAL [12] (DBS Intrinsic Template Atlas), and THOMAS [13] (THalamus Optimized Multi Atlas Segmentation). Among them, DISTAL has proven particularly valuable for DBS planning, offering high-resolution, multimodal segmentations of key subcortical targets within the MNI (Montreal Neurological Institute-Hospital) space [14].

Integrated into the Lead-DBS [15,16,17] platform, DISTAL facilitates accurate localization of structures such as the red nucleus (RN), globus pallidus externus (GPe), globus pallidus internus (GPi), and STN, which are crucial for neurosurgical targeting in the treatment of movement disorders. By offering anatomically consistent boundaries, the atlas reduces inter-operator variability and facilitates both research and clinical applications, leading to more accurate electrode placement and better patient outcomes.

One challenge in this context is the limited visibility of deep brain nuclei in T1-weighted (T1w) images, especially compared to surrounding regions such as the thalamus, due to iron deposition [18]. T2-weighted (T2w) images offer higher contrast in these areas and can be integrated into the segmentation framework to improve performance [19]. However, T2w images suffer from poor contrast in the cortical regions [18], which limits their standalone effectiveness.

Therefore, the development of automated segmentation methods becomes essential, particularly for cases where structures like the STN are not clearly visible. These methods can greatly assist in neurosurgical planning and targeting [1], and they may be grounded on deep learning (DL) frameworks [20,21,22] such as nnU-Net [23,24], whose encoder–decoder architecture allows for automatically learning features from the image by exploiting ground-truth masks, which can either be manually annotated or generated through atlas-based approaches. Remarkably, Baniasadi and colleagues [3] exploited nnU-Net for the segmentation of 30 deep brain structures; however, their approach was based only on T1w images and, consequently, their proposed tool, DBSegment, cannot be applied to T2w images. Furthermore, by using only one image modality, they did not assess the eventual advantage of processing multimodal MRI data.

A group from Karolinska Institutet and Stockholm University proposed a DL approach, based on nnU-Net, for segmenting the STN, RN, and substantia nigra (SN) [19]. They leveraged multimodal data, involving Quantitative Susceptibility Mapping (QSM), T1w, FLAIR, and $R_2^{*}$ images. They trained models on 40 manually annotated subjects, finding that the best combination was QSM and FLAIR. The models were then applied to an independent 3-year longitudinal dataset of 175 healthy individuals to study age-related iron accumulation.

Solomon et al. [25] introduced GP-Net, a DL approach that leverages attention-gated networks to segment GPe and GPi from 7T T2w scans. The model was trained using manual annotations from 58 subjects and validated on 43 held-out cases. The overall cohort of 101 subjects included 24 healthy controls and 77 patients with movement disorders.

Beliveau and colleagues [26] evaluated five 3D CNN architectures (U-Net, V-Net, U-Net++, FC-Dense Net, and Dilated FC-Dense Net) for the automated segmentation of iron-rich deep brain nuclei (SN, STN, RN, and dentate nucleus) from susceptibility-weighted imaging (SWI). The model was trained on a dataset of 30 SWI images from healthy controls and externally validated on 17 SWI images from the Forrest Gump dataset [27].

While these studies show good performance on their designated validation datasets, they also have some limitations: they used small datasets, which may not be enough to ensure adequate generalization capabilities with respect to patient conditions or scanning protocols; these methods rely on 7T MRI scans or on QSM or SWI image modalities, which are not routinely used in clinical practice; and these works did not focus on all of the structures of DISTAL.

In this work, we propose a fully automated workflow for the segmentation of the deep brain structures included in DISTAL (i.e., GPe-L, GPe-R, GPi-L, GPi-R, RN-L, RN-R, STN-L, STN-R), through DL. After a careful stage of data collection and preparation, three different models, two unimodal (UM) and one multimodal (MM), were trained with the nnU-Net framework: UM T1w, UM T2w, and MM. These models were thoroughly compared with 5-fold cross-validation and on a designated test set, offering precise details about the evaluation of such models and whether a multimodal approach should be more suitable for segmentation in this case.

Our main contributions are fourfold and can be summarized as follows:

• A heterogeneous dataset comprising 325 T1w and 325 T2w MRI scans was built by integrating data from eight public sources, enabling robust training and evaluation.
• An end-to-end multimodal framework was developed for labeling MR images, preprocessing data, and training and evaluating deep learning models for the segmentation of deep brain structures in neurosurgical settings.
• A detailed comparison between unimodal and multimodal models was conducted, highlighting the benefits and limitations of each approach.
• The developed T1w-based models were benchmarked against the state-of-the-art DBSegment tool, demonstrating clear improvements across all metrics.

The remainder of this article is structured as follows: Section 2 describes the materials (i.e., the publicly available dataset from which we gathered images, together with data annotation and preparation for segmentation) and methods (i.e., the training, evaluation, and statistical comparison of the presented DL models). The results are shown and discussed in Section 3 and Section 4, respectively. Ultimately, conclusions about the conducted study and suggestions for future research are reported in Section 5.

2. Materials and Methods

The general workflow employed for this study is portrayed in Figure 1. It encompasses data acquisition and annotation through atlas-based segmentation, followed by a preprocessing phase involving defacing, resampling, and reorientation. The resulting images were then used to train and evaluate three nnU-Net models—UM T1w, UM T2w, and MM—supporting the automated segmentation of deep brain structures for neurosurgical planning.

2.1. Dataset Collection

The models used in the proposed framework have been trained, validated, and tested by means of data from 8 public datasets:

• HCP (Human Connectome Project) [28], which includes diffusion and anatomical neuroimaging data openly available to the scientific community for examination and exploration.
• OASIS3 (Open Access Series of Imaging Studies 3) [29], which is a retrospective compilation of data for 1378 participants with 2842 MRI sessions (encompassing T1w, T2w, and FLAIR, among others).
• ADNI (Alzheimer’s Disease Neuroimaging Initiative) [30], which is a longitudinal study started in 2004 that continuously expanded its data collection through multiple phases, contributing significantly to Alzheimer’s research. The ADNI dataset includes a variety of data types, such as clinical, biofluid, genetic, and imaging data, all of which are accessible to authorized researchers through the LONI Image and Data Archive (IDA). The latest phase of the study, ADNI4, is used in our final dataset and includes both T1w and T2w MRI scans.
• IXI (Information eXchange from Images) [31], which is a project that collected nearly 600 MR images from healthy subjects. The MR image acquisition protocols encompass T1w, T2w, and PD (Proton Density)-weighted images.
• UNC (University of North Carolina) [32], which includes paired T1-weighted and T2-weighted MRI scans acquired at both 3T and 7T from 10 healthy volunteers. The images were collected as part of a brain imaging study conducted by the University of North Carolina.
• THP (Traveling Human Phantom) [33]: This OpenNeuro dataset (accession number ds000206) was collected as part of a multi-site neuroimaging reliability study. It contains repeated multimodal MRI scans acquired from five healthy individuals across eight different imaging centers.
• NLA (Neural Correlates of Lidocaine Analgesic) [34]: This OpenNeuro dataset (accession number ds005088) includes T1w and T2w MRI scans acquired at 3T from 27 adults who participated in a single-arm, open-label study investigating the neural effects of lidocaine as an analgesic.
• neuroCOVID [35]: This OpenNeuro dataset (accession number ds005364) includes MRI data from a total of 100 participants who underwent T1w and T2w scans as part of an evaluation of the neurological effects of COVID-19.

The cross-validation set (CVAL) was built with 260 subjects from HCP, OASIS3, ADNI4, IXI, UNC, THP, NLA, and neuroCOVID, whereas the test set (TEST) was realized with 65 subjects from OASIS3, ADNI4, IXI, and neuroCOVID.

We collected a heterogeneous dataset, including images acquired from various scanners, using different acquisition protocols and intensity ranges. The data included both healthy individuals and patients with cognitive decline (from mild conditions to Alzheimer’s disease) and respiratory disorders (infectious diseases also including COVID-19) across a wide range of ages. To prevent overfitting to any single dataset, a similar number of subjects was selected from each dataset. All of the datasets, except for THP and UNC, contain only one scan for each modality from each subject. Detailed information about the training and test data is provided in

Table 1. Details of the study datasets. Acronyms: CVAL (cross-validation set), TEST (test set), HCP (Human Connectome Project), OASIS3 (Open Access Series of Imaging Studies 3), ADNI (Alzheimer’s Disease Neuroimaging Initiative), IXI (Information eXchange from Images), UNC (University of North Carolina), THP (Traveling Human Phantom), NLA (Neural Correlates of Lidocaine Analgesic), SM (Siemens), PL (Philips), FS (Field Strength), T1w (T1-weighted), MPRAGE (Magnetization Prepared RApid Gradient Echo), T2w (T2-weighted), SPACE (Sampling Perfection with Application Optimized Contrast Using Different Flip Angle Evolution), VISTA (Volume ISotropic Turbo Spin Echo Acquisition), TSE (Turbo Spin Echo), HT (healthy), MCI (Mild Cognitive Impairment), CI (Cognitive Impairment), AD (Alzheimer’s disease), RI (Respiratory Infection), M/F (male/female), N/A (not available).

Subset	Dataset	Scanner	FS	T1w	T2w	Disease	Age	M/F	MRI Scans
CVAL (n = 260)	HCP	SM	3T	MPRAGE	SPACE	HT	N/A	16/13	29
	OASIS3	SM	3T	MPRAGE	SPACE	HT, CI	52–84	11/19	30
	ADNI4	PL, SM	3T	MPRAGE	SPACE/VISTA	HT, MCI, AD	55–85	13/46	59
	IXI	N/A	N/A	N/A	N/A	HT	21–74	16/13	29
	UNC	SM	3T, 7T	MPRAGE, MP2RAGE	SPACE, TSE	HT	25–41	11/5	16
	THP	PL, SM	3T	MPRAGE	SPACE	HT	N/A	N/A	40
	NLA	SM	3T	MPRAGE	SPACE	HT	20–55	13/14	27
	neuroCOVID	SM	3T	MPRAGE	SPACE	COVID, RI	21–64	17/13	30
TEST (n = 65)	OASIS3	SM	3T	MPRAGE	SPACE	HT, CI	59–97	5/11	16
	ADNI4	SM	3T	MPRAGE	SPACE	HT, MCI, AD	56–85	7/9	16
	IXI	N/A	N/A	N/A	N/A	HT	23–63	9/8	17
	neuroCOVID	SM	3T	MPRAGE	SPACE	COVID, RI	19–66	9/7	16

.

2.2. Data Annotation

The segmentation maps of the structures were derived from the probabilistic DISTAL [12]. All labels were generated in MNI space (ICBM 2009b Nonlinear Asymmetric) [36]. DISTAL [12] was specifically developed for Lead-DBS and is precisely aligned to the MNI space. DISTAL is the result of the fusion of neuroimaging, neurobiology, and computational neuroscience expertise, by integrating multimodal data including MRI, histology, and structural connectivity to ensure high anatomical fidelity. Unlike other neuroanatomical atlases, DISTAL was explicitly built for DBS applications, addressing the issue of mismatch between the atlas and template space that can arise when millimeter-level accuracy is essential [15].

A label file containing the segmentations of the 8 selected brain structures, together with a brain mask designed to enhance network performance by guiding it to the brain region, was generated. Each structure was first resampled to the MNI space, using the ANTsPy library (v. 0.5.4), and then binarized by applying a threshold of 0.5. The brain mask was also obtained in MNI space with Lead-DBS. Then, it was combined with the segmentation labels of the other structures into a single file using 3D Slicer [37]. Here, a unique index was assigned to each structure for consistent identification. The label names and their associated indices are shown in

Table 2. Considered deep brain structures for this study. Full name, acronym, and label values are reported for each of the structures included in our segmentation models.

Full Name	Acronym	Label
Brain mask	BM	1
Globus pallidus externus (left)	GPe-L	2
Globus pallidus externus (right)	GPe-R	3
Globus pallidus internus (left)	GPi-L	4
Globus pallidus internus (right)	GPi-R	5
Red nucleus (left)	RN-L	6
Red nucleus (right)	RN-R	7
Subthalamic nucleus (left)	STN-L	8
Subthalamic nucleus (right)	STN-R	9

.

The T1w and T2w images of each subject were annotated using the generated label file, following an atlas-based method similar to those described in previous pipelines [3,38]. The GPe, GPi, RN, and STN were directly segmented in the native space of each subject through the Lead-DBS workflow [16], excluding electrode localization and reconstruction steps. Initially, a bias-field correction was applied to the T1w and T2w images using the N4 algorithm [39]. After correction, the T2w images were co-registered to the T1w images using ANTs [40] through a two-step linear registration process (rigid followed by affine) to compute the corresponding transformation matrix. The images were then normalized to the MNI 2009b template space using the Symmetric Normalization (SyN) algorithm from ANTs [40]; this step generated a nonlinear deformation field representing the transformation from the subject’s native space to the MNI space, saved as an NIfTI transformation file. Using the computed transformations, the segmentation labels were warped from the MNI space to each subject’s native image space, resulting in subject-specific segmentations of the targeted brain structures. All registrations were visually inspected to ensure accuracy and consistency. Only images that passed this quality control step were included in the dataset. In cases of a registration failure producing visibly incorrect segmentations, the corresponding subjects were excluded to prevent potential misguidance during network training.

2.3. Data Preparation

Facial information haws removed using PyDeface (v. 2.0.2) [41] to ensure consistency and anonymization across the dataset; defacing was applied only to MRI scans where facial structures had not yet been removed by the original creators of the dataset.

In this study, all MR images and corresponding ground-truth masks were isotropically resampled to a voxel spacing of 1 mm³, resized to dimensions of 256 × 256 × 256, and reoriented to the left–posterior–inferior (LPI) coordinate system. After preprocessing, the dataset was organized into the folder structure required by the nnU-Net framework [23]. This format was inspired by the data organization used in the Medical Segmentation Decathlon (MSD). Each MSD-like dataset has three components: raw images, the corresponding ground-truth segmentation maps, and a JSON file containing appropriate metadata.

Three separate datasets, one for each of the implemented models, were prepared: UM T1w, UM T2w, and MM (incorporating both T1w and T2w). For the unimodal models, the input consisted of the original T1w or T2w images, respectively, together with their corresponding segmentation maps. For the multimodal model, the dataset included co-registered T1w and T2w images, accompanied by a single set of segmentation maps consistent across both modalities, obtained during the labeling process in the co-registered space.

All default nnU-Net preprocessing steps were applied prior to training, including automatic cropping around the foreground to reduce memory usage, intensity normalization (z-score with mean = 0 and standard deviation = 1) for MRI data, and resampling the images to voxel spacing and dimensions that are the median of the training data. Data augmentation was employed during training. The following transformations were included in the data augmentation: rotations, scaling, Gaussian noise, Gaussian blur, brightness, contrast, simulation of low resolution, gamma correction, and mirroring.

2.4. Models’ Architecture and Training

Three 3D full-resolution models were trained with the nnU-Net framework, which implements a U-Net architecture with an encoder–decoder structure and skip connections. The automatic configuration of nnU-Net is driven by a combination of fixed parameters, rule-based parameters, and empirical parameters derived from the specific dataset. To assess the generalization capability of each model, nnU-Net automatically performed a 5-fold cross-validation on a dataset of 260 MR images (CVAL), with each fold comprising 208 training and 52 validation scans.

All models were trained using a batch size of 2 and a patch size of 128 × 128 × 112. The network architecture was composed of 6 stages, with the depth of the feature maps increasing from 32 to 320. Each stage included two 3D convolutional layers with kernel sizes of 3 × 3 × 3, followed by instance normalization and the Leaky ReLU activation. Downsampling was performed with 3D convolutional layers with strides of [2, 2, 2] after each level, except for the last stage, which used [2, 2, 1]. Upsampling was carried out with 3D transposed convolutional layers that compensated for downsample levels introduced in the symmetrical part of the encoder. A detailed representation of the employed architecture is portrayed in Figure 2.

The networks were trained for a fixed number of 1000 epochs, with each epoch comprising 250 training iterations. Stochastic gradient descent with Nesterov momentum (μ = 0.99) was used as an optimizer. The multimodal variant shares the same architectural design as the unimodal models, except for the input layer, which has been adapted to accept two channels corresponding to the T1w and T2w modalities.

Overall, five different networks (five folds) were trained for each of the three configurations (UM T1w, UM T2w, and MM). The performance of the network for each fold was evaluated on the 52 validation scans. Finally, an ensemble with all of the networks from the folds was constructed. The final predictions obtained by this ensemble were evaluated on 65 new MR images (TEST), which were completely independent from the CVAL.

The models were trained and validated using Python (v. 3.10.16), including the libraries nnunetv2 (v. 2.6.0), torch (v. 2.6.0+cu118), and torchvision (v. 0.21.0+cu118).

2.5. Evaluation Metrics

The network performances were evaluated using the Dice coefficient (Dice), Relative Volume Difference (RVD), Average Symmetric Surface Distance (ASSD), and Intersection over Union (IoU). These metrics were computed by comparing the segmentations generated by the network with those obtained by the registration-based method (gold standard). All of the metrics were calculated separately for each structure.

Dice and IoU are defined by Equation (1) and Equation (2), respectively. These quality measures are based on the volumetric overlap between predicted and ground-truth masks. RVD, based on the Relative Volume Difference between predicted and ground-truth masks, is defined by Equation (3). In all of the definitions, $P$ and $G$ indicate the volume predicted by the network and the ground-truth volume, respectively.

ASSD is a metric based on the external surface distances. To define these distances, we define a metric space $(X, d)$, where $X$ is a 3D Euclidean space and $d$ is the Euclidean distance over the space. We can define $S\left(P\right), S\left(G\right)\subset X$ as the external surfaces of the predicted and ground-truth volumes, respectively [42]. Then, ASSD can be calculated as defined in Equation (4).

$$\mathrm{Dice} \left(P,G\right)={\displaystyle \frac{2·\left|P\right|\cap \left|G\right|}{\left|P\right|+\left|G\right|}}$$

(1)

$$\mathrm{IoU} \left(P,G\right)={\displaystyle \frac{\left|P\cap G\right|}{\left|P\cup G\right|}}$$

(2)

$$\mathrm{RVD} \left(P,G\right)={\displaystyle \frac{\left|P\right|-\left|G\right|}{\left|G\right|}}$$

(3)

$$\mathrm{ASSD}\left(S\left(P\right), S\left(G\right)\right)={\displaystyle \frac{1}{\left|S\left(P\right)\right|+\left|S\left(G\right)\right|}}\left( \sum _{s_P\in S\left(P\right)}d\left(s_P, S\left(G\right)\right)+\sum _{s_G\in S\left(G\right)}d\left(s_G, S\left(P\right)\right)\right)$$

(4)

2.6. Statistical Evaluation

The models were statistically compared by means of the Wilcoxon signed-rank test with the two-sided option for each metric and on each segmented structure, allowing for the detection of statistically significant differences in both directions. Multiple hypothesis testing was considered for the analyzed deep brain structures, by employing corrected p-values with the Benjamini–Hochberg [43] method for false discovery rate.

All of the performance evaluations and their statistical analyses were conducted using Python (v. 3.12.3), including the libraries scipy (v. 1.14.1), matplotlib (v. 3.10.0), MedPy (v. 0.5.2), numpy (v. 1.26.4), pandas (v. 2.2.3), and seaborn (v. 0.13.2).

3. Results

3.1. Cross-Validation Results

The boxplots of the T1w CVAL comparing all of the considered structures (GPe-L, GPe-R, GPi-L, GPi-R, RN-L, RN-R, STN-L, and STN-R), using Dice, IoU, RVD, and ASSD, are pictorially depicted in Figure 3. These results are complemented by the means and standard deviations reported in

Table 3. T1w CVAL metrics: Dice, IoU, RVD, and ASSD are reported in terms of mean ± standard deviation for each considered anatomical structure.

Label	Model	Dice [%]	IoU [%]	RVD [%]	ASSD [mm]
BM	MM	98.18 ± 0.49	96.44 ± 0.92	0.10 ± 1.23	0.68 ± 0.17
	UM	98.14 ± 0.58	96.36 ± 1.08	0.46 ± 1.42	0.72 ± 0.20
	DBSegment	96.31 ± 0.70	92.89 ± 1.28	1.21 ± 2.32	1.52 ± 0.77
GPe-L	MM	86.48 ± 7.28	76.71 ± 8.37	1.77 ± 5.93	0.43 ± 1.44
	UM	86.49 ± 6.07	76.56 ± 7.10	1.47 ± 8.72	0.38 ± 0.38
	DBSegment	81.30 ± 4.67	68.74 ± 6.37	−9.84 ± 8.65	0.48 ± 0.17
GPe-R	MM	85.91 ± 6.96	75.80 ± 8.57	2.29 ± 5.08	0.40 ± 0.73
	UM	86.72 ± 4.55	76.81 ± 6.39	2.14 ± 6.51	0.35 ± 0.11
	DBSegment	81.66 ± 4.76	69.27 ± 6.58	−12.17 ± 7.93	0.49 ± 0.47
GPi-L	MM	86.74 ± 7.28	77.12 ± 8.60	1.30 ± 7.31	0.43 ± 1.34
	UM	87.44 ± 4.62	77.95 ± 6.43	1.08 ± 8.03	0.36 ± 0.16
	DBSegment	81.13 ± 5.66	68.61 ± 7.56	−10.43 ± 7.52	0.51 ± 0.15
GPi-R	MM	86.39 ± 7.09	76.57 ± 8.82	1.60 ± 6.43	0.40 ± 0.64
	UM	87.91 ± 3.75	78.61 ± 5.76	1.54 ± 7.28	0.35 ± 0.10
	DBSegment	81.23 ± 5.92	68.79 ± 8.01	−12.05 ± 7.40	0.52 ± 0.21
RN-L	MM	86.23 ± 7.24	76.37 ± 9.31	1.20 ± 9.83	0.36 ± 0.19
	UM	87.97 ± 8.48	79.18 ± 8.97	1.70 ± 8.11	0.35 ± 0.59
	DBSegment	83.10 ± 4.86	71.38 ± 6.90	−14.88 ± 7.52	0.43 ± 0.12
RN-R	MM	86.11 ± 6.29	76.08 ± 8.73	−0.01 ± 7.51	0.35 ± 0.15
	UM	87.92 ± 8.56	79.10 ± 9.03	1.45 ± 7.82	0.35 ± 0.64
	DBSegment	82.34 ± 5.31	70.31 ± 7.41	−15.66 ± 7.31	0.44 ± 0.13
STN-L	MM	80.93 ± 8.96	68.74 ± 10.48	0.27 ± 9.14	0.40 ± 0.89
	UM	83.15 ± 7.33	71.72 ± 9.12	2.13 ± 9.97	0.36 ± 0.53
	DBSegment	75.14 ± 7.61	60.76 ± 9.58	−12.68 ± 8.94	0.45 ± 0.14
STN-R	MM	81.27 ± 8.43	69.18 ± 10.32	0.37 ± 8.59	0.37 ± 0.38
	UM	83.29 ± 7.42	71.91 ± 8.82	2.14 ± 9.65	0.39 ± 1.07
	DBSegment	75.75 ± 7.99	61.60 ± 9.97	−13.64 ± 8.51	0.46 ± 0.15

. Overall, the MM and T1w UM exhibit similar performance, with all structures showing average Dice scores between 80.93% and 87.97%, IoU between 68.74% and 79.18%, RVD values not exceeding +2.29% (overestimation) or −0.01% (underestimation), and ASSD values ranging from 0.35 mm to 0.43 mm. The DBSegment model, on our CVAL, demonstrates a consistent trend toward volume underestimation and achieves the lowest scores across all structures.

The boxplots of the T2w CVAL are shown in Figure 4, whereas the means and standard deviations are reported in

Table 4. T2w CVAL metrics: Dice, IoU, RVD, and ASSD are reported in terms of mean ± standard deviation for each considered anatomical structure.

Label	Model	Dice [%]	IoU [%]	RVD [%]	ASSD [mm]
BM	MM	98.18 ± 0.49	96.44 ± 0.91	0.09 ± 1.22	0.67 ± 0.17
	UM	98.07 ± 0.56	96.22 ± 1.04	0.21 ± 1.47	0.74 ± 0.19
GPe-L	MM	86.39 ± 7.27	76.56 ± 8.40	1.65 ± 6.08	0.42 ± 1.44
	UM	84.36 ± 6.67	73.36 ± 7.40	1.22 ± 9.96	0.45 ± 0.61
GPe-R	MM	85.90 ± 6.90	75.79 ± 8.55	2.41 ± 5.24	0.39 ± 0.71
	UM	84.92 ± 3.36	73.94 ± 4.97	2.90 ± 6.86	0.40 ± 0.09
GPi-L	MM	86.63 ± 7.34	76.95 ± 8.71	1.45 ± 7.48	0.43 ± 1.35
	UM	84.76 ± 6.55	73.95 ± 7.34	1.11 ± 10.65	0.41 ± 0.11
GPi-R	MM	86.48 ± 6.90	76.71 ± 8.75	1.56 ± 6.69	0.39 ± 0.63
	UM	85.34 ± 3.89	74.62 ± 5.70	2.25 ± 9.56	0.46 ± 0.60
RN-L	MM	86.20 ± 7.61	76.37 ± 9.60	1.09 ± 9.75	0.35 ± 0.21
	UM	85.54 ± 4.34	74.97 ± 6.26	1.40 ± 9.97	0.38 ± 0.10
RN-R	MM	86.09 ± 6.17	76.04 ± 8.64	0.13 ± 8.02	0.34 ± 0.15
	UM	84.73 ± 3.98	73.70 ± 5.89	1.81 ± 9.59	0.40 ± 0.10
STN-L	MM	80.85 ± 8.91	68.62 ± 10.48	0.14 ± 9.79	0.38 ± 0.86
	UM	78.92 ± 6.63	65.63 ± 8.32	1.88 ± 12.59	0.40 ± 0.16
STN-R	MM	81.13 ± 8.63	69.00 ± 10.60	0.67 ± 8.73	0.36 ± 0.37
	UM	79.45 ± 5.74	66.29 ± 7.90	2.54 ± 11.18	0.40 ± 0.11

. Generally, the MM consistently outperforms the T2w UM across all structures, achieving average Dice scores ranging from 80.85% to 86.63%, IoU from 68.62% to 76.95%, RVD values with a maximum overestimation of +2.41% and minimum overestimation of +0.13%, and ASSD ranging between 0.34 mm and 0.43 mm.

Regarding T1w images from CVAL, the MM model consistently achieves the highest performance for all metrics (Dice, IoU, RVD, and ASSD) on the GPe and GPi, with statistically significant improvements (p < 0.001) over the DBSegment model in all structures. The UM yields the highest performance for all metrics (Dice, IoU, RVD, and ASSD) on the RN and STN, significantly outperforming DBSegment for each evaluated metric and structure (p < 0.001).

When considering T2w images from CVAL, the MM significantly outperforms the UM (p < 0.001) in terms of Dice, IoU, and ASSD across all anatomical structures, confirming the benefit of combining T1w and T2w information. Interestingly, no statistically significant differences were found in RVD, suggesting that both models maintain comparable volumetric estimations.

3.2. Test Results

The boxplots of the T1w TEST cohort comparing all of the considered structures (GPe-L, GPe-R, GPi-L, GPi-R, RN-L, RN-R, STN-L, and STN-R), using Dice, IoU, RVD, and SSD, are pictorially depicted in Figure 5. These results are complemented by the means and standard deviations reported in

Table 5. T1w TEST metrics: Dice, IoU, RVD, and ASSD are reported in terms of mean ± standard deviation for each considered anatomical structure.

Label	Model	Dice [%]	IoU [%]	RVD [%]	ASSD [mm]
BM	MM	98.24 ± 0.41	96.54 ± 0.79	−0.25 ± 1.20	0.67 ± 0.15
	UM	98.19 ± 0.54	96.45 ± 1.02	0.34 ± 1.56	0.70 ± 0.20
	DBSegment	96.31 ± 0.59	92.88 ± 1.08	−0.00 ± 2.02	1.42 ± 0.21
GPe-L	MM	86.76 ± 4.01	76.83 ± 5.87	1.81 ± 6.13	0.34 ± 0.10
	UM	86.93 ± 2.96	77.00 ± 4.61	1.48 ± 6.17	0.35 ± 0.07
	DBSegment	82.44 ± 4.86	70.40 ± 6.62	−10.84 ± 5.69	0.44 ± 0.12
GPe-R	MM	86.88 ± 3.95	77.01 ± 5.91	1.83 ± 5.19	0.34 ± 0.10
	UM	86.48 ± 3.54	76.34 ± 5.39	2.42 ± 5.46	0.36 ± 0.09
	DBSegment	82.04 ± 5.43	69.88 ± 7.44	−11.44 ± 7.47	0.52 ± 0.66
GPi-L	MM	87.39 ± 3.80	77.80 ± 5.82	1.70 ± 6.15	0.34 ± 0.10
	UM	87.53 ± 3.54	78.00 ± 5.55	1.51 ± 5.12	0.35 ± 0.09
	DBSegment	82.52 ± 4.31	70.46 ± 6.16	−10.10 ± 5.31	0.47 ± 0.11
GPi-R	MM	88.01 ± 3.43	78.75 ± 5.40	0.23 ± 5.36	0.33 ± 0.09
	UM	87.68 ± 3.12	78.20 ± 4.94	1.84 ± 5.44	0.36 ± 0.09
	DBSegment	83.04 ± 4.89	71.29 ± 7.05	−10.98 ± 5.75	0.48 ± 0.20
RN-L	MM	88.17 ± 3.21	78.98 ± 5.05	0.06 ± 5.55	0.31 ± 0.08
	UM	88.91 ± 3.12	80.18 ± 5.02	0.25 ± 6.07	0.29 ± 0.08
	DBSegment	84.69 ± 3.57	73.61 ± 5.16	−14.61 ± 6.14	0.39 ± 0.09
RN-R	MM	88.29 ± 2.59	79.12 ± 4.12	−0.46 ± 5.13	0.30 ± 0.07
	UM	88.08 ± 3.40	78.87 ± 5.44	1.13 ± 5.89	0.31 ± 0.08
	DBSegment	83.42 ± 3.25	71.69 ± 4.77	−14.63 ± 5.29	0.42 ± 0.08
STN-L	MM	82.79 ± 5.55	70.99 ± 7.63	2.09 ± 8.18	0.31 ± 0.09
	UM	83.37 ± 4.53	71.73 ± 6.62	2.21 ± 7.24	0.31 ± 0.08
	DBSegment	75.44 ± 6.20	60.95 ± 7.86	−10.03 ± 6.81	0.44 ± 0.11
STN-R	MM	83.86 ± 4.95	72.50 ± 7.19	0.60 ± 6.94	0.30 ± 0.09
	UM	84.01 ± 4.58	72.69 ± 6.83	3.45 ± 6.64	0.31 ± 0.09
	DBSegment	76.45 ± 6.72	62.34 ± 8.68	−10.13 ± 6.20	0.44 ± 0.12

. In summary, the MM and T1w UM exhibit similar performance, with all structures showing average Dice scores between 82.79% and 88.91%, IoU between 70.99% and 80.18%, RVD values not exceeding +3.45% (overestimation) or −0.46% (underestimation), and ASSD values ranging from 0.29 mm to 0.36 mm. The DBSegment model, on our TEST cohort, demonstrates a consistent trend toward volume underestimation and achieves the lowest scores across all structures.

The boxplots of the T2w TEST cohort are shown in Figure 6, whereas the means and standard deviations are reported in

Table 6. T2w TEST metrics: Dice, IoU, RVD, and ASSD are reported in terms of mean ± standard deviation for each considered anatomical structure.

Label	Model	Dice [%]	IoU [%]	RVD [%]	ASSD [mm]
BM	MM	98.24 ± 0.41	96.54 ± 0.78	−0.25 ± 1.20	0.66 ± 0.15
	UM	98.10 ± 0.51	96.27 ± 0.97	−0.10 ± 1.48	0.73 ± 0.19
GPe-L	MM	86.74 ± 4.09	76.79 ± 5.97	2.01 ± 6.19	0.34 ± 0.10
	UM	82.49 ± 8.16	70.83 ± 9.32	0.70 ± 12.61	0.52 ± 0.64
GPe-R	MM	86.93 ± 3.87	77.08 ± 5.78	1.96 ± 5.27	0.34 ± 0.10
	UM	84.38 ± 3.69	73.16 ± 5.49	1.86 ± 5.08	0.41 ± 0.09
GPi-L	MM	87.30 ± 3.89	77.66 ± 5.94	1.83 ± 5.97	0.34 ± 0.10
	UM	83.24 ± 7.28	71.83 ± 8.96	−0.21 ± 11.18	0.48 ± 0.27
GPi-R	MM	88.06 ± 3.51	78.83 ± 5.51	0.19 ± 5.61	0.33 ± 0.09
	UM	84.74 ± 4.61	73.78 ± 6.60	0.18 ± 7.80	0.52 ± 0.75
RN-L	MM	88.16 ± 3.20	78.97 ± 5.04	0.37 ± 6.06	0.30 ± 0.08
	UM	84.81 ± 4.85	73.92 ± 7.13	−0.18 ± 7.14	0.40 ± 0.12
RN-R	MM	88.47 ± 2.71	79.43 ± 4.33	−0.95 ± 5.20	0.29 ± 0.07
	UM	85.06 ± 4.19	74.23 ± 6.38	−1.43 ± 7.62	0.38 ± 0.10
STN-L	MM	82.72 ± 5.69	70.89 ± 7.76	1.26 ± 8.32	0.30 ± 0.09
	UM	77.75 ± 5.82	63.97 ± 7.75	1.80 ± 10.76	0.41 ± 0.10
STN-R	MM	83.59 ± 5.12	72.13 ± 7.45	0.72 ± 7.31	0.30 ± 0.09
	UM	78.85 ± 5.89	65.48 ± 8.12	2.29 ± 8.88	0.40 ± 0.11

. In general, the MM consistently outperforms the T2w UM across all structures, achieving average Dice scores ranging from 82.72% to 88.47%, IoU from 70.89% to 79.43%, RVD values with a maximum overestimation of +2.01% and maximum underestimation of −0.95%, and ASSD ranging between 0.29 mm and 0.34 mm.

Concerning T1w images from the TEST cohort, the MM model consistently achieves the highest performance across all metrics (Dice, IoU, RVD, and ASSD), with statistically significant improvements (p < 0.001) over the DBSegment model in all structures. The T1w UM yields intermediate results, significantly outperforming DBSegment for each evaluated metric and structure (p < 0.001), and achieving comparable scores to the MM ones in several cases.

Regarding T2w images, the MM significantly outperforms the T2w UM (p < 0.001) in Dice, IoU, and ASSD across all anatomical structures, confirming the benefit of combining T1w and T2w information. Interestingly, no statistically significant differences were found in RVD, suggesting that both models maintain comparable volumetric estimations.

Details of dataset-specific TEST metrics are reported in Table S1 and Table S2 for T1w and T2w images, respectively.

3.3. Qualitative Results

A qualitative comparison of the results of the MM, UM, and DBSegment models on the T1w TEST cohort is reported in Figure 7, which shows masks superimposed on regions of interest (ROIs) from four subjects, displayed in axial and coronal views. Every case belongs to a different collection included in the TEST cohort. Similarly, Figure 8 depicts the same ROIs extracted from T2w MRI images with the same views (axial and coronal). In this case, the comparison with DBSegment is missing, since it only supports T1w segmentation.

The segmentations predicted by all models are qualitatively accurate and visually analogous to the reference segmentations obtained using the atlas-based approach, since all structures are correctly localized. In some regions, the predicted segmentations even exhibit smoother and more anatomically plausible boundaries compared to our ground truth, possibly indicating a better generalization capability of DL models than atlas-based approaches. Moreover, the DBSegment model tends to underestimate volume predictions, as confirmed by quantitative results and the qualitative ones that are shown in Figure 7b (Ax) and Figure 7c (Ax).

Examples of the worst-case segmentations according to ASSD for T1w images are portrayed in Figure S1 and Figure S2 for the UM and MM models, respectively. Examples of the worst-case segmentations according to ASSD for T2w images are shown in Figure S3 and Figure S4 for the UM and MM models, respectively.

4. Discussion

Automated segmentation of subcortical structures is necessary for accurate diagnosis and therapy planning. Volume discrimination of adjacent structures is also a prerequisite for surgical interventions such as DBS. The most used approaches for structure segmentation are based either on local edge information, intensity, or try to combine both of them by incorporating prior shape information [44]. Structural MRI provides complementary information in the classical sequences T1w and T2w, where the first modality outlines brain structures and provides crisper images, whereas T2w highlights pathological regions, i.e., with inflammations or lesions, and renders fluids in a brighter manner.

In this study, a comprehensive comparison of UM and MM models for segmentation of deep brain structures from T1w and T2w MRI was performed. The MM approach serves as a benchmark for understanding to what extent merging these modalities affects the segmentation performance relative to the specific structural composition. Considering the Dice coefficient, as for the CVAL from T1w, the UM and MM scores are not statistically significant for 3/8 structures (GPe-L, GPe-R, and GPi-L), with the former significantly outperforming the latter for the remaining 5/8 structures (GPi-R, RN-L, RN-R, STN-L, and STN-R). These are deep, small nuclei located in the midbrain and surrounded by white matter with overlapping intensity distributions. Remarkably, an effective segmentation of the STNs through classical T1w images is also crucial in cases of neurodegenerative disorders with iron accumulation [45], where iron-sensitive sequences like susceptibility-weighted imaging and T2*w are the most employed alternative sequences. As concerns the TEST cohort from T1w, the UM and MM scores do not significantly differ for any structures but the GPe-R, in which case MM significantly outperforms UM. On the other hand, MM significantly outperformed UM from T2w on CVAL and TEST for all structures.

With regards to T1w segmentation, the proposed models (both UM and MM) significantly outperformed a reference method, DBSegment, for both the CVAL and TEST cohorts. However, no fair comparison can be carried out between DBSegment and the proposed models on T2w images, since those authors did not train on this modality.

The considerable improvement of MM over UM in T2w experiments suggests that the inclusion of T1w information effectively compensates for the lower anatomical contrast in subcortical regions, as is typically associated with T2w images. Furthermore, both MM and UM revealed comparably high performance on T1w-based segmentation, thus indicating that T1w MR images are the most relevant for segmentation of deep brain structures from a deep learning perspective.

From a clinical viewpoint, these findings are especially relevant for such neurosurgical procedures as DBS, where accurate localization of structures like the STN and GPi is critical for accurate preoperative planning, ultimately leading to improved patient outcomes.

Limitations

Despite the promising outcomes of the proposed approach, two main limitations must be acknowledged.

First, most of the MRIs included in our dataset came from either healthy subjects or patients with a condition different from Parkinson’s disease. Therefore, despite the robustness of the developed framework, as shown from our cross-validation and test experiments, it still lacks a more specific performance evaluation on the patients who are more likely to need surgical interventions for DBS. Nevertheless, the segmentation of deep brain structures remains clinically relevant not only for PD but also for other neurological conditions. In our study, patients affected by cognitive decline disorders, such as Alzheimer’s disease (AD)—for which DBS has shown potential in alleviating memory loss, a common symptom of AD [46]—have been considered. Additionally, the inclusion of patients with a history of COVID-19 offers a valuable starting point for exploring the long-term cognitive consequences of the disease. Recent evidence indicates that even individuals without a prior diagnosis of AD may experience neurodegenerative effects due to prolonged COVID-19-related hypoxia [47].

Then, our dataset was created by using registration and templates to annotate a large amount of MRI data, consisting of 325 T1w and 325 T2w images. While employing DISTAL, specifically built for DBS interventions, allowed us to obtain quality labels for this study, a validation involving neuroimaging specialists was not performed. Neuroradiologists and neurosurgeons could help in refining the dataset, providing more reliable estimates of the developed segmentation models’ performance.

5. Conclusions

This study presents a completely automated deep learning pipeline, encompassing UM and MM models, for the segmentation of eight deep brain structures that are relevant to neurosurgical procedures, especially DBS, using MRI data.

The obtained results show that the proposed UM and MM models perform better than the state-of-the-art DBSegment method on T1w MR images, on both cross-validation and an independent test set. Remarkably, no publicly available methods prior to this work exist for automatic segmentation on T2w MR images.

Future research could focus on extending the proposed pipeline to incorporate other state-of-the-art 3D segmentation models and evaluate their performance and capabilities in real-world neurosurgical environments. Collaboration with neurosurgeons would allow for more objective benchmarks for the developed models, instead of relying on ground truth generated from atlas-based approaches.

Domain adaptation, encompassing image-to-image translation [48] methods to convert T1w images to T2w images and vice versa, or even to other modalities, could be an interesting direction for future studies—for instance, exploiting generative adversarial networks [8] or diffusion models [49].

Finally, future work could also explore the use of a larger and more diverse training dataset. Particularly, datasets should be selected to include patients with PD, so that models can be more specifically tailored to DBS applications. Moreover, incorporating not only additional 3T scans but also ultra-high-field 7T MRI acquisitions, the increased spatial resolution and contrast provided by 7T imaging could enhance segmentation accuracy, particularly for small subcortical structures.