Validation of Replicable Pipeline 3D Surface Reconstruction for Patient-Specific Abdominal Aortic Lumen Diagnostics

Abstract

Background: Accurate prognoses are challenging in high-risk vascular conditions, such as abdominal aortic aneurysms, and limited diagnostic standards, decision-making criteria, and data semantics often hinder clinical reliability and impede diagnostics’ digital transition. This study aims to evaluate the performance, robustness, and usability of an automatic, replicable pipeline for aortic lumen surface reconstruction for pathological vessels. The goal is to provide a solid tool for geometric reconstruction to a more complex enhanced diagnostic framework. Methods: A U-Net convolutional neural network is trained using preoperative CTA scans, with 101 for model training and 14 for model testing, covering a wide anatomical and aortoiliac pathology spectrum. Validation included segmentation metric, robustness, reliability, and usability assessments. Performances are investigated by means of the test set’s prediction metrics for several instances of the model’s input. Clinical reliability is evaluated based on manual measurements performed by a vascular surgeon on the obtained 3D aortic lumen surfaces. Results: The test set is selected to cover a wide portion of aortoiliac pathologies. The algorithm demonstrated robustness with an average F1-score of 0.850 ± 0.120 and an intersection over union score of 0.760 ± 0.150 in the test set. Clinical reliability is assessed using the mean absolute errors for diameter and length measurements, respectively, of 1.73 mm and 2.27 mm. The 3D surface reconstruction demonstrated reliability, low processing times, and clinically valid reconstructions. Conclusions: The proposed algorithm can correctly reconstruct pathological vessels. Secondary aortoiliac pathologies are detected properly for challenging anatomies. To conclude, the proposed 3D reconstruction application to a digital, patient-specific diagnostic tool is, therefore, possible. Automatic replicable pipelines ensured the usability of the model’s outputs.

1. Introduction

An AAA is an enlargement of the abdominal aorta, such that the diameter is more than 3 cm or 50% larger than its healthy, baseline value. Accurate monitoring, diagnostics, and timely intervention are crucial for preventing the high morbidity and mortality rates associated with such pathology. EVAR treatment represents the gold standard when the aortic anatomy is feasible, and its planning is, in fact, fundamental for technical success and endograft durability.

The cardiovascular community has proposed guidelines to streamline the surgery decision-making process and planning of abdominal aortic aneurysms. One such approach, proposed by Ouriel et al. [1], is based on a threshold-based method using maximum diameter measurements. The suggested threshold values are 55 mm for men and 50 mm for women. However, this approach has proven to be unreliable in predicting abdominal aortic aneurysm ruptures, as indicated by subsequent studies. In fact, Finol et al. [2] showed how the rupture risk does not linearly depend on the maximum diameter. To improve decision-making criteria, El Chaikoff et al. [3] introduced more complex methods for rupture risk assessment, moving beyond purely geometrical, threshold-based approaches. Additionally, Parkinson et al. [4] demonstrated that the rupture rate for abdominal aortic aneurysms ranging from 40 mm to 49 mm, though low, was still significant. Similarly, Darling et al. [5] reported that 23% of abdominal aortic aneurysms ruptured at sizes under 50 mm. Over the past decade, DL techniques have been vastly employed in the semantic segmentation of the aortic lumen in EVAR planning. Lately, Fantazzini et al. [6] have contributed to the segmentation task by training single- and multi-view U-Net models for aortic lumen semantic segmentation on 70 CTAs. However, the need for more widely distributed datasets is crucial, as the network is trained on a limited spectrum of abdominal aortic aneurysm anatomical features. An extension of previous work by the same research group proposed a novel model for the automatic segmentation and geometrical analysis of abdominal aortic aneurysms, along with the identification of intraluminal thrombus presence, as presented in Brutti et al. [7]. The model’s clinical validation involved a comparison between network measurements and those made by expert users. In another study by Adam et al. [8], fully automated routine presurgical measurements of the axial aortic lumen and outer-to-outer aortic wall diameter for endovascular aortic repair were proposed. The authors trained a V-Net [9] in large datasets consisting of 489 and 62 CTA sets for training and testing, respectively. The results showed mean Dice–Sørensen coefficients of 0.95 for diseased abdominal aortas and 0.84 for healthy aortas, indicating reliable performance. However, both studies acknowledged the necessity of including vastly different abdominal aortic aneurysm topologies in the training and validation sets to improve the model’s consistency and stability. Lastly, Lopez-Linares et al. [10] similarly proposed a work in which a fully automatic segmentation pipeline is adopted for a thrombus and an aortic lumen for postoperative CTAs. A convolutional neural network was trained on postoperative CTA images to automatically segment the aortic lumen and thrombus after EVAR. The absence of any required input makes the model very suitable for the interoperability and replicability of the results. A mean Dice–Sørensen coefficient of 0.83 is obtained. Although the results fit with the ones provided by the state of the art, such an approach does not clinically validate the model. Aortic measurements are, in fact, crucial to assessing how well the network does approximate 3D surface reconstruction.

The purpose of this study is to validate a 3D reconstruction methodology for the abdominal aortic vessel. Its application will not be solely limited to EVAR planning; it also serves as an initial step in the development of an automatic patient-specific CFD workflow with the goal of future works aiming to assess the aortic wall’s rupture risk. This first step is to obtain an anatomically accurate 3D model of the reconstructed aorta. By addressing the segmentation performance, robustness, and clinical reliability of the pipeline on several anatomies, the model’s usability and clinical consistency are assessed. The employment of such a mono-input 3D surface generation algorithm in complex hemodynamic solvers will enhance the quantitative CFD analysis of the aneurysmatic blood flow. To accomplish this, obtaining quality meshes is key in terms of the numerical stability and accuracy of the solution. The DL approach will help users in obtaining anatomically correct 3D models, which will be used for EVAR planning and patient-specific hemodynamic digital analysis.

2. Materials and Methods

2.1. Study Design

This study evaluates the performance, replicability and usability of a DL-based aortic lumen reconstruction and measurement pipeline, which is crucial when planning safety-critical tasks like abdominal aortic aneurysm EVAR. The goal is to develop and validate a robust single-class model that is capable of accurately detecting specific aorto-iliac anatomies. Four key characteristics—segmentation performance, robustness, reliability, and usability—are assessed based on [11,12].

Segmentation performances are evaluated by means of intersection over union and F1-score metrics; the former evaluates more accurately bad classification instances, and they, respectively, reflect the worst and average performance of the network among each test set anatomy.

Robustness is tested on input ROI translation along absolute and partial cardinal directions. By predicting multiple input instances, the network’s behavior is observed so as to assess segmentation metrics sensitivity with respect to input perturbations.

Reliability is validated by comparing manual measurements, performed by a vascular surgeon, on 3D reconstructions computed by the model, together with the ones obtained with reference software. Usability is evaluated based on input requirements and elapsed time.

The methodology workflow is schematized in Figure 1. From the original CTA image and its relative mask, the original–GT image couples are extracted by a Python (v. 3.11.6) script. Masks and GT are obtained from widely known reference software ITK-Snap, (v. 4.0.2) [13] (ITK-SNAP Home, itksnap.org). Once the dataset is built, the network is trained. At testing time, the raw prediction’s output stack is processed to obtain the reconstructed aortic lumen’s surface and test metrics, following the performance, robustness, reliability, and usability assessments cited above.

Several tools are exploited by the model and the evaluation algorithm. Images are semi-automatically processed with ITK-Snap and Python [14]. The network is implemented using the Pytorch framework. Centerlines are obtained with the VMTK [15] toolkit. Measurements are performed manually on the widely known post-processing toolkit Paraview [16]. It must be underlined that all segmentations are performed exclusively using a CPU device to assess the algorithm performance in the most general usage context. The user is provided with a graphics user interface to perform CTA slicing, prediction, centerline extraction and 3D reconstructions.

2.2. Data Acquisition and GT Extraction

In this study, training, evaluation, and test data were provided by a single sanitary structure of the Lombardia region in anonymous form. The following diagnostic devices are used to collect imaging data: General Electric’s LightSpeed16 Rad (Cincinnati, OH, USA), Philip’s (Amsterdam, Netherlands) iCT SP, Ingenuity CT, and Brilliance 16. The spatial resolution ranges from 0.779 mm to 0.977 in the x and y directions. Axial slices are extracted by slicing the DICOM volume of interest. The data pool is composed of ${\mathrm{N}}_{\mathrm{C}\mathrm{T}\mathrm{s},\mathrm{t}\mathrm{o}\mathrm{t}}^{\mathrm{t}\mathrm{r}\mathrm{a}\mathrm{i}\mathrm{n}-\mathrm{e}\mathrm{v}\mathrm{a}\mathrm{l}}=101,{\mathrm{N}}_{\mathrm{C}\mathrm{T}\mathrm{s},\mathrm{t}\mathrm{o}\mathrm{t}}^{\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}}=14$ axial CTA and includes aneurysmatic patients with a mean age of 75.6 ± 7.67. The data pool consists of 114,141 images split into training, validation and testing with an 80%–15%–5% distribution.

GT extraction is performed using ITK-Snap’s thresholding semi-automatic active contouring and a custom Python script to generate input and target images. ITK-snap (v. 4.0.2) is the chosen reference labeling software for its wide consensus across the biomedical engineering community. The interest area includes the abdominal section, ranging from common iliac arteries to the celiac trunk. Upper and lower threshold values are selected to isolate CTA’s contrast medium in the abdominal aortic region. Usually, this process takes some parameter tuning, repeating the labeling multiple times. Despite this, its low characteristic operative time and simple active contouring based semi-automatic segmentation guarantee a reasonable amount of time effort by the trained personnel. The engineering team has been trained in the aortic lumen labeling task under the supervision of a vascular surgeon.

The Python script extracts the input CTA and GT, respectively, starting from the DICOM series pixel array and NIFTI (Neuroimaging Informatics Technology Initiative) GT array proxy files, the latter obtained from ITK-Snap labeling method. Then, images and GT volumes are cropped along the ROI. Finally, histogram equalization is applied to the original images. The output of the script is a stack of mono-channel gray CTA images and a stack of mono-channel binary GT images. The extraction algorithm is schematized in Figure 2. The target patient’s CTA and GT NIFTI file is loaded into the Python script following DICOM Series attributes [17], such as Image Position Patient, Pixel Spacing, Slice Thickness and Spacing Between Slices, to sort and extract angiography series. The algorithm now pre-processes GT and CTA slice images input volumes, proceeding as shown below:

1.

ROI extraction: Once loaded, ROI slicing is performed, extracting the inner volume from the DICOM pixel array and NIFTI array proxies with $(\mathrm{256,256},N_{slices})$ dimension.

2.

Pre-processing: The sorted and cropped three-dimensional image volume is equalized. Original images and GT stacks are then stored into an anonymized folder in image format.

2.3. Model Architecture

The architecture of choice is a 2D U-Net [18] for axial CTA images segmentation, as one of the most popular architectures used for the semantic segmentation of CTA images, which is suitable for a proof of concept. The input is a gray image with shape $({\mathrm{n}}_{\mathrm{b}\mathrm{a}\mathrm{t}\mathrm{c}\mathrm{h}}$, ${\mathrm{n}}_{\mathrm{i}\mathrm{n},\mathrm{c}\mathrm{h}\mathrm{a}\mathrm{n}\mathrm{n}\mathrm{e}\mathrm{l}\mathrm{s}},256,256)$ with ${\mathrm{n}}_{\mathrm{b}\mathrm{a}\mathrm{t}\mathrm{c}\mathrm{h}}=4$ and ${\mathrm{n}}_{\mathrm{i}\mathrm{n},\mathrm{c}\mathrm{h}\mathrm{a}\mathrm{n}\mathrm{n}\mathrm{e}\mathrm{l}\mathrm{s}}=1$. The output is a raw logits tensor with shape $({\mathrm{n}}_{\mathrm{b}\mathrm{a}\mathrm{t}\mathrm{c}\mathrm{h}}$, ${\mathrm{n}}_{\mathrm{o}\mathrm{u}\mathrm{t},\mathrm{c}\mathrm{h}\mathrm{a}\mathrm{n}\mathrm{n}\mathrm{e}\mathrm{l}\mathrm{s}},256,256)$ with ${\mathrm{n}}_{\mathrm{o}\mathrm{u}\mathrm{t},\mathrm{c}\mathrm{h}\mathrm{a}\mathrm{n}\mathrm{n}\mathrm{e}\mathrm{l}\mathrm{s}}=1$. During training, which takes place from scratch, a binary stochastic gradient descent optimizer with an initial learning rate of 0.001 is used. Binary cross-entropy on logits loss is the model’s criterion of choice. Moreover, a learning rate scheduler is adopted to reduce the learning rate value when loss plateauing occurs with a decay factor 0.5. A manually tuned static binarization threshold is used. Image augmentation is performed by applying horizontal and vertical random flips, color jittering, and center cropping. Such a choice is given to provide the network with more flexibility when predicting on different image orientations and intensity spectra. Zoom factors are considered as performing center cropping.

The algorithm converged after 87 epochs, imposing 0.0001 as convergence criteria and four epochs as patience. To achieve the best-performing model, several trainings are launched on different hyperparameter sets and dataset distributions. The best-performing model is selected based on IoU and F1 metrics and AOC value at the testing time.

2.4. Test Dataset Description

The developed test and training sets’ inclusion criteria aim to the prioritization of (i) pathologic vessels affected by abdominal aortic aneurysm, (ii) complex anatomies such as tortuous and asymmetric vessels, (iii) large abdominal aortic aneurysms with a diameter greater than 5 cm, (iv) other major aorto-iliac pathologies, and (v) data coming from different diagnostic equipment: General Electrics’ BrightSpeed, Philips’ Optima CT660, and Siemens’ Definition Edge. Patients with aortic dissection and thoracic aneurysms were not included in either of the sets. Two women and twelve men were in the test set; the mean age was $76.3\pm 5.1$. All participants included had an aneurysmatic aorta.

Each training, the evaluation and test CTA scan in the dataset were anonymized. Due to the retrospective nature of such a study and the anonymization of source data, the ethical committee did not retain necessary an ethical approval; sensible data and information were neither included nor published.

In this study, only pathologic vessels are segmented, subdividing them first into A and AC. The descriptor A indicates an aneurysmatic aorta in one or more sections/branches, e.g., iliac aneurysm, whilst AC accounts for anatomically complex pathologic vessels related to other significant complications such as stenosis, occlusion, and penetrating arteriosclerotic ulcers. To further characterize test set anatomies, two properties are set for each patients’ vessel by means of the following qualitative vessel’s properties: (i) aneurysm location and (ii) other significant pathologies. Lastly, to accurately investigate the network’s performance on feature recognition of high-interest sections, aneurysm and other pathologies’ locations are introduced as characterizing anatomical properties. In

Table 1. The test set anatomical and pathological classification.

Patient ID	Aneurysm (Location)	Pathology	Class	Patient ID	Aneurysm (Location)	Pathology	Class
ID_1	JR	AAA	A	ID_8	IR	AAA	A
ID_2	JR	AAA + LIIO	AC	ID_9	PR	AAA	A
ID_3	JR	AAA + RCI PAU	AC	ID_10	IR	AAA	A
ID_4	JR	AAA + BCIA	AC	ID_11	IR	AAA	A
ID_5	IR	AAA	A	ID_12	IR	AAA	A
ID_6	IR	AAA	A	ID_13	IR	AAA	A
ID_7	IR	AAA + LCI PAU	AC	ID_14	JR	AAA	A

, the aforementioned classification and description of the test set’s anatomies is reported.

2.5. Surface Generation Algorithm

Aortic lumen 3D surface reconstruction is obtained from raw predictions using the following method: firstly, the point cloud is extracted from the predicted volume. Secondly, each axial closed contour is computed alongside its centroid. Vertex normal vectors are computed, guaranteeing correct orientation of the surface curvature, inwardly to the aortic lumen’s centerline inner point. This is mandatory to prevent holes and non-manifold, unclosed surfaces. Proceeding from bottom to top, point cloud normal are computed for each axial section of the cloud following the Python pseudo-code below:

1.

Axial single contour ${\mathrm{C}}_{\mathrm{i}}$ coordinates matrix $[{\overline{\mathrm{x}}}_{{\mathrm{C}}_{\mathrm{i}}},{\overline{\mathrm{y}}}_{{\mathrm{C}}_{\mathrm{i}}},{\overline{\mathrm{z}}}_{{\mathrm{C}}_{\mathrm{i}}}]$ extraction with marching squares algorithm [19] from U-Net predictions. With ${\overline{\mathrm{x}}}_{{\mathrm{C}}_{\mathrm{i}}},{\overline{\mathrm{y}}}_{{\mathrm{C}}_{\mathrm{i}}},{\overline{\mathrm{z}}}_{{\mathrm{C}}_{\mathrm{i}}}$ the column vector contains, respectively, the x, y and z coordinates of each single contour’s point. Each contour belonging to an axial section is characterized by the same z coordinate.

2.

Axial contour ${\mathrm{C}}_{\mathrm{i}}$ centroid extraction $\left\{\begin{array}{c}{\mathrm{x}}_{\mathrm{j},{\mathrm{c}}_{\mathrm{i}}}={\displaystyle \frac{\min \left({\overline{\mathrm{x}}}_{{\mathrm{C}}_{\mathrm{i}}}\right)+\mathrm{m}\mathrm{a}\mathrm{x}\left({\overline{\mathrm{x}}}_{{\mathrm{C}}_{\mathrm{i}}}\right)}{2}}\\ {\mathrm{y}}_{\mathrm{j},{\mathrm{c}}_{\mathrm{i}}}={\displaystyle \frac{\min \left({\overline{\mathrm{y}}}_{{\mathrm{C}}_{\mathrm{i}}}\right)+\mathrm{m}\mathrm{a}\mathrm{x}\left({\overline{\mathrm{y}}}_{{\mathrm{C}}_{\mathrm{i}}}\right)}{2}}\end{array}\right.$.

3.

Normal computation for each point of the contour ${\mathrm{C}}_{\mathrm{i}}$ on each section of the point cloud.

$\mathrm{f}\mathrm{o}\mathrm{r} \mathrm{x},\mathrm{y},\mathrm{z} \mathrm{i}\mathrm{n} \mathrm{z}\mathrm{i}\mathrm{p}\left({\overline{\mathrm{x}}}_{{\mathrm{C}}_{\mathrm{i}}},{\overline{\mathrm{y}}}_{{\mathrm{C}}_{\mathrm{i}}},{\mathrm{z}}_{{\mathrm{C}}_{\mathrm{i}}}\right):$

$\mathrm{i}\mathrm{f} {\mathrm{z}}_{{\mathrm{C}}_{\mathrm{i}}}=={\mathrm{z}}_{\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{r}\mathrm{t}}$:

                       #Bottom patches

                       $\overline{\mathrm{n}}=[\mathrm{0,0},1]$

$\mathrm{e}\mathrm{l}\mathrm{i}\mathrm{f} {\mathrm{z}}_{{\mathrm{C}}_{\mathrm{i}}}=={\mathrm{z}}_{\mathrm{e}\mathrm{n}\mathrm{d}}:$

                       #Top patches

                       $\overline{\mathrm{n}}=[\mathrm{0,0},-1]$

$\mathrm{e}\mathrm{l}\mathrm{s}\mathrm{e}:$

                       #Wall patches

                       $\overline{\mathrm{n}}=\left({\displaystyle \frac{{\mathrm{x}}_{\mathrm{j},{\mathrm{C}}_{\mathrm{i}}}-{\mathrm{x}}_{{\mathrm{C}}_{\mathrm{i}}}}{\mathrm{d}}},{\displaystyle \frac{{\mathrm{y}}_{\mathrm{j},{\mathrm{C}}_{\mathrm{i}}}-{\mathrm{y}}_{{\mathrm{C}}_{\mathrm{i}}}}{\mathrm{d}}},{\displaystyle \frac{{\mathrm{z}}_{\mathrm{j},{\mathrm{C}}_{\mathrm{i}}}-{\mathrm{z}}_{{\mathrm{C}}_{\mathrm{i}}}}{\mathrm{d}}}\right)$,

                       with $\mathrm{d}=\sqrt{{\left({\mathrm{x}}_{\mathrm{j},{\mathrm{C}}_{\mathrm{i}}}-{\mathrm{x}}_{{\mathrm{C}}_{\mathrm{i}}}\right)}^2+{\left({\mathrm{y}}_{\mathrm{j},{\mathrm{C}}_{\mathrm{i}}}-{\mathrm{y}}_{{\mathrm{C}}_{\mathrm{i}}}\right)}^2+{\left({\mathrm{z}}_{\mathrm{j},{\mathrm{C}}_{\mathrm{i}}}-{\mathrm{z}}_{{\mathrm{C}}_{\mathrm{i}}}\right)}^2}$

A 3D triangulated aortic lumen surface is obtained using a screened Poisson surface reconstruction algorithm [20]. The obtained mesh is then automatically cleaned from any spurious and/or disconnected components. The algorithm is visually represented in Figure 3.

2.6. Model Evaluation

Segmentation performance is evaluated by means of IoU and F1-score metrics. Each metric is computed with its mean value and standard deviation along the total slice number. Moreover, the AOC, ROC, confusion matrix, precision and recall are also evaluated over the test set.

Robustness is assessed by perturbating the region of interest as input only. For each test CTA, the aortic lumen on nine different ROIs is predicted—one for each absolute or partial cardinal direction: C, N, S, W, E, NW, NE, SW, and SE. Each ROI volume is extracted following a fixed rule which imposes a positive or negative variation in the ROI’s origin’s coordinates. The magnitude of such a perturbation corresponds to the α = 12.5% of the ROI axial shape. To evaluate such an incremental value in pixels, one can simply perform the following: $\mathsf{\Delta }={\displaystyle \frac{\mathrm{N}\mathsf{\alpha }}{100}}=32\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}$, where $\mathrm{N}$ is the ROI’s edge dimensions in pixels. For example, the coordinates of the ROI’s center for the north (N) direction can be expressed by the following: ${\mathrm{x}}_0={\mathrm{x}}_0, {\mathrm{y}}_0={\mathrm{y}}_0·(1+\mathsf{\alpha })$, where ${\mathrm{x}}_0=256$ and ${\mathrm{y}}_0=256$ are the axial section of the volume of interest’s center. It has to be underlined that the absolute origin of a single axial section is the upper left corner of the image with a positive x direction from left to right and positive y direction from top to bottom. To correctly analyze how perturbing the input ROI influences predictions’ anatomical coherence from one orientation to another proceeding slice-wise, the IoU, F1-score average and standard deviation values between predictions for the perturbed ROI along a generic direction and its relative GT are computed. The IoU and F1-score are defined as follows in Equations (1) and (2):

$$\mathrm{F}1={\displaystyle \frac{2\cdot \mathrm{T}\mathrm{P}}{2\cdot \mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N}}}$$

(1)

$$\mathrm{I}\mathrm{o}\mathrm{U}={\displaystyle \frac{\left|\mathrm{A}\cap \mathrm{B}\right|}{\left|\mathrm{A}\cup \mathrm{B}\right|}}$$

(2)

where $\mathrm{T}\mathrm{P}, \mathrm{F}\mathrm{P}, \mathrm{a}\mathrm{n}\mathrm{d} \mathrm{F}\mathrm{N}$ are, respectively, the true positives, false positives, and false negatives. $\mathrm{A}$ and $\mathrm{B}$ are, respectively, the output of the network and GT tensors. Moreover, for each patient and each ROI orientation, the global metrics’ average and standard deviation value are computed. The global average over each patient and each predicted volume obtained with ROI perturbations is also computed, following Equation (3).

$$\overline{\mathrm{m}}={\displaystyle \frac{{\sum }_{\mathrm{i}=0}^{{\mathrm{N}}_{\mathrm{p}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{s}}}{\sum }_{\mathrm{j}=\mathrm{C}}^{\mathrm{S}\mathrm{E}}{\sum }_{\mathrm{k}=0}^{{\mathrm{N}}_{\mathrm{s}\mathrm{l}\mathrm{i}\mathrm{c}\mathrm{e}\mathrm{s}}^{\mathrm{I}{\mathrm{D}}_{\mathrm{i}}}}{\mathrm{m}}_{\mathrm{k},\mathrm{I}{\mathrm{D}}_{\mathrm{i}}^{\mathrm{j}}}}{{\sum }_{\mathrm{i}=0}^{{\mathrm{N}}_{\mathrm{p}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{s}}}({\mathrm{N}}_{\mathrm{o}\mathrm{r}\mathrm{i}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{s}}·{\sum }_{\mathrm{k}=0}^{{\mathrm{N}}_{\mathrm{s}\mathrm{l}\mathrm{i}\mathrm{c}\mathrm{e}\mathrm{s}}^{\mathrm{I}{\mathrm{D}}_{\mathrm{i}}}}{\mathrm{m}}_{\mathrm{k},\mathrm{I}{\mathrm{D}}_{\mathrm{i}}^{\mathrm{j}}})}}$$

(3)

where $\overline{\mathrm{m}}$ is the average generic metric (F1 or IoU), ${\mathrm{N}}_{\mathrm{p}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{s}}$ is the total number of test patients, ${\mathrm{N}}_{\mathrm{s}\mathrm{l}\mathrm{i}\mathrm{c}\mathrm{e}\mathrm{s}}^{\mathrm{I}{\mathrm{D}}_{\mathrm{i}}}$ is the total number of slices for test patient $\mathrm{i}$, ${\mathrm{m}}_{\mathrm{k},\mathrm{I}{\mathrm{D}}_{\mathrm{i}}^{\mathrm{j}}}$ is the value of the generic metric at slice number $\mathrm{k}$ and orientation $\mathrm{j}\in [\mathrm{N},\mathrm{S},\mathrm{W},\mathrm{E},\mathrm{N}\mathrm{W},\mathrm{N}\mathrm{E},\mathrm{S}\mathrm{W},\mathrm{S}\mathrm{E}]$ for patient $\mathrm{i}$, and ${\mathrm{N}}_{\mathrm{o}\mathrm{r}\mathrm{i}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{s}}=9$ is the total number of direction for the input ROI perturbations.

Clinical reliability is assessed by comparing centerline-based measurements performed on both the ITK-Snap and U-Net reconstructed geometries by means of absolute and relative percentage errors, as defined in Equations (4) and (5).

$${\mathsf{ϵ}}_{\mathrm{a}}=|{\mathrm{v}}^{\mathrm{G}\mathrm{T}}-{\mathrm{v}}^{\mathrm{m}}|$$

(4)

$${\mathsf{ϵ}}_{\mathrm{r}}={\displaystyle \frac{\left|{\mathrm{v}}^{\mathrm{G}\mathrm{T}}-{\mathrm{v}}^{\mathrm{m}}\right|}{{\mathrm{v}}^{\mathrm{G}\mathrm{T}}}}\cdot 100$$

(5)

where ${\mathrm{v}}^{\mathrm{G}\mathrm{T}}$ and ${\mathrm{v}}^{\mathrm{m}}$ are the diameter or centerline length values, respectively, measured on ITK-Snap and U-Net 3D reconstruction geometries. Bland–Altman plots are also provided in the Appendix section from Figure A1, Figure A2, Figure A3 and Figure A4 for diameters and from Figure A5, Figure A6 and Figure A7 for the centerline’s length. Once a region of interest is reconstructed, the vessel’s centerline is computed. The user can then perform the required reference measurements, taken on a plane orthogonal to the centerline, as shown in Figure 4e. The abdominal aneurysm and iliac arteries’ lumen diameter are measured at three distinct locations:

• Right/Left Iliac Luminal Diameter, ${\mathsf{\varphi }}_{\mathrm{r},\mathrm{ }\mathrm{i}\mathrm{l}\mathrm{i}\mathrm{a}\mathrm{c}}$, ${\mathsf{\varphi }}_{\mathrm{l},\mathrm{ }\mathrm{i}\mathrm{l}\mathrm{i}\mathrm{a}\mathrm{c}}$: right and left common iliac artery proximally to their bifurcation, as shown in Figure 4a.
• L3 Infra Renal Aortic Diameter, ${\mathsf{\varphi }}_{\mathrm{L}3}$: located on the third lumbar vertebral body’s upper margin (L3), as shown in Figure 4b, and at a distance ${\mathrm{l}}_{\mathrm{A}\mathrm{B}-\mathrm{L}3}$ from AB, as shown in Figure 4d.
• Juxta Renal Aortic Diameter, ${\mathsf{\varphi }}_{\mathrm{j}\mathrm{u}\mathrm{x}\mathrm{t}\mathrm{r}}$: the aortic lumen immediately below the lowest renal artery, as shown in Figure 4c.

The coronal plane was used to measure the vertical distance between the aortic bifurcation and L3 body ${\mathrm{l}}_{\mathrm{A}\mathrm{B}-\mathrm{L}3}$, as shown in Figure 4d. The centerline-based aortic lumen’s length measure between reference sections is taken considering the following bound segments:

• Lowest RA to Aortic Bifurcation length, ${\mathrm{L}}_{\mathrm{L}\mathrm{R}-\mathrm{A}\mathrm{B}}$;
• Aortic Bifurcation to Right/Left Iliac Bifurcation, ${\mathrm{L}}_{\mathrm{r},\mathrm{ }\mathrm{A}\mathrm{B}-\mathrm{I}\mathrm{B}}$, ${\mathrm{L}}_{\mathrm{l},\mathrm{ }\mathrm{A}\mathrm{B}-\mathrm{I}\mathrm{B}}$.

The centerline’s length is computed as a sum of the Euclidean distances between the centerline points location as ${\mathrm{d}}_{\mathrm{E}\mathrm{u}}={\sum }_{\mathrm{i}=1}^{\mathrm{i}={\mathrm{N}}_{\mathrm{p}\mathrm{o}\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{s}}}|{\overrightarrow{\mathrm{r}}}_{\mathrm{i}+1}-{\overrightarrow{\mathrm{r}}}_{\mathrm{i}}|$, where ${\mathrm{N}}_{\mathrm{p}\mathrm{o}\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{s}}$ is the total number of points of the interest centerline’s branch, and $\overrightarrow{\mathrm{r}}$ is the position vector, which is referenced at the starting point of the centerline itself. Such a validation strategy is different from the recognized preoperative AAA measurement standards, according to the latest ESVS European Society of Vascular Surgery guidelines outlined in Wanhainen et al. [21]. To minimize measurement errors, the approach is modified and standardized by both vascular surgeon and engineers as previously described. The diameter was measured on the longest axis of the lumen from the inner-to-inner wall. Arterial thrombus and calcification were not included as the studied U-Net is of the mono-class type.

Usability and performance are also assessed by computing the average elapsed time to obtain the aortic lumen reconstruction and the average elapsed time per slice. It should be emphasized that all the reconstructions obtained by the algorithm presented in this study were retrieved from central ROI orientation predictions and CPU devices.

3. Results

3.1. Model Performance

Here, we reported three cases for performance discussion, which represent the top, average, and worst-performing cases among test sets ID_3, ID_2, and ID_12, respectively. Their performance by means of the above-mentioned metrics for each orientation are reported in

Table 2. Mean value and standard deviation of the metrics distribution. Average performance on test set patient ID_3.

Patient	ROI	IoU		F1
		Avg	STD	Avg	STD
ID_3	C	0.857	0.084	0.920	0.051
	N	0.864	0.080	0.925	0.048
	S	0.841	0.084	0.911	0.051
	W	0.841	0.092	0.911	0.057
	E	0.862	0.077	0.924	0.046
	NW	0.854	0.084	0.919	0.051
	NE	0.869	0.078	0.928	0.046
	SW	0.829	0.092	0.904	0.057
	SE	0.833	0.097	0.906	0.061

,

Table 3. Mean value and standard deviation of the metrics distribution. Average performance on test set patient ID_2.

Patient	ROI	IoU		F1
		Avg	STD	Avg	STD
ID_2	C	0.816	0.084	0.896	0.052
	N	0.822	0.083	0.900	0.050
	S	0.795	0.087	0.883	0.055
	W	0.810	0.093	0.892	0.059
	E	0.814	0.089	0.894	0.055
	NW	0.825	0.081	0.901	0.050
	NE	0.819	0.086	0.898	0.052
	SW	0.796	0.091	0.883	0.058
	SE	0.793	0.094	0.881	0.060

and

Table 4. Mean value and standard deviation of the metrics distribution. Average performance on test set patient ID_12.

Patient	ROI	IoU		F1
		Avg	STD	Avg	STD
ID_12	C	0.770	0.164	0.859	0.117
	N	0.778	0.164	0.864	0.117
	S	0.749	0.170	0.844	0.123
	W	0.771	0.172	0.858	0.125
	E	0.773	0.164	0.861	0.116
	NW	0.774	0.174	0.860	0.127
	NE	0.778	0.166	0.864	0.119
	SW	0.743	0.177	0.839	0.130
	SE	0.745	0.167	0.842	0.120

. Metrics for other test cases are reported in the appendix from

Table A1. Mean value and standard deviation of the metrics distribution. Average performance on test set patient ID_1.

Patient	ROI	IoU		F1
		Avg	STD	Avg	STD
ID_1	C	0.828	0.077	0.904	0.047
	N	0.830	0.078	0.905	0.047
	S	0.743	0.174	0.838	0.145
	W	0.829	0.076	0.904	0.046
	E	0.824	0.077	0.901	0.047
	NW	0.830	0.078	0.905	0.047
	NE	0.832	0.077	0.906	0.047
	SW	0.738	0.171	0.835	0.139
	SE	0.717	0.229	0.805	0.225

,

Table A2. Mean value and standard deviation of the metrics distribution. Average performance on test set patient ID_4.

Patient	ROI	IoU		F1
		Avg	STD	Avg	STD
ID_4	C	0.847	0.099	0.913	0.066
	N	0.844	0.096	0.912	0.062
	S	0.837	0.116	0.906	0.080
	W	0.855	0.095	0.919	0.062
	E	0.846	0.105	0.913	0.069
	NW	0.850	0.092	0.916	0.059
	NE	0.843	0.099	0.911	0.064
	SW	0.836	0.119	0.905	0.083
	SE	0.826	0.133	0.898	0.095

,

Table A3. Mean value and standard deviation of the metrics distribution. Average performance on test set patient ID_5.

Patient	ROI	IoU		F1
		Avg	STD	Avg	STD
ID_5	C	0.758	0.103	0.858	0.071
	N	0.761	0.101	0.860	0.069
	S	0.716	0.127	0.828	0.089
	W	0.757	0.108	0.857	0.075
	E	0.761	0.104	0.860	0.072
	NW	0.762	0.105	0.861	0.072
	NE	0.753	0.111	0.854	0.076
	SW	0.707	0.134	0.820	0.096
	SE	0.730	0.114	0.839	0.080

,

Table A4. Mean value and standard deviation of the metrics distribution. Average performance on test set patient ID_6.

Patient	ROI	IoU		F1
		Avg	STD	Avg	STD
ID_6	C	0.748	0.101	0.848	0.072
	N	0.751	0.111	0.850	0.070
	S	0.726	0.127	0.848	0.090
	W	0.757	0.109	0.867	0.074
	E	0.761	0.103	0.870	0.071
	NW	0.782	0.104	0.851	0.073
	NE	0.723	0.101	0.844	0.078
	SW	0.707	0.124	0.810	0.095
	SE	0.710	0.104	0.835	0.075

,

Table A5. Mean value and standard deviation of the metrics distribution. Average performance on test set patient ID_7.

Patient	ROI	IoU		F1
		Avg	STD	Avg	STD
ID_7	C	0.789	0.092	0.879	0.061
	N	0.779	0.097	0.872	0.065
	S	0.785	0.094	0.876	0.064
	W	0.773	0.120	0.866	0.087
	E	0.792	0.091	0.881	0.060
	NW	0.751	0.142	0.849	0.104
	NE	0.785	0.098	0.876	0.066
	SW	0.775	0.120	0.867	0.089
	SE	0.789	0.094	0.879	0.063

,

Table A6. Mean value and standard deviation of the metrics distribution. Average performance on test set patient ID_8.

Patient	ROI	IoU		F1
		Avg	STD	Avg	STD
ID_8	C	0.843	0.094	0.911	0.059
	N	0.841	0.110	0.909	0.080
	S	0.841	0.101	0.910	0.063
	W	0.829	0.121	0.900	0.094
	E	0.834	0.111	0.904	0.078
	NW	0.827	0.132	0.898	0.105
	NE	0.828	0.135	0.898	0.103
	SW	0.820	0.130	0.895	0.089
	SE	0.833	0.111	0.904	0.071

,

Table A7. Mean value and standard deviation of the metrics distribution. Average performance on test set patient ID_9.

Patient	ROI	IoU		F1
		Avg	STD	Avg	STD
ID_9	C	0.710	0.14	0.821	0.107
	N	0.711	0.156	0.820	0.119
	S	0.691	0.171	0.803	0.138
	W	0.702	0.153	0.814	0.116
	E	0.720	0.133	0.830	0.095
	NW	0.705	0.158	0.816	0.119
	NE	0.718	0.147	0.827	0.109
	SW	0.674	0.190	0.787	0.162
	SE	0.703	0.157	0.815	0.120

,

Table A8. Mean value and standard deviation of the metrics distribution. Average performance on test set patient ID_10.

Patient	ROI	IoU		F1
		Avg	STD	Avg	STD
ID_10	C	0.858	0.094	0.920	0.059
	N	0.866	0.077	0.926	0.047
	S	0.801	0.147	0.881	0.105
	W	0.842	0.143	0.906	0.104
	E	0.861	0.082	0.923	0.051
	NW	0.834	0.136	0.902	0.097
	NE	0.865	0.077	0.925	0.048
	SW	0.781	0.171	0.865	0.124
	SE	0.796	0.146	0.878	0.102

,

Table A9. Mean value and standard deviation of the metrics distribution. Average performance on test set patient ID_11.

Patient	ROI	IoU		F1
		Avg	STD	Avg	STD
ID_11	C	0.771	0.090	0.867	0.063
	N	0.792	0.092	0.881	0.064
	S	0.776	0.087	0.870	0.060
	W	0.768	0.099	0.865	0.069
	E	0.779	0.090	0.872	0.063
	NW	0.790	0.115	0.877	0.082
	NE	0.751	0.141	0.849	0.110
	SW	0.774	0.109	0.868	0.079
	SE	0.780	0.092	0.873	0.063

,

Table A10. Mean value and standard deviation of the metrics distribution. Average performance on test set patient ID_13.

Patient	ROI	IoU		F1
		Avg	STD	Avg	STD
ID_13	C	0.769	0.130	0.862	0.102
	N	0.789	0.127	0.875	0.098
	S	0.717	0.150	0.824	0.123
	W	0.753	0.142	0.850	0.118
	E	0.767	0.133	0.860	0.106
	NW	0.779	0.150	0.865	0.126
	NE	0.787	0.140	0.871	0.114
	SW	0.696	0.198	0.799	0.191
	SE	0.713	0.164	0.819	0.137

and

Table A11. Mean value and standard deviation of the metrics distribution. Average performance on test set patient ID_14.

Patient	ROI	IoU		F1
		Avg	STD	Avg	STD
ID_14	C	0.814	0.132	0.891	0.088
	N	0.817	0.123	0.893	0.080
	S	0.816	0.132	0.892	0.089
	W	0.781	0.146	0.869	0.102
	E	0.832	0.116	0.903	0.077
	NW	0.787	0.135	0.874	0.092
	NE	0.833	0.110	0.904	0.072
	SW	0.778	0.147	0.867	0.104
	SE	0.836	0.118	0.905	0.080

.

Case ID_3 shows the best metrics among the other patients, respectively, achieving an IoU = 0.850 ± 0.087 and an F1 = 0.916 ± 0.053 average values over each ROI perturbation. The superior mesenteric artery has been detected successfully, including the tripod. Common iliac arteries and aneurysmatic sections are also segmented correctly as well as the right common iliac PAU, as shown in Figure 5a.

Case ID_2, representing the average U-Net performance, reported an IoU = 0.810 $\pm$ 0.088 and an F1 = 0.892 $\pm$ 0.055 over each ROI’s orientation. The LIIO of such a vessel has been successfully detected. Moreover, the common iliac section and the superior mesenteric artery were also identified and reconstructed successfully, as can be seen from Figure 5b.

Case ID_12 represents the worst performance scenario, reporting an IoU = 0.752 $\pm$ 0.153 and an F1 = 0.847 $\pm$ 0.126 over each ROI’s orientation. The network provided solid and reliable results despite lower average metrics’ value and a higher standard deviation across slices. The 3D reconstruction for case ID_12 is reported in Figure 5c.

The maximum average metric’s values difference between orientations are computed for the mentioned patients, which are defined as follows:

$${\mathrm{m}}_{\mathrm{a}\mathrm{v}\mathrm{g}}^{\mathrm{I}\mathrm{D}\_\mathrm{i}}=\left({\mathrm{m}}_{\mathrm{a}\mathrm{v}\mathrm{g},\mathrm{m}\mathrm{a}\mathrm{x}}^{\mathrm{I}{\mathrm{D}}_{\mathrm{i}}}-{\mathrm{m}}_{\mathrm{a}\mathrm{v}\mathrm{g},\mathrm{m}\mathrm{i}\mathrm{n}}^{\mathrm{I}{\mathrm{D}}_{\mathrm{i}}}\right)·100$$

(6)

with ${\mathrm{m}}_{\mathrm{a}\mathrm{v}\mathrm{g},\mathrm{m}\mathrm{a}\mathrm{x}}^{\mathrm{I}\mathrm{D}\_\mathrm{i}}$ and ${\mathrm{m}}_{\mathrm{a}\mathrm{v}\mathrm{g},\mathrm{m}\mathrm{i}\mathrm{n}}^{\mathrm{I}\mathrm{D}\_\mathrm{i}}$, respectively, representing the maximum and minimum value of the average metrics among all orientations. Each of the three cases, ID_3, ID_2, and ID_12, show the small variability of the average metrics and its standard deviation across all perturbing orientations. For case ID_3, following Equation (6), $\mathsf{\Delta }\mathrm{F}{1}_{\mathrm{a}\mathrm{v}\mathrm{g}}^{\mathrm{I}\mathrm{D}\_3}=\left(0.925-0.904\right)·100=2.1\%$ and $\mathsf{\Delta }{\mathrm{I}\mathrm{o}\mathrm{U}}_{\mathrm{a}\mathrm{v}\mathrm{g}}^{\mathrm{I}\mathrm{D}\_3}=4\%$ (Table 2), for case ID_2 $\mathsf{\Delta }\mathrm{F}{1}_{\mathrm{a}\mathrm{v}\mathrm{g}}^{\mathrm{I}\mathrm{D}\_2}=2.0\%$ and $\mathsf{\Delta }{\mathrm{I}\mathrm{o}\mathrm{U}}_{\mathrm{a}\mathrm{v}\mathrm{g}}^{\mathrm{I}\mathrm{D}\_2}=3.2\%$ (Table 3), and for case ID_12 $\mathsf{\Delta }\mathrm{F}{1}_{\mathrm{a}\mathrm{v}\mathrm{g}}^{\mathrm{I}\mathrm{D}\_12}=2.5\%$ and $\mathsf{\Delta }{\mathrm{I}\mathrm{o}\mathrm{U}}_{\mathrm{a}\mathrm{v}\mathrm{g}}^{\mathrm{I}\mathrm{D}\_12}=3.5\%$ (Table 4). Such values indicate that the algorithm can segment the aortic lumen independently by the ROI instance, indicating coherent results with low inter-variability. To provide a better insight into the network performance on the test set, the AUC, ROC curve, and confusion matrix of the model are shown, starting with predictions and GT volumes as input data.

In Figure 6, the ROC and AUC value are plotted, showing an $\mathrm{A}\mathrm{U}\mathrm{C}=0.98$. Such a value suggests excellent performance on unseen data.

In Figure 7, the confusion matrix is reported with precision $\mathrm{p}=83.57\%$ and recall $\mathrm{r}=86.75\%$. For the sake of clarity, in

Table 5. True positive, true negative, false positive and false negative number and relative percentage value.

	Number (#)	Percentage (%)
True Positive	3,622,473	1.71
True Negative	207,186,594	97.7
False Positive	712,005	0.33
False Negative	553,424	0.26
Total Pixel Population	212,074,496	100

, more detailed metrics such as the true positive, true negative, false positive, and false negative number, precision and recall are reported.

3.2. Model Reliability

A comparison between obtained measurements and those evaluated from ITK-Snap’s reference reconstructions is reported by means of absolute errors on diameters and aortic lumen’s lengths in

Table 6. Absolute errors on diameters and lengths. Best performance case ID_3, average performance case ID_2, and worst performance ID_12 are highlighted in gray.

$\mathbf{DIAMETERS} \mathsf{\varphi }$$\mathbf{AND} \mathbf{CENTERLINE} \mathbf{LENGTHS} {\mathbf{L}}_{\mathbf{i}}$—ABSOLUTE ERRORS [mm]
Patient	${\mathsf{\varphi }}_{\mathbf{j}\mathbf{u}\mathbf{x}\mathbf{t}\mathbf{r}}$	${\mathsf{\varphi }}_{\mathbf{L}3}$	${\mathsf{\varphi }}_{\mathbf{l},\mathbf{i}\mathbf{l}\mathbf{i}\mathbf{a}\mathbf{c}}$	${\mathsf{\varphi }}_{\mathbf{r},\mathbf{i}\mathbf{l}\mathbf{i}\mathbf{a}\mathbf{c}}$	${\mathbf{L}}_{\mathbf{L}\mathbf{R}-\mathbf{A}\mathbf{B}}$	${\mathbf{L}}_{\mathbf{l},\mathbf{A}\mathbf{B}-\mathbf{I}\mathbf{B}}$	${\mathbf{L}}_{\mathbf{r},\mathbf{A}\mathbf{B}-\mathbf{I}\mathbf{B}}$
ID_1	1.910	0.160	3.000	1.000	0.620	1.260	3.000
ID_2	0.240	1.010	1.000	1.000	3.700	2.420	7.000
ID_3	3.220	0.940	1.000	1.000	0.420	0.580	2.000
ID_4	10.610	1.250	1.000	1.000	1.810	7.110	5.000
ID_5	0.420	0.220	0.000	2.000	0.530	0.430	0.000
ID_6	0.330	0.080	1.000	8.000	5.020	5.220	0.000
ID_7	0.970	0.870	4.000	0.000	0.420	1.000	3.000
ID_8	1.740	1.450	3.000	0.000	0.460	0.670	1.000
ID_9	1.320	1.940	3.000	4.000	0.780	0.670	9.000
ID_10	0.850	0.730	1.000	3.000	0.710	0.120	1.000
ID_11	2.070	2.280	3.000	1.000	1.530	0.720	2.000
ID_12	1.290	1.680	1.000	2.000	3.960	3.160	1.000
ID_13	0.920	2.280	3.000	3.728	4.600	2.400	1.000
ID_14	1.500	0.180	1.000	2.000	3.410	1.090	5.000

. Single measurement’s value from reference software and U-Net surface reconstructions can be found in Table S1 Supplementary File. The worst, average and best test cases are highlighted in gray. Looking at Table 6 and the Bland–Altman plots in the Appendix, from Figure A1, Figure A2, Figure A3 and Figure A4 for diameters and to Figure A5, Figure A6 and Figure A7 for centerline length, the computed mean absolute error for reference aortic lumen diameters measurement in the juxta renal, L3 body, and left and right iliac sections is, respectively, 1.956 mm, 1.076 mm, 1.917 mm, and 1.997 mm.

The mean absolute error for the reference aortic lumen centerline’s length measurement in the aneurysm and iliac sections is, respectively, 2.857 mm, 1.857 mm, and 2.123 mm. The average absolute error obtained from the reliability assessment is, respectively, ${\overline{\mathsf{ϵ}}}_{\mathsf{\varphi },\mathrm{a}}=1.73 \mathrm{m}\mathrm{m}$ for diameters and ${\overline{\mathsf{ϵ}}}_{\mathrm{l},\mathrm{a}}=2.27 \mathrm{m}\mathrm{m}$ for the centerline’s length. In addition, from

Table 7. Relative errors on diameters and lengths. Best performance case ID_3, average performance case ID_2, worst performance ID_12 are highlighted in gray.

$\mathbf{DIAMETERS} \mathsf{\varphi }$$\mathbf{AND} \mathbf{CENTERLINE} \mathbf{LENGTHS} {\mathbf{L}}_{\mathbf{i}}$—RELATIVE ERRORS [mm]
Patient	${\mathsf{\varphi }}_{\mathbf{j}\mathbf{u}\mathbf{x}\mathbf{t}\mathbf{r}}$	${\mathsf{\varphi }}_{\mathbf{L}3}$	${\mathsf{\varphi }}_{\mathbf{l},\mathbf{i}\mathbf{l}\mathbf{i}\mathbf{a}\mathbf{c}}$	${\mathsf{\varphi }}_{\mathbf{r},\mathbf{i}\mathbf{l}\mathbf{i}\mathbf{a}\mathbf{c}}$	${\mathbf{L}}_{\mathbf{L}\mathbf{R}-\mathbf{A}\mathbf{B}}$	${\mathbf{L}}_{\mathbf{l},\mathbf{A}\mathbf{B}-\mathbf{I}\mathbf{B}}$	${\mathbf{L}}_{\mathbf{r},\mathbf{A}\mathbf{B}-\mathbf{I}\mathbf{B}}$
ID_1	7.343	0.318	3.896	8.885	2.631	5.454	1.960
ID_2	1.002	1.802	30.756	20.133	5.223	1.818	1.408
ID_3	9.226	1.485	4.030	5.164	1.015	2.702	1.785
ID_4	40.511	2.136	10.197	52.901	2.994	3.125	2.325
ID_5	1.867	0.786	3.854	2.230	0.000	0.000	3.508
ID_6	1.475	0.234	42.078	54.262	0.000	1.219	9.756
ID_7	4.334	2.381	3.736	5.848	2.631	5.263	0.000
ID_8	8.438	2.575	2.287	3.030	1.041	3.947	0.000
ID_9	5.263	5.250	6.951	6.660	7.627	4.761	7.415
ID_10	4.231	1.979	5.186	0.827	0.990	2.631	5.368
ID_11	10.232	5.234	8.420	4.072	1.652	6.000	2.439
ID_12	6.796	3.571	45.622	28.468	0.934	1.724	4.166
ID_13	3.355	6.182	40.744	20.797	0.862	4.918	7.652
ID_14	5.654	0.377	26.495	7.045	3.649	2.083	3.846

, the average relative error obtained from the reliability assessment is, respectively, ${\overline{\mathsf{ϵ}}}_{\mathsf{\varphi },\mathrm{r}}=10.689\%$ for diameters and ${\overline{\mathsf{ϵ}}}_{\mathrm{l},\mathrm{r}}=3.060\%$ for the centerline’s length.

It is clear how, for cases ID_2, ID_4, ID_6, ID_11, ID_13 and ID_14, relative errors are characterized by higher values, especially for the common iliac arteries. Considering such sections, for instance in case ID_2, the underestimation of the aortic lumen by ITK-Snap segmentation leads to higher measurement relative errors, especially for diameters, which are lower in magnitude with respect to aortic lengths. To this purpose, a detailed view of the GT and prediction masks are reported in Figure 8.

From the axial, coronal, and sagittal view, the severe underestimation cited above can be seen. For the cases reported previously, the authors present the same detailed view from Figure A8, Figure A9, Figure A10 and Figure A11, qualitatively also clarifying how U-Net segmentations are more accurate when segmenting the aortic lumen. Such a phenomenon is due to the thresholding method used in ITK-Snap to segment the aortic lumen, which in most cases underestimates the lumen area. In fact, excessively lowering the lower threshold in ITK-Snap often expands the segmentation to other undesired areas and tissues. Hence, the very first layers of the aortic lumen are eroded from the GT mask. This leads to an underestimation of the aortic lumen and consequently higher error values.

3.3. Model Usability

Elapsed time, which aims to assess usability, is computed from prediction time to the end of the 3D surface reconstruction. The average elapsed time to segment a single slice is computed through linear fitting of the elapsed time-number of slice point samples and shown in Figure 9.

The correlation between the elapsed CPU computing time and number of slices is linear with an average slope of 0.1057 s per slice. To further assess the model’s usability, the required number of inputs is accounted for as well. The user must in fact solely place the region of interest in the GUI view-box, enclosing the vessel. This peculiarity provided the model with high learnability and usability.

The correlation between the elapsed time and the number of slices showed a linear dependence: there was a minimum elapsed time of 16.86 s, average elapsed time of 27.99 s and maximum elapsed time of 63.4 s. Such information can be found in Table S2 Supplementary File. Considering the use of CPU device to obtain predictions, these processing times could be drastically reduced by adopting GPU-based computing. Although, it is better to evaluate predictions on GPUs from a time efficiency perspective, such a choice is forced by the urge to provide a general tool that is able to work under different system architectures. Elapsed time are still comparable to the ones obtained by other studies; see, e.g., Fantazzini et al. [6] and Brutti et al. [7]. Moreover, a single input ROI selection will hopefully favor learnability and widen the user spectrum.

4. Discussion

In this study, a DL-based algorithm to segment and reconstruct the abdominal aorta has been proposed. Validation has been focused on the model’s performance, robustness, reliability, and usability based on the network’s metrics and reference measurements for EVAR planning taken by a vascular surgeon. The goal was to lay the groundwork for the development of more complex enhanced diagnostic techniques and preoperative planning, which was used by medical personnel without requiring prior training. The result’s reproducibility highly reduces the risk of accidental errors, thus enhancing the overall reliability of the algorithm. Providing a complex validation pipeline for aneurysmatic aortic lumen 3D reconstruction, the proposed replicable model is considered a starting point toward an enhanced 3D aortic diagnostic framework.

Despite some fluctuations in metrics, the network showed a good approximation of the aortic lumen, with sufficient results reproducibility and reliable 3D-based aortic measurements, even considering the uncertainty due to user input replicability. Segmentation performance analysis reports an average intersection over union and F1-score of 0.760 ± 0.150 and 0.850 ± 0.120, respectively, as well as an absolute error on aortic measurements provided and average value of ${\mathsf{ϵ}}_{\mathrm{a},\mathsf{\varphi }}=1.73 \mathrm{m}\mathrm{m}$, and ${\mathsf{ϵ}}_{\mathrm{a},\mathrm{L}}=2.27 \mathrm{m}\mathrm{m}$, respectively. Some cases are characterized by high across-slice standard deviation. However, it is crucial to bear in mind that GT extraction from CTA is performed using the ITK-Snap semi-automatic threshold segmentation mode. By proceeding this way, in certain cases, such an extraction method might exclude border pixels, which belong to the aortic lumen. This leads to an increase in false negatives, affecting the network’s performance and measurement process. The mentioned behavior can be observed in many cases: for example, cases ID_2, ID_6, ID_12, and ID_13. Moreover, a large spectrum of anatomies can determine a general decrease in test set metrics. Despite these limitations, 3D reconstructions still demonstrated good agreement with the ones obtained using the reference software.

Although this study has been conducted on a small test set, the results are overall in accordance with previous studies reporting AI-driven infra-renal abdominal aortic segmentation and automatic measurements of lumen diameter [22,23] whilst also including various anatomies and aorto-iliac pathologies. Although one may consider the average relative error high, as it exceeds the standard 5% value for the majority of cases, it has to be underlined that in the definition of relative error, the magnitude of such quantity is strongly influenced by the GT measure’s value order of magnitude. In fact, considering Equation (5), it can be seen how by keeping constant absolute errors, the value of the relative error will increase if referring to small distances, such as diameters, which lay in the ${10}^1$ order of magnitude. Thus, to perform a fair evaluation of the dispersion between U-Net and ITK-Snap 3D measurements, this important property of the relative error’s definition must be considered, which allows the relative error uncertainty interval to be larger, as a small order of magnitude of the measure is considered. On the other hand, such behavior is negligible for the aortic section’s length, as the order of magnitude lays around ${{10}^1-10}^2$ mm, as most relative errors on the aortic section’s length lay inside the 5% value.

To improve the model’s performance and reconstruction quality, the dataset’s anatomical spectrum needs to be enriched to include both wider areas and secondary vessels and particularly complex anatomies, which return unstable 3D reconstructions.

The network’s performance on secondary vessels segmentation also needs to be improved in the model’s next versions by expanding aortic lumen labeling ROI so as to include wider semantic areas. High standard deviations shown by metrics across slices can be drastically reduced by performing fine tuning on a specific subset of the training dataset which contains challenging anatomies.

The urge to obtain a model able to segment challenging data inevitably leads to taking into consideration a multi-center study. Comprising CTA scans from several institutes might boost network performance with larger and semantically richer datasets.

Future developments will involve the implementation of multi-class CNN for aortic lumen and intraluminal thrombus semantic segmentation to extend clinical usability to post-EVAR CTA scans. Intraluminal thrombus is, in fact, clinically crucial when estimating the outer maximum aortic diameter. Further steps also involve coupling a custom CFD solver to the AI model for the numerical simulation of patient-specific aortic flows which can further represent the true hemodynamic state of the diseased vessel. Quantitative hemodynamic data correlation with geometric vessel properties will enhance simulation accuracy, especially if laid on a knowledge-based semantic structure such as taxonomy or ontology, favoring digital translatability to patient-specific diagnostics addressing complex problems such as rupture risk assessment.