Neural Vessel Segmentation and Gaussian Splatting for 3D Reconstruction of Cerebral Angiography

Abstract

Cerebrovascular diseases are a leading cause of global mortality, underscoring the need for objective and quantitative 3D visualization of cerebral vasculature from dynamic imaging modalities. Conventional analysis is often labor-intensive, subjective, and prone to errors due to image noise and subtraction artifacts. This study tackles the challenge of achieving fast and accurate volumetric reconstruction from angiography sequences. We propose a multi-stage pipeline that begins with image restoration to enhance input quality, followed by neural segmentation to extract vascular structures. Camera poses and sparse geometry are estimated through Structure-from-Motion, and these reconstructions are refined by leveraging the segmentation maps to isolate vessel-specific features. The resulting data are then used to initialize and optimize a 3D Gaussian Splatting model, enabling anatomically precise representation of cerebral vasculature. The integration of deep neural segmentation priors with explicit geometric initialization yields highly detailed 3D reconstructions of cerebral angiography. The resulting models leverage the computational efficiency of 3D Gaussian Splatting, achieving near-real-time rendering performance competitive with state-of-the-art reconstruction methods. The segmentation of brain vessels using nnU-Net and our trained model achieved an accuracy of 84.21%, highlighting the improvement in the performance of the proposed approach. Overall, our pipeline significantly improves both the efficiency and accuracy of volumetric cerebral vasculature reconstruction, providing a robust foundation for quantitative clinical analysis and enhanced guidance during endovascular procedures.

1. Introduction

Cerebrovascular diseases, including conditions such as stenosis and aneurysms, remain a leading cause of global mortality and long-term disability, making accurate and dynamic visualization of the cerebral vasculature essential for diagnosis, procedural navigation, therapeutic decision-making, and treatment evaluation. X-ray Digital Subtraction Angiography (DSA) serves as the primary imaging modality for this purpose, providing sequential 2D projections that capture the temporal progression of contrast flow.

Within this context, the goal of our work is to support clinicians in more efficiently identifying cerebral venous pathologies and enable the collected angiographic data to be used for training neural networks capable of automatic pathology detection. By integrating 3D Gaussian Splatting into the reconstruction pipeline, we further enhance the spatial continuity and structural detail of vascular representations, leading to more informative and clinically useful visualizations that improve both diagnostic accuracy and downstream computational analysis.

During our review of existing methods, we did not identify any prior systems that explicitly combine flow information and segmentation to generate 3D head vascular reconstructions from angiographic DSA images within a unified pipeline. While some approaches focus on vessel segmentation or on the analysis of blood-flow patterns, none integrate these components into a complete framework capable of reconstructing a true 3D point cloud from standard 2D DSA sequences and representing it using a Gaussian rendering-based representation. This absence of comparable solutions was one of the primary motivations for developing the proposed method.

1.1. Neural Vessel Segmentation and 3D Reconstruction

In recent years, deep learning methods have substantially advanced vessel segmentation in cerebrovascular imaging. Convolutional and transformer-based architectures have enabled automated delineation of vascular structures from modalities such as MRI, CTA, and DSA [1,2]. Networks such as DeepVesselNet [3] and its variants exploit spatial continuity and multi-scale context to segment complex vascular trees while preserving topology. More recently, multi-sequence vessel-wall imaging systems such as VWI Assistant [4] have demonstrated fully automatic segmentation and volumetric reconstruction of intracranial arteries, supporting large-scale clinical analysis.

Beyond segmentation, several works address the reconstruction of patient-specific 3D vascular geometry from limited 2D angiographic views. Frisken et al. [5] proposed methods leveraging temporal and structural priors to recover detailed 3D vessel networks from rotational or biplanar angiography. However, these classical pipelines often require significant manual intervention and are sensitive to noise, motion, and subtraction artifacts.

1.2. Neural Rendering and Gaussian Splatting

Feature-EndoGaussian [6] incorporates segmentation-derived semantic priors into the Gaussian optimization process approaches particularly relevant for medical and endoscopic reconstruction tasks. Recent advances in neural scene representation have introduced the paradigm of 3D Gaussian Splatting (3DGS) [7], which replaces dense volumetric sampling with an explicit, differentiable set of 3D Gaussians optimized for color, opacity, and covariance. Compared with Neural Radiance Fields (NeRF), 3DGS achieves near-real-time rendering performance while maintaining photorealistic quality [8]. To address the time-varying nature of dynamic imaging, TOGS (Temporal Opacity Gaussian Splatting) [9] introduces per-Gaussian temporal opacity offsets for reconstructing 4D angiographic sequences [10,11,12,13,14,15,16].

Subsequent developments, including Deformable3DGS and 4D-RotorGS, extended the method to dynamic scenes with motion and lighting variations [17,18]. In the broader medical domain, MedGS [19] adapts Gaussian Splatting to multi-modal imaging (CT, MRI, ultrasound), providing noise-robust volumetric interpolation and efficient surface representation.

1.3. Current Challenges and Research Gap

Despite these advances, applying Gaussian-based reconstruction to cerebral angiography remains in its infancy. DSA data present unique challenges: sparse view angles, contrast-agent dynamics, and strong background-subtraction artifacts. Most Gaussian-splatting methods assume static, Lambertian scenes and are not tailored to the non-rigid, temporally varying vascular structures seen in DSA. Furthermore, segmentation priors although powerful have rarely been integrated into the Gaussian optimization pipeline for vessel-specific geometry reconstruction. Bridging this gap motivates the present study, which seeks to combine neural segmentation with Gaussian Splatting for efficient, anatomically precise 3D reconstruction of cerebral vasculature from angiographic sequences.

Limitations in Artery–Vein Segmentation and 3D Reconstruction in DSA

While DSA offers crucial insights into cerebral blood flow dynamics, its interpretation traditionally depends on manual visual assessment by neuroradiologists. This process is subjective, labor-intensive, and prone to error, further challenged by noise, patient motion, and subtraction artifacts that may mimic vascular structures. Automatic segmentation of arteries and veins has the potential to overcome these limitations, enabling applications such as venous roadmaps for transvenous procedures and quantitative extraction of flow-related biomarkers [20,21,22,23,24,25,26,27,28,29,30,31,32,33,34]. However, robust segmentation requires integrating both spatial and temporal dynamics of contrast flow, as methods relying purely on spatial features are vulnerable to false positives from artifacts [35,36,37,38,39].

Achieving accurate volumetric 3D reconstruction of the vascular tree represents the next critical step toward objective quantitative analysis and near-real-time surgical guidance [40,41,42,43]. Traditional methods for scene reconstruction and novel view synthesis, such as Neural Radiance Fields, have demonstrated high-quality reconstructions but are hindered by computational complexity, slow training, and limited low-latency rendering performance. These challenges are particularly pronounced in angiography, where thin and intricate vascular structures demand both high spatial fidelity and computational efficiency [44,45,46].

The emergence of 3D Gaussian Splatting has introduced a powerful alternative, providing explicit, unstructured scene representations with high fidelity, rapid training, and low-latency rendering capabilities. Conventional 3DGS pipelines, however, depend on sparse point clouds and camera poses generated by Structure-from-Motion (SfM) algorithms such as COLMAP. When applied to medical angiography data, standard SfM often incorporates background features, noise, and artifacts, compromising the accuracy of vascular reconstructions [47,48,49].

To address these challenges, we propose an integrated, segmentation-guided reconstruction pipeline. This approach combines: (1) noise reduction and image refinement using tools like BM3D or Real-ESRGAN; (2) a neural segmentation model, trained extensively to achieve precise vessel delineation; and (3) a segmentation-informed initialization of 3DGS via Splatfacto [50,51,52]. By constraining the sparse point cloud generated by SfM with segmentation masks, our method ensures that the geometric initialization is focused exclusively on vascular structures, thereby enhancing reconstruction fidelity.

The main contributions of this work are as follows:

• Development of a practical preprocessing pipeline that applies noise reduction to mitigate artifacts in clinical DSA image sequences;
• Deployment of a specialized nnU-Net-based vessel segmentation model, to achieve high-precision vascular segmentation from angiographic DSA data;
• Adaptation of DSA image sequences for SIFT through tailored camera modeling and parameter tuning, enabling reliable feature extraction and camera pose estimation;
• Introduction of a segmentation-guided Structure-from-Motion pipeline that filters COLMAP-derived point clouds to retain only vascular structures;
• Demonstration of the feasibility and efficiency of integrating 3D Gaussian Splatting with segmentation-derived geometric priors to generate anatomically precise 3D reconstructions of cerebral vasculature.

The remainder of this article is organized as follows: Section 2 describes the materials and methods, including noise reducers like Real-ESRGAN or BM3D methods of preprocessing, neural segmentation, segmentation-guided COLMAP, and the Splatfacto pipeline. Section 3 and Section 4 present the proposed methods and experimental results, including the validation process. Section 5 discusses clinical implications and future research directions, and Section 6 provides concluding remarks.

2. Materials and Methods

In this study, we developed a pipeline for robust 3D reconstruction of cerebral vessels from clinical DSA sequences. The workflow integrates image restoration, deep learning-based vessel segmentation, and volumetric reconstruction to overcome challenges such as noise, motion artifacts, and low contrast that commonly degrade DSA quality.

2.1. Data Acquisition and Preprocessing

The primary data source consisted of clinical DSA series, typically presented as 2D+time sequences capturing contrast flow through cerebral blood vessels. DSA images are often degraded by noise, motion blur, and subtraction artifacts. To mitigate these issues and ensure high-quality input for subsequent neural segmentation, an image restoration step was applied.

2.2. Image Restoration (Noise Removal)

Each DSA image frame was initially processed using a practical image restoration method (Real-ESRGAN) to reduce high-frequency noise. While this tool was used in our preliminary experiments, in the current research we strongly recommend primarily use BM3D, a non-generative denoising algorithm that provides better results for vascular imaging tasks without relying on a neural network. These improvements were confirmed in subsequent stages of our pipeline.

BM3D operates by grouping similar image patches and collaboratively filtering them in a transform domain, effectively removing noise while preserving fine vessel structures. This denoising step generates cleaner images, minimizing the risk of misclassification by the segmentation network and improving feature matching in subsequent 3D reconstruction. Mathematically, BM3D can be represented as a patch-based collaborative filtering operation, which suppresses noise while retaining important structural details. The enhanced quality of these images facilitates accurate detection of vascular structures and provides reliable input for downstream multi-view reconstruction and segmentation pipelines.

2.3. Vessel Segmentation Using Neural Network

Utilizing the powerful deep learning framework of the nnU-Net architecture, we trained our model on the available dataset to achieve accurate vessel segmentation results. The network, based on the advanced and self-configuring nnU-Net model (widely recognized for its strong performance in biomedical image segmentation).

2.3.1. U-Net: Network Architecture and Suitability for Biomedical Image Segmentation

The U-Net architecture was originally developed for biomedical image segmentation, where precise pixel-level classification is required despite limited annotated data. Since its introduction, it has become one of the most widely used and influential models in medical image segmentation due to its efficiency, robustness, and strong performance across diverse modalities [53].

Although our work primarily relies on nnU-Net, the classical U-Net architecture remains essential for contextualizing our comparisons. Because both nnU-Net and CAVE U-Net are ultimately based on the U-Net design, understanding the underlying U-Net principles is important for interpreting segmentation quality, evaluating differences between models, and comparing automatic predictions with manual reference masks. This architectural foundation allows for a clearer assessment of how each model improves upon or deviates from the original U-Net framework.

U-Net follows a characteristic U-shaped architecture, consisting of two main components: a contracting path (encoder) and an expansive path (decoder).

• Contracting Path (Encoder): This path is responsible for capturing contextual information. It repeatedly applies two $3\times 3$ convolutions, each followed by a ReLU activation, and then performs a $2\times 2$ max pooling operation with stride 2 for downsampling. With each downsampling step, the number of feature channels doubles, allowing the network to learn increasingly abstract representations.
• Expansive Path (Decoder): This symmetric path focuses on precise localization. Each step consists of an upsampling operation followed by a $2\times 2$ convolution (often called an up-convolution) that halves the number of feature channels. Crucially, high-resolution feature maps from the corresponding layer of the contracting path are concatenated with the upsampled output via skip connections. This fusion restores fine-grained spatial details lost during downsampling.

The final layer applies a $1\times 1$ convolution to map the resulting feature vectors to the desired number of output classes, effectively generating the segmented output.

Why U-Net Is Effective for Medical Imaging:

U-Net demonstrates exceptional performance on medical and biomedical segmentation tasks due to several key properties:

• Data Efficiency and Robustness: Medical imaging datasets often contain limited labeled samples. U-Net compensates through extensive use of data augmentation, especially random elastic deformations, which help the model learn invariance to tissue deformations common in microscopy and radiology images.
• Separation of Touching Structures: The network employs a weighted loss function that assigns higher penalties to boundaries between adjacent objects. This mechanism enables the accurate separation of touching cells or tissues belonging to the same class.
• Speed and Practicality: Despite its depth and accuracy, U-Net is computationally efficient. On modern GPUs, it can segment a $512\times 512$ image in under a second, making it practical for real-time or large-scale medical analyses.

The U-Net architecture, with its symmetric encoder–decoder design, skip connections for preserving spatial detail, and efficient training strategies, has proven exceptionally effective for biomedical image segmentation. It enables precise delineation of complex anatomical and cellular structures while remaining computationally efficient qualities that have made it a cornerstone in medical image analysis research.

Building on this foundation, nnU-Net takes the strengths of U-Net even further. It retains the familiar architecture but introduces an automated framework that dynamically adapts to different datasets and imaging modalities. Rather than requiring manual fine-tuning, nnU-Net automatically configures preprocessing, network architecture parameters, and training settings to best suit the data at hand. This adaptability makes nnU-Net not just a model, but a robust and generalizable system for medical image segmentation.

2.3.2. Description of Creating Segmentation Model

For training the nnU-Net model, we employed a merged dataset composed of two sources: a proprietary dataset from the Slovak Ambulance Clinic, derived from real patient DSA examinations, and an additional DIAS: DSA - dataset commonly used in neurovascular studies. This combination provided sufficient variability in patient anatomy and imaging conditions, enabling robust model training.

All images were standardized in size and preprocessed appropriately to accommodate the nnU-Net framework. Training hyperparameters, including the number of epochs, were carefully adjusted to ensure optimal performance. This preparation allowed the nnU-Net architecture to effectively handle the merged 2D dataset, producing accurate vessel segmentation maps.

2.3.3. Dataset Description

The training and evaluation process utilized datasets from two individual patients (Figure 1), each comprising 122 angiographic images. For training the nnU-Net model, we used the first patient dataset with fully annotated images, which were manually segmented by our team. This dataset was split into training and validation subsets and served as the primary source for model development. In total, the full training dataset contained 273 images, including DSA images from publicly available datasets and images from the first patient. A separate portion of the data, fully processed and anonymized, was reserved as an independent test case to ensure unbiased performance evaluation.

Due to technical limitations in preparing the input data for segmentation, only 20% of the second patient’s dataset was annotated, which was insufficient for training but adequate for testing. These images remained suitable for comparative evaluation and were primarily used as the main test case. To enable a fair comparison with the automatic nnU-Net output, one angiographic frame from the second patient was manually segmented and used as a reference mask. The same process was applied to the 20% subset to compute the confusion matrix for quantitative evaluation.

The manual segmentation process was conducted in collaboration with experts from the Faculty of Electrical Engineering and Information Technology (FEIT) at the University of Žilina (UNIZA, Slovakia), who work on related topics and maintain regular contact with neurosurgeons. This collaboration ensured a consistent segmentation approach and enhanced the quality of the reference masks.

2.4. Computational Environment

All experiments described in this study were conducted on a PC running Windows 11 with WSL (Ubuntu) support. The hardware consisted of an Intel Core i5-12400F CPU and an NVIDIA GeForce RTX 3060 GPU, providing a robust platform for high-performance experiments.

For each experiment, a dedicated container with a separate Conda Python environment was used. This allowed isolation of dependencies for different frameworks, including Gaussian Splatting from Nerfstudio. Importantly, the entire experimental stack was fully open-source, utilizing Python 3.11 with cuda 11.8, COLMAP v3.14.0, and nnU-Net v2, with each framework configured in its own environment as specified on their respective documentation pages.

This setup ensured reproducibility, modularity, and efficient resource management across all stages of the pipeline.

3. Methodology

This section describes the proposed methodology for 3D reconstruction of cerebral vessels from DSA sequences. The approach combines resolution normalization, noise reduction, neural vessel segmentation, and segmentation-guided geometric reconstruction to generate accurate and visually coherent 3D vascular models.

3.1. 3D-VessGaussReconstructor: Description of Pipeline

We created a workflow called 3D-VessGaussReconstructor, illustrated in Figure 2, which is designed for precise 3D reconstruction of cerebral vasculature from DSA image sequences. The pipeline integrates noise reduction, vessel segmentation, and segmentation-informed volumetric reconstruction to achieve anatomically precise vascular models suitable for visualization and analysis.

The workflow starts with a resolution normalization step, where all input DSA images are resized to $1024\times 1024$ pixels. This step is required because the segmentation network was trained on data with a fixed spatial resolution. Resolution normalization ensures consistent input characteristics and stable segmentation performance across different acquisitions.

After normalization, a noise reduction stage is applied to improve vessel visibility and reduce imaging artifacts. Several denoising strategies can be used at this stage, depending on the data quality:

• Classical denoising methods: such as BM3D or Non-Local Means (NLM);
• Deep learning-based denoising: for example, DnCNN;
• Super-resolution combined with denoising: such as Real-ESRGAN.

BM3D is chosen as the primary denoising method because it was originally developed for scientific and medical imaging and demonstrates effective noise suppression while preserving fine structural details (Figure 3). However, this preprocessing step is optional and highly dependent on the quality of the input data. For high-quality DSA sequences with good vessel contrast, denoising may be reduced or omitted entirely. Super-resolution methods such as Real-ESRGAN can also be effective for noise reduction, but they may sometimes suppress or alter small vessel structures, affecting anatomical fidelity.

Following noise reduction, the preprocessed images are passed to the segmentation module. This process leverages both spatial vascular appearance and flow dynamics to reduce false positives caused by subtraction artifacts, static instruments, or image noise.

3.2. Segmentation

Normalized and denoised images are then sent to the segmentation network. While multiple segmentation models could be used, we implemented a nnU-Net model trained on our own dataset, as no publicly available model exists for human cerebral vessels. The images are converted to NIfTI format for processing by the network. The trained network generates 2D vessel segmentation maps, which are then converted back to PNG format. Both the segmentation masks and the segmented images are critical: the masks guide downstream SIFT-based reconstruction, while the segmented images provide input for Splatfacto during volumetric reconstruction. Segmentation is therefore a required step and forms the backbone of the entire 3D reconstruction pipeline.

3.3. Camera Pose Estimation (SIFT and SfM)

The same images used for segmentation are also sent to a substep where camera parameters are estimated. Since DSA images lack meta-information about the camera, this step is essential for robust 3D reconstruction. Correctly configured camera models ensure the highest number of detectable keypoints, which is critical for accurate Structure-from-Motion reconstruction. In preliminary experiments, default camera models and the pinhole camera model failed to generate sufficient keypoints. In contrast, the Simple_RADIAL camera model successfully produced usable keypoints for all 122 images in our dataset. We recommend using the automatic reconstruction option in COLMAP for optimal results.

3.4. Segmentation-Guided SfM

Using the SIFT keypoints and segmentation masks, COLMAP produces a sparse point cloud and refined camera positions. The segmentation masks are used to filter the point cloud, retaining only points corresponding to cerebral vessels. This segmentation-guided sparse point cloud provides a clean geometric prior for the next stage of volumetric reconstruction.

3.5. 3D Gaussian Splatting Reconstruction and Optimization (Splatfacto)

The final anatomically precise 3D model was reconstructed using 3DGS, implemented via Splatfacto within the Nerfstudio framework. The filtered sparse point cloud, obtained from the semantically constrained SfM output, serves as the initialization for the 3DGS model.

Each Gaussian element is initialized with:

• Positions ($\mathbf{p}$): set from the segmentation-guided sparse point cloud,
• Attributes: opacity ($\alpha$), anisotropic covariance ($\Sigma$), and Spherical Harmonic (SH) color coefficients (c).

In our methodology, the covariance matrix of each Gaussian is computed based on the local distribution of points in the filtered SfM point cloud. Specifically, for a local neighborhood of 3D points around the Gaussian center, the covariance matrix can be initialized using the following pseudocode for Gaussian initialization:

• Compute mean: $\mathit{\mu }=\mathrm{mean}\left(\mathcal{N}\right)$, where $\mathit{\mu }$ is the local center of nearby points $\mathcal{N}$.
• Compute covariance: $\mathbf{C}=\mathrm{cov}\left(\mathcal{N}\right)$, representing local 3D point distribution. This 3 × 3 matrix captures the local point distribution and is used to perform PCA.
• PCA: compute eigenvectors ${\mathbf{e}}_1,{\mathbf{e}}_2,{\mathbf{e}}_3$ and eigenvalues ${\lambda }_1\ge {\lambda }_2\ge {\lambda }_3$, ${\mathbf{e}}_1$ → along vessel, ${\mathbf{e}}_2,{\mathbf{e}}_3$ → across vessel.
• Set Gaussian covariance:

  $$\mathbf{\Sigma }=\mathbf{E}\mathrm{diag}({\lambda }_1,{\lambda }_2,{\lambda }_3){\mathbf{E}}^{\top },\mathbf{E}=\left[{\mathbf{e}}_1{\mathbf{e}}_2{\mathbf{e}}_3\right]$$

  where $\mathbf{\Sigma }\in {\mathbb{R}}^{3\times 3}$ is the anisotropic covariance of the Gaussian. The transpose ${\mathbf{E}}^{\top }$ rotates the diagonal covariance back to the original 3D coordinate frame.

This initialization produces anisotropic ellipsoids elongated along the vessel and compressed across it, accurately representing thin and elongated vascular branches, preserving the smoothness of curved segments, and enhancing the detail of small anatomical structures.

Finally, Splatfacto renders these Gaussians into the 3D scene by projecting and blending them in image space. This process allows explicit control over which image regions contribute to the volumetric model, further improving anatomical fidelity and the level of detail in the reconstructed 3D vascular network.

3.6. Post-Processing: k-Nearest Neighbor Filtering

Finally, irrelevant vertices far from the main vascular structures are removed using a k-nearest neighbor filtering method. This eliminates extraneous points while preserving the integrity of the 3D vessel reconstruction.

3.7. Summary of the Reconstruction Pipeline

In summary, the preprocessed DSA sequence is passed to a neural segmentation model, generating 2D binary vessel masks. These masks guide a segmentation-informed SfM reconstruction, where COLMAP-derived point clouds are filtered to retain only vascular structures. The filtered point clouds are then integrated with 3D Gaussian Splatting using Splatfacto, enabling anatomically precise, near-real-time 3D reconstruction of cerebral angiography. This pipeline demonstrates a reliable and efficient method for precise 3D modeling of cerebral vasculature from DSA sequences.

4. Results

4.1. Comparative Analysis of Segmentation Performance

We benchmarked our proposed nnU-Net-based vein segmentation model against the U-Net CAVE baseline network [12], originally developed for cerebral artery–vein segmentation in digital subtraction angiography. In this work, we deliberately chose U-Net and nnU-Net as baseline methods for comparison, as these architectures were specifically developed and optimized for medical segmentation. The CAVE U-Net segmentation method was published in 2023, while nnU-Net remains a state-of-the-art framework that automatically adapts to medical data and demonstrates high reliability in clinical applications. Consequently, the models included in our study already cover contemporary and relevant approaches to vascular segmentation.

4.1.1. Computational Efficiency

Average segmentation time for a single DSA image using the nnU-Net model was approximately 2–3 s, while processing an entire DSA sequence to generate segmentation masks required roughly 1–2 min. The full 3D reconstruction pipeline, including segmentation, SfM, and Gaussian Splatting, takes approximately 15–20 min. Notably, an initial visualization of the Gaussian Splatting reconstruction is available after about 5 min, following segmentation and SfM processing. Regarding memory usage, GPU and system memory consumption remained moderate throughout training and inference. During continued training of the nnU-Net model, even with 32 GB of system RAM, memory usage was not a limiting factor due to the nnU-Net architecture’s adaptive patching and multi-resolution strategy, which efficiently handles large 2D image datasets. These findings indicate that the pipeline is computationally feasible for practical use and supports low-latency visualization and 3D reconstruction of vascular structures from DSA sequences.

4.1.2. Context and Limitations of Prior Work

The U-Net (CAVE) architecture, as employed in prior work, was designed to capture spatial features in vascular segmentation. Previous studies reported an overall vessel Dice score of $0.84\pm 0.04$ and a vein Dice (V-Dice) of $0.76\pm 0.054$, but several limitations were noted: venous segmentation performance remained consistently lower than arterial segmentation due to reduced vessel contrast and ambiguous vein boundaries; U-Net-based baselines were prone to false positives caused by subtraction artifacts and static instruments; and temporal modeling modules provided no statistically significant improvement over one another in practice [12].

These findings indicate that, although U-Net provides a strong baseline for medical segmentation, its performance is sensitive to dataset heterogeneity, artifacts, and domain-specific acquisition conditions. Additionally, approaches relying on sequential DSA inputs are less adaptable to static or partially incomplete sequences commonly encountered in clinical workflows.

4.1.3. Our Model Performance

Our nnUNet-based vein segmentation model achieved a peak pseudo-Dice coefficient of 0.8391 on the validation set, exceeding the performance of both the conventional U-Net baseline ($0.63\pm 0.07$) and the U-Net model from the CAVE study ($0.76\pm 0.054$) reported in the same segmentation context. Importantly, this performance was attained without temporal recurrence or transformer modules, underscoring the efficiency of the nnUNet framework’s self-configuring architecture and adaptive training scheme.

This demonstrates that our trained model, trained for 250 epochs (see Figure 4) on a composite dataset combining publicly available DSA data with proprietary clinical datasets, achieves substantially better results.

4.1.4. Interpretation

The superior performance of our nnUNet variant can be attributed to its:

• Dynamic architecture adaptation: nnUNet automatically optimizes preprocessing, patch size, and normalization per dataset, enabling better generalization than fixed architectures used in CAVE.
• Robustness to DSA variability: Unlike CAVE’s dependency on full temporal sequences, our model efficiently segments veins from single-frame or minimal temporal data, making it clinically versatile.
• Enhanced optimization and loss design: The nnUNet pipeline’s combined Dice–cross entropy loss and extensive augmentation strategy provide improved boundary sensitivity and artifact suppression.

Overall, our results demonstrate that an optimized nnUNet configuration can surpass both the baseline U-Net and the state-of-the-art U-Net CAVE network in vein segmentation, despite operating in a more constrained data regime. This highlights the capacity of the nnUNet framework to efficiently capture vascular morphology and contrast-flow patterns without explicit temporal modeling.

4.2. Cross-Dataset Considerations and Practical Evaluation

Because the experiments were conducted on different datasets, a direct comparison between neural network architectures is not entirely feasible. However, this was not the primary objective of our study. The main focus was to assess how well the segmentation output from each model performs in a practical workflow, particularly within our 3D vascular reconstruction pipeline. In this context, the segmentation accuracy is most valuable when it translates into coherent and anatomically plausible vessel maps suitable for volumetric reconstruction.

Therefore, in addition to quantitative metrics, we conducted a qualitative comparison between the segmentation maps produced by our nnUNet and those from the U-Net model described in the CAVE study. Each prediction was visually evaluated against expert-annotated reference segmentations to ensure that the resulting maps accurately captured the venous and arterial structures essential for 3D modeling.

4.3. Strict Pixel-Wise Mask Comparison

To evaluate the quality of predicted segmentation masks, we use a strict comparison metric that focuses on meaningful regions. This method ensures that predictions are only rewarded when they match the ground truth (reference), and penalized when they detect objects in background areas. The core evaluation logic is implemented in Python as follows:

Let:

• $R_{i,j}$ be the pixel value at position $(i,j)$ in the reference mask,
• $P_{i,j}$ be the pixel value at position $(i,j)$ in the predicted mask,
• $1[·]$ be the indicator function, which equals 1 if the condition is true, and 0 otherwise.

Then:

$$\begin{array}{cc}\hfill \mathrm{TP}& =\sum _{i,j}1[R_{i,j}\ne 0\wedge R_{i,j}=P_{i,j}]\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill \mathrm{FP}& =\sum _{i,j}1[R_{i,j}=0\wedge P_{i,j}\ne 0]\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill \mathrm{Total}\mathrm{Relevant}& =\sum _{i,j}1[R_{i,j}\ne 0]+\mathrm{FP}\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill \mathrm{Accuracy}& =\left\{\begin{array}{cc}{\displaystyle \frac{\mathrm{TP}}{\mathrm{Total}\mathrm{Relevant}}}\times 100,\hfill & \mathrm{if}\mathrm{Total}\mathrm{Relevant}>0\hfill \\ 0,\hfill & \mathrm{otherwise}\hfill \end{array}\right.\hfill \end{array}$$

(4)

• True Positives (TP): Pixels where the reference mask contains an object (non-zero), and the prediction matches it exactly.
• False Positives (FP): Pixels where the reference is background (zero), but the prediction mistakenly labels something.

This design avoids artificially inflated accuracy by excluding background agreement and ensures that over-segmentation is penalized. Figure 5 presents a visual comparison between the reference mask and two predicted masks. Correct predictions align with the reference, while extra or missing areas indicate false positives or false negatives respectively. The segmentation performance evaluation of vessel masks is shown in

Table 1. Segmentation performance evaluation of vessel masks.

Mask	Correct (TP)	Wrong Extra (FP)	Total Considered	Accuracy (%)
Mask1	27,292	20,558	73,740	37.01
Mask2	52,962	9709	62,891	84.21

.

Mask 2 achieves better results because it was generated by a model trained on our own real-world clinical data, which were specifically prepared and annotated solely for vessel detection (Figure 6). This targeted training enabled the model to focus exclusively on vascular structures, thereby reducing false detections of background or irrelevant anatomy. In contrast, Mask 1 can be reasonably assumed to have been produced using a model trained on less specialized data, which led to increased over-segmentation and a higher number of false positives. We want to clarify that the comparison is between model predictions, not architectures.

Confusion Matrix-Based Quantitative Evaluation

To obtain a standardized and objective quantitative evaluation of segmentation performance, a confusion matrix-based analysis was conducted using the annotated 20% subset of the second patient’s dataset. This subset was sufficient for reliable testing while remaining fully independent from the training process.

Binary vessel masks generated by the nnU-Net model were quantitatively evaluated against manually annotated reference masks using a pixel-wise comparison across all selected images. The aggregation of true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN) over the entire dataset yielded the global confusion matrix reported in

Table 2. Pixel-wise confusion matrix for nnU-Net vessel segmentation.

	Reference Vessel	Reference Background
Predicted Vessel	15,798,790 (TP)	779,523 (FP)
Predicted Background	108,865 (FN)	3,497,910 (TN)

.

The large number of true positive pixels (15,798,790) indicates that the model successfully detected the majority of vessel structures present in the ground truth annotations. At the same time, the relatively low number of false negatives (108,865) suggests that only a small fraction of vessel pixels were missed, reflecting strong sensitivity to vascular structures. However, false positives amounted to 779,523 pixels, indicating that some background regions were incorrectly classified as vessels. While this over-segmentation contributes to a reduction in precision, its magnitude remains limited when compared to the total number of correctly classified vessel pixels. The high count of true negatives (3,497,910) further demonstrates the model’s ability to correctly identify background regions, supporting robust discrimination between vessel and non-vessel pixels.

Based on this confusion matrix, several standard segmentation metrics were computed to provide a comprehensive assessment of performance. These metrics are summarized in

Table 3. Segmentation performance metrics computed from the global confusion matrix.

Metric	Value
Accuracy	0.955988
F1-score	0.887321
Dice coefficient	0.887321
Matthews correlation coefficient (MCC)	0.865049

. These metrics confirm strong segmentation performance.

The achieved accuracy of 0.955988 reflects a high overall proportion of correctly classified pixels across both vessel and background classes. However, given the inherent class imbalance typical of vessel segmentation tasks where background pixels dominate accuracy alone is not sufficient to fully characterize segmentation quality. The F1-score and Dice coefficient, both equal to 0.887321, indicate a high degree of overlap between the predicted vessel masks and the ground truth annotations. These values demonstrate a well-balanced trade-off between precision and recall, suggesting that the model effectively captures vessel structures while limiting false detections. The Matthews correlation coefficient (MCC) of 0.865049 further supports this conclusion, as MCC provides a balanced evaluation even in the presence of class imbalance and reflects a strong overall correlation between predictions and reference labels. These metrics indicate strong agreement between the predicted vessel masks and the ground truth annotations, with balanced precision and recall and a high overall correlation.

Overall, these results indicate that the nnU-Net model achieves accurate and reliable vessel segmentation, with strong agreement between predicted masks and manual annotations. The combination of high overlap-based metrics and a robust MCC value suggests that the model generalizes well across images and provides a dependable basis for downstream quantitative vascular analysis. Importantly, the accuracy obtained from this confusion matrix-based evaluation is consistent with the accuracy trends reported earlier using an alternative evaluation strategy. This agreement demonstrates that both evaluation approaches yield comparable results, confirming the robustness and reliability of the proposed segmentation pipeline.

4.4. Impact of Vessel Segmentation Quality on 3D Reconstruction

The objective of this experiment was to evaluate how different vessel segmentation methods influence the accuracy and visual fidelity of 3D reconstructions derived from cerebral (DSA) images. The reconstruction pipeline consisted of several stages: contrast enhancement and denoising, vessel segmentation, Scale-Invariant Feature Transform (SIFT) feature extraction, SfM, and a final surface refinement using Gaussian Splatting (Splatfacto).

We compared the downstream 3D reconstruction performance of two segmentation approaches: (1) a U-Net model following the configuration described in the CAVE framework [12], and (2) our custom-trained nnUNet-based segmentator. The goal was to determine how segmentation fidelity translates to structural coherence and artifact suppression in the final 3D model.

4.5. Comparative Segmentation Performance and Reconstruction Fidelity

Segmentation accuracy directly influences the quality of 3D reconstruction, as errors in the vessel masks propagate through SIFT-based feature detection and subsequent SfM alignment. The baseline U-Net segmentation, which relies exclusively on spatial vasculature features extracted from Minimum Intensity Projection (MinIP) frames, achieved a vessel Dice coefficient of $0.80\pm 0.050$ according to the CAVE study [12]. However, such spatial-only models are known to introduce false positives, especially around subtraction artifacts and nonvascular structures.

Our optimized nnU-Net-based segmentation model demonstrated a higher pseudo Dice coefficient of 0.8873 (EMA: 0.8309) on the validation set, highlighting its superior ability to delineate vessel boundaries and suppress noise. First, this improved segmentation ensures that fine vascular structures are accurately captured, providing a reliable representation of the anatomical features. Second, these precise masks reduce the propagation of errors in subsequent processing steps, such as surface reconstruction and volumetric modeling. Finally, when these high-quality segmentations are used as input for 3D reconstruction, including SfM processing, the enhanced precision significantly improves the quality of the reconstructed point clouds. In the subsequent neural-network-based Gaussian Splatting pipeline, the configuration of hyperparameters becomes critical: parameters such as the learning rate, the number of steps, and those controlling the evolution of the Gaussian primitives strongly influence convergence stability and the preservation of fine structural details. It is important to note that epochs are not used in the training process of Gaussian Splatting; instead, optimization is defined in terms of steps, typically around 30,000 even in standard Gaussian Splatting workflows.

Visual inspection of the reconstructed volumes (Figure 7a,b) further confirms that the nnUNet-based segmentation delivers superior geometric consistency and vascular detail. The 3D model initialized with U-Net-derived masks exhibited residual background noise and blurred vessel contours due to misclassified regions. In contrast, the nnUNet-based reconstruction preserved the anatomical structure of cerebral veins and arteries, achieving higher visual coherence and continuity of vascular topology.

It is very important to note that, for clarity, all experiments dependent on Gaussian Splatting were organized to examine different aspects of the reconstruction process. In one of the initial experiments (Figure 8), we evaluated the performance of COLMAP using only the data from the first patient, without applying any segmentation. The results showed that such unfiltered data are difficult to use for identifying patient pathologies based on venous structures. Moreover, they are unsuitable for training a neural network intended for automatic detection of similar conditions. This highlights the practical necessity of applying segmentation-based filtering during data preparation.

The threshold was defined as:

$$\mathrm{threshold}=d_{\max }+\sigma ,$$

(5)

where $d_{\max }$ represents the most frequently occurring distance between corresponding points, $\sigma$ is the standard deviation of all measured distances.

Next, we conducted experiments to evaluate segmentation accuracy. Specifically, we compared the nnU-Net results (Figure 9) with manual segmentation (Figure 10) and CAVE U-Net against manual segmentation.

The histograms (Figure 11 and Figure 12) and the point cloud comparison in

Table 4. Segmentation performance evaluation using 3D reconstructed point clouds.

Technique	Threshold	Accuracy (%)
CAVE U-Net	0.0141	78.18
nnU-Net	0.0015	90.13

from these evaluations demonstrate that nnU-Net achieves significantly higher accuracy than CAVE U-Net. In addition to our custom accuracy metric, we employed the Cloud-to-Cloud (C2C) distance metric in CloudCompare, which provides a direct quantitative measure of geometric correspondence between the reconstructed point clouds and their manual segmentation references. The C2C metric captures local spatial deviations, reflecting the true 3D accuracy of the segmented point clouds generated by both CAVE U-Net and nnU-Net.

The resulting histograms from CloudCompare illustrate the distribution of C2C distances, showing the degree of alignment between automatic and manual segmentations. Furthermore, the point cloud comparison presents the percentage accuracy derived from the C2C evaluation, enabling an objective assessment of segmentation quality in 3D space. To further quantify the accuracy of our reconstructions, we established a threshold criterion based on both maximum distance and standard deviation. Accuracy was calculated as the ratio of points within the threshold to the total number of points in the cloud, expressed as a percentage. This approach accounts for both spatial separation and statistical variability, including natural fluctuations and potential errors from the reconstruction process.

Finally, we explored the application of 3D reconstruction using NeRF-based methods, including Instant-NGP and Nerfacto. The results (Figure 13) revealed that these approaches are not well-suited for this type of data, as they are designed to operate on color-based ray information. For highly specific data, such as human cerebral venous structures, this is problematic because the images are predominantly grayscale.

One of the key advantages of using 3D Gaussian Splatting is its independence from color contrast, as it operates on Gaussian fields rather than relying on color-rich ray inputs. Furthermore, it allows the generation of detailed structural visualizations that cannot be achieved using only traditional SfM and MVS pipelines.

Taken together, these findings clearly demonstrate that high-quality segmentation is a decisive factor in achieving accurate and artifact-free 3D reconstructions. Incorporating a self-configuring segmentation framework such as nnU-Net significantly enhances the downstream spatial precision of SfM and Gaussian Splatting processes, enabling reliable reconstruction of complex vascular geometries.

5. Discussion

The proposed integrated, segmentation-guided reconstruction pipeline successfully addresses key challenges in volumetric 3D reconstruction of vascular trees from DSA. Achieving accurate volumetric reconstruction represents a crucial step toward objective quantitative analysis and low-latency surgical guidance. Conventional neural rendering methods, such as NeRF, are often limited by computational complexity and slow training times, issues that are particularly pronounced when modeling fine, intricate vascular structures. In contrast, 3DGS, implemented via Splatfacto, enables rapid training and low-latency rendering. However, traditional 3DGS pipelines rely on SfM algorithms, such as COLMAP, which may inadvertently incorporate background features, noise, and artifacts when applied to angiographic data. These factors degrade the accuracy and clinical reliability of vascular reconstructions.

Our work primarily focuses on the process of 3D image reconstruction, although numerous other approaches exist, most of which are generally based on SfM and depth estimation techniques. In cases where other segmentation techniques were considered, we also evaluated alternative segmenters such as YOLO. However, nnU-Net is specifically designed for medical data, and, as previously mentioned, Gaussian Splatting provides high-detail processing and rendering. Therefore, 3D Gaussian Splatting and nnU-Net currently represent the key components of our proposed pipeline, ensuring high-quality segmentation and reconstruction.

In this study, the vascular points within the point clouds are used only for visualization. In practice, 3D reconstructions are produced using Gaussian Splatting, with each point represented by an anisotropic Gaussian kernel aligned to the local vessel geometry. These kernels are elongated along the vessel and compressed across it, allowing precise modeling of thin and elongated vascular branches while preserving their shape and curvature smoothness. A collection of such kernels forms a continuous volumetric model that can be rendered from any viewpoint, rather than merely displaying discrete points. Unlike classical DSA reconstruction methods, our approach requires only a single pair of 2D DSA sequences, reducing radiation exposure. Background structures are effectively removed through segmentation, allowing the pipeline to focus solely on vascular features. The use of anisotropic Gaussian kernels preserves fine vessel details, while the self-supervised decomposition between the static background and the dynamics of the contrast agent further improves the accuracy of blood flow reconstruction. As a result, our pipeline produces a comprehensive 3D representation of cerebral vessels with high spatial detail and clinical relevance, substantially surpassing the capabilities of conventional point-based or voxel-based reconstructions.

Our findings demonstrate that integrating neural priors into the reconstruction workflow substantially enhances both the geometric fidelity and the structural coherence of the resulting 3D vascular model. Although in this work we did not perform a traditional ablation study isolating each module. Instead, we evaluated the influence of different segmentation strategies to observe how variations in vascular masks affect the final 3D reconstruction. This approach allowed us to assess the stability and robustness of the entire pipeline under realistic conditions. The primary goal of this study is to demonstrate the effectiveness of the integrated pipeline as a whole, rather than to benchmark each component in isolation. Therefore, emphasis is placed on a comprehensive evaluation of the final reconstructed 3D vascular structures. The nnU-Net architecture proved particularly effective for vessel segmentation, achieving a peak Pseudo Dice coefficient of 0.8391 on the validation set significantly surpassing the conventional U-Net baseline and the CAVE U-Net model. Notably, this superior performance was achieved without explicit temporal recurrence or transformer-based modules. The improvement is attributed to nnU-Net’s adaptive training scheme and self-configuring architecture, which demonstrated robustness to variations in DSA intensity and temporal sparsity. In pixel-wise comparisons, the segmentation derived from nnU-Net (Mask 2) achieved an accuracy of 84.21%, dramatically outperforming the U-Net segmentator (Mask 1), which achieved 37.01%.

The high segmentation accuracy translated directly into improved 3D reconstruction outcomes. By constraining the SfM-derived sparse point cloud exclusively to vascular structures, the segmentation-guided initialization of 3DGS effectively filtered out background clutter and imaging artifacts. The baseline U-Net-based reconstructions retained residual background noise and produced blurred vessel contours, while the nnU-Net-based pipeline generated structurally coherent and artifact-free 3D reconstructions. The results confirm that segmentation quality is a decisive factor in achieving precise and clinically interpretable vascular models.

Future Research Directions

Building on the success of cerebral vessel reconstruction, the proposed pipeline not only enhances the accuracy and reliability of anatomical assessment thereby supporting improved surgical planning but also lays the groundwork for future extensions to other angiographic modalities, such as coronary and peripheral vasculature. These expansions would allow evaluation of the framework’s performance across varied anatomical geometries and imaging conditions. Continuous refinement of the nnU-Net segmentation model through domain-specific retraining could further improve boundary sensitivity, artifact suppression, and robustness across diverse clinical datasets.

Looking ahead, translating this framework into practical surgical applications will require converting the unstructured 3D Gaussian Splatting representation into a clean, manifold mesh suitable for simulation, quantitative analysis, and patient-specific medical device design. Such developments could also enable advanced functionalities, including automatic aneurysm detection or real-time 4D reconstruction, although these applications would demand substantial computational resources and further methodological enhancements. Together, these future directions highlight the potential of the pipeline to not only improve clinical workflows but also expand its utility for research, simulation, and personalized interventional planning.

6. Conclusions

This study presents a novel, integrated pipeline for anatomically precise, near-real-time 3D reconstruction of cerebral angiography using neural priors and 3D Gaussian Splatting. The workflow combines an image restoration tool, a customized nnU-Net segmentation model trained for 250 epochs, and a segmentation-guided Structure-from-Motion strategy. By filtering COLMAP-derived point clouds with segmentation masks, the pipeline ensures that geometric initialization for 3DGS focuses exclusively on vascular structures, effectively mitigating the limitations of conventional SfM methods in medical imaging.

The resulting 3D reconstructions exhibit superior geometric accuracy, reduced background noise, and improved vascular topology compared with baseline approaches. These findings demonstrate the feasibility and efficiency of integrating 3D Gaussian Splatting with neural segmentation priors, paving the way for objective quantitative vascular analysis and real-time surgical navigation in clinical environments.