Automatic Hepatic Vessels Segmentation Using RORPO Vessel Enhancement Filter and 3D V-Net with Variant Dice Loss Function

Abstract

The segmentation of hepatic vessels is crucial for liver surgical planning. It is also a challenging task because of its small diameter. Hepatic vessels are often captured in images of low contrast and resolution. Our research uses filter enhancement to improve their contrast, which helps with their detection and final segmentation. We have designed a specific fusion of the Ranking Orientation Responses of Path Operators (RORPO) enhancement filter with a raw image, and we have compared it with the fusion of different enhancement filters based on Hessian eigenvectors. Additionally, we have evaluated the 3D U-Net and 3D V-Net neural networks as segmentation architectures, and have selected 3D V-Net as a better segmentation architecture in combination with the vessel enhancement technique. Furthermore, to tackle the pixel imbalance between the liver (background) and vessels (foreground), we have examined several variants of the Dice Loss functions, and have selected the Weighted Dice Loss for its performance. We have used public 3D Image Reconstruction for Comparison of Algorithm Database (3D-IRCADb) dataset, in which we have manually improved upon the annotations of vessels, since the dataset has poor-quality annotations for certain patients. The experiments demonstrate that our method achieves a mean dice score of 76.2%, which outperforms other state-of-the-art techniques.

1. Introduction

Liver vessel segmentation from Computed Tomography (CT) images is important before liver surgery resection. Clinicians must know the liver’s morphology and its venous system location and diameter to plan the path of the surgical cutting. Hence, the segmentation of the small structures such as hepatic veins is a crucial task. These structures have a small diameter, and low image contrast and resolution. Today, accurate hepatic vessel labeling still relies primarily on doctors’ manual segmentation, which is time-consuming and depends on the specialists’ expertise and skills. As a result, an autonomous, robust, and accurate hepatic vascular segmentation algorithm is critical and highly desired.

In this work, we address the issues of low-contrast vessels and labour-intensive manual vessel segmentation. We develop a fully automatic segmentation method that provides high accuracy and precision for hepatic veins segmentation. The solution does not require a doctor’s assistance, such as in semi-automatic methods.

Although many vessel segmentation approaches have been investigated by other authors, such as the threshold method, region growing, and morphology-based methods, the state-of-the-art methods use Convolutional Neural Networks (CNNs) and Deep Learning (DL) strategies. We use the DL approach in our method as well. Typically, enhancement filters such as those based on the Hessian matrix are used to improve the contrast and visibility of the vascular structures. As opposed to that, our method is based on the fusion of the raw image with the RORPO (The Ranking Orientations Responses of Path Operators) [1] enhanced image. This is then utilized by the segmentation architecture of 3D U-Net or 3D V-Net. The final segmentation pipeline consists of RORPO and 3D V-Net, which provide the best segmentation results.

In the paper, several experiments are performed, which test the main parts of the proposed method and make comparisons with alternative state-of-the-art approaches. The first experiment compares five classical vessel enhancement filters separately: Frangi, Hessian, Meijering, RORPO, and Sato. The following two experiments evaluate a fusion of different vessel enhancement filters with two variants of segmentation models (3D U-Net and 3D V-Net). The fusion of the RORPO enhancement filter with the CT image to detect the liver vessel structures supported by the 3D V-Net is proven to be the most efficient. We also suggest the best value of the blending coefficient between the RORPO enhanced image and the raw CT image. In such a setup, we further experiment with different variants of dice loss functions to solve the problem of pixel imbalance between the foreground and background categories. The following experiment then performs the ablation study of the selected architecture. In the last experiment, a comparison between our proposed algorithm and other algorithms is provided.

The proposed algorithm is proven to be effective, robust, and accurate for liver vessel segmentation, even for images with low contrast and high noise. Experiments are performed on the public 3D-IRCADb [2] dataset; the average dice score and precision are 76.2% and 77.7%, respectively.

During the work on this topic, we have used 3D Slicer [3] software, for which we have created a software extension that can provide doctors and other healthcare professionals with tools for automatic vessels segmentation, and tools for validating the segmented datasets. Employing deep learning and 3D Slicer, we can segment vascular structures automatically, improve the annotations, automatically collect the annotated datasets, and then fine-tune the neural network models so they can be quickly exposed for further use in automatic segmentation.

2. Related Work

The trend of the proposed segmentation methods leads to the automation of liver vessel segmentation [4,5]. The methods for liver vessel segmentation from CT abdominal images can be divided into several groups: tracking-based algorithms, active contours, and machine learning [6]. The popular approach to segmenting liver vessels is based on machine learning methods and Deep Learning (DL) specifically.

The application of 3D U-Net [7,8,9,10] is often used for the segmentation of liver vessels from CT images. The 3D U-Net is composed of analytical and synthetical parts [11], similar to the standard U-Net architecture, but it uses 3D operators instead. Yu et al. [7] described the 3D Residual U-Net technique, which is built on 3D U-Net with a 3D morphological closure operation in the postprocessing phase. The algorithm was tested on their private CT abdominal dataset. Huang et al. [8] used 3D U-Net with a combination of the variant dice loss function. The segmented veins were more continuous and complete when training was carried out using the enhanced manual expert annotations rather than the original dataset. The tests were carried out using the 3D-IRCADb, SLiver007 [12], and private datasets. Affane et al. [9] experimented with three distinct 3D U-Net approaches: basic U-Net, MultiRes U-Net, and Dense U-Net. On the 3D-IRCADb dataset, they determined that MultiRes U-Net architecture was superior for segmenting hepatic blood veins. Golla et al. [10] employed an ensemble technique. They combined the predictions of networks into ensemble E and averaged the resulting probability for each class. Probability maps of all networks (2D U-Net, 3D U-Net, 2D V-Net, and 3D V-Net) and ensembles were resampled back to the original data, resolution and the maximum probability was applied to the data to extract the predicted segmentation. The approach was tested on the publicly available datasets 3D-IRCADb and MICCAI [12], with unsuitable data being excluded.

V-Net [13] is the next commonly used neural network architecture that has been studied in recent years for segmenting liver blood vessels. Su et al. [14] proposed the DV-Net algorithm. DV-Net represents a V-Net with a dense block structure. They utilize a combination of DCDS (dense connection downsampling approach) and D-BCE loss function for capturing blood vessel structure. The method was tested on 3D-IRCADb. Yang et al. [15] used an improved V-Net to segment liver blood vessels on the 3D-IRCADb dataset. Improved V-Net is based on inter-scale dense connections in the decoder. Altini et al. [16] used 2.5D V-Net for the segmentation of liver blood vessels. They trained the method on the 3D-IRCADb dataset and tested it on their dataset. They used Tversky index-based loss function in combination with 2.5D V-Net. The Tversky index ensures a short convergence time due to the unbalanced voxels problem.

A different approach than the common utilization of either the U-Net or V-Net architecture is applied in [17,18]. In [17], the authors propose a fusion network called TransfusionNet. This method is based on a transformer for semantic segmentation. The authors use the 3D-IRCADb and LiTS [19] datasets for pre-training, and their data for fine-tuning the segmentation network. Xu et al. [18] applied a deep neural network based on the bootstrapping technique to the 3D-IRCADb dataset. A convex combination of the model predictions and preliminary predictions with the subsequent application of a noise filter is used to segment the liver blood vessels.

There is a full body of work that focuses on image enhancement that can be applied before training the segmentation model. This is especially true if complex structures such as vessels are segmented. Shahid and Taj [20] apply a set of preprocessing operations, including filtering via the Frangi filter, to enhance retinal vessels before their segmentation. Soomro et al. [21] use steps to suppress irregular illumination and to improve low and varying contrast via contrast-limited adaptive histogram equalization for the task of retinal vessel segmentation. Blaiech et al. [22] studied the effect of enhancing the 2D images for coronary artery segmentation. Different enhancement methods (Frangi, CLAHE, and RORPO) were used in normal conditions and in the presence of noise. Lamy et al. [23] compared seven different vessel enhancement filters on real (3D-IRCADb) and synthetical (VascuSynth [24]) datasets. RORPO has been evaluated as being the best-performing method on the whole liver area when applied to the real 3D-IRCADb dataset.

With a focus on the use of vessel-enhancing filters as a part of the pipeline for hepatic vessel segmentation, Survarachakan et al. [25] applied four enhancement filters (Hessian, Frangi, Sato, and Meijering), and proposed to fuse their outcomes in two different segmentation designs. The proposed methods are evaluated on the clinical OSLO-COMET dataset.

In our research, we propose an algorithm that effectively fuses the vessel enhancement technique represented by RORPO [1] with the raw CT image, and handles the problem of pixel imbalance by employing a variant of the dice loss function to optimize the segmentation even further. Evaluation is performed on the publicly available 3D-IRCADb dataset. Compared to the above-mentioned related works [16,23,25], our proposed method can provide better segmentation results, and it adopts a much simpler approach of combining properly the original raw image with the enhanced image to tackle the problem of vessel segmentation. Numerous related works likewise fail to account for the foreground–background pixel mismatch. We use Weighted Dice Loss to provide penalties for the number of incorrectly categorized voxels to tackle the problem. Additionally, some of the related research ignores the dataset’s inferior quality annotations for some patients. To ensure that no vessels are missed and that the segmentation accuracy provides an accurate portrayal of various algorithms, we resolve this by enlisting the aid of medical professionals to annotate any missed vessels.

3. Segmentation Platform

Together with the proposed algorithm, we have developed a plugin extension for 3D Slicer [26] that (i) provides a remote AI-Assisted Annotation service (AIAA) to medical doctors from a High-Performance Computing (HPC) cluster, allowing them to use state-of-the-art methods to perform the automatic segmentation of desired tissue from medical images, and (ii) provides a mechanism to collect the segmented and validated data generated in step (i). We can use the stored data to fine-tune the existing neural network models. These enhanced models can then be employed for automatic tissue segmentation in step (i).

Since the current state-of-the-art medical image processing methods are based on Deep Learning (DL), and typically, DL algorithms are trained with a large amount of data, we allow for the use of multiple GPUs during the training phase to produce models of the required quality in a reasonable amount of time. We use the multi-GPU nodes of an HPC cluster to train models from scratch, as well as single GPU nodes to provide model inference using AIAA. All of the connections and data are encrypted using Secure Shell (SSH).

The entire concept is depicted in Figure 1. Two main sections can be distinguished: one runs at a medical doctor’s site in a local hospital (frontend), and the other operates at an HPC cluster facility (backend). The frontend allows the doctor to load, view, and perform automatic segmentation, and improve annotation on the medical data using HPC resources. The backend part provides the computational power and other required features.

The provided concept adopts the Clara Train SDK [27] from NVIDIA. It contains several APIs, such as those for AIAA, and a training framework for DL-based model training. We use Clara Train SDK, version 4.0, which is solely PyTorch-based. The Clara Training Framework builds on the open-source MONAI framework [28], which is specifically devoted to deep learning in healthcare imaging.

Clara’s AIAA is a client-server-based architecture that delivers a C++ or Python client API. Many medical image viewers, such as 3D Slicer (see Figure 2) can be interfaced as clients [29] to obtain the AIAA services offered by the server running on the network. Based on the pre-loaded models on the server, the AIAA can perform the automatic segmentation of specific tissues and display the results. Doctors can edit and postprocess the resulting tasks. A modified version of this extension, accommodated for HPC cluster usage, has been used by medical doctors to provide improved annotations of the 3D-IRCADb dataset.

4. The Hardware Used

For training the models, we utilized cluster nodes with 8 × NVIDIA A100 (40 GB HBM2) per node. Each node is a powerful x86–64 computer, equipped with 2 × 64-core AMD Zen 3 EPYC™ 7763, 2.45 GHz processor and 1024 GB of memory.

5. Datasets and Improved Annotations

Publicly available datasets that contain labeled hepatic vessels are listed in

Table 1. Overview of publicly available datasets.

Name of dataset	Published	Number of patients
3D-IRCADb [2]	5 May 2019	22 patients
Medical Segmentation Decathlon (MSD) [30]	20 December 2020	430 patients
Vascular Synthesizer (VascuSynth) [24]	March 2013	120 synthetic samples

. 3D-IRCADb is the most commonly used dataset. We use it in our experiments as well. This particular dataset is described in more detail in the following subsection.

The MSD [30] is another publicly available dataset. However, its ground truth is of poor quality, far inferior to the 3D-IRCADb dataset. Therefore, we have not used this dataset at all in this work.

We are also aware of synthetically generated vascular datasets that can be provided by the VascuSynth [24] software. However, since these datasets cannot replace clinical data where patients have curvilinear vessels, we have not used them either.

5.1. Dataset 3D-IRCADb

The proposed method has been evaluated on the publicly available 3D-IRCADb [2] dataset. We have specifically used its subset 3D-IRCADb-01, which is used in most of the similar research, and thus allows for the comparison. The dataset contains 20 CT volumes (10 male and 10 female). Fifteen cases (75%) of this dataset have hepatic tumors. The dataset contains various classes. The classes used in our study are portalvein and venoussystem. We combined them into one class of veins. The pixel spacing varies from $0.56$ to $0.84$ mm, and the slice thickness varies from 1 to 4 mm.

There are several challenges within the dataset. In some cases, the inferior vena cava is not part of the liver mask. Besides that, one patient had many liver tumors that covered the majority of the liver vessels. Another patient had inner metallic objects, such as stents, that significantly affected the brightness of the volume. Despite these bottlenecks, we kept all of the volumes from the dataset in the experiment to have more training data. Furthermore, slight abnormalities make the network more robust, even if the output quality suffers. Although low-quality data were used, our technique was still able to achieve a high prediction quality. The training and testing sets should equally include cases of liver vessels of similar appearance (both containing difficult and easy cases). To ensure this, we split the dataset as follows. Training set contains patients—2, 5, 7, 12, 10, 11, 13, 16, 18, 19. Validation includes patients—4, 9, 20, 6, 17. Testing set is represented by patients—1, 3, 8, 14, 15.

Improving Annotations of 3D-IRCADb

Since there are some limitations in the annotations of hepatic vessels in the 3D-IRCADb dataset, we decided to improve them. The limitations are mainly because of inadequately annotated hepatic veins, although they are clearly visible and could be marked correctly. Similar problems have been found and tackled by [8,31]. Some of the CT volumes in 3D-IRCADb are under-segmented, while others are over-segmented. These issues might lead to the misinterpretation of results when segmented vessels appear as false positives or false negatives while testing the neural networks [18]. Another major limitation is the inconsistency of the annotated data for the vena cava. The vena cava is visible in some images but not in others. It also makes vessel enhancement difficult [8].

Because the annotated datasets used in [8,31] are not publicly available, the additional labeling of the missing veins in 3D-IRCADb has been performed using the Slicer built-in tools and the extension explained in Section 3. It increased the quality of annotations; see Figure 3. The comparison of the dice scores with the original and improved dataset is in the experimental part of the paper. If it is not stated differently, the experiments used refined annotations since the vessel prediction is better and more continuous with them.

6. Methods

The core of our solution lies in the use of enhancement filters and deep learning methods for vessel segmentation. We perform experiments on the dataset, as explained in Section 5.1. We apply different filters to improve the contrast between the liver and hepatic vessels, and use the enhanced data as input to a segmentation network.

6.1. Vessel Enhancement Filters

We have used five filters to enhance the tubular structures and other structural information of the veins. Similar to Survarachakan et al. [25], we have applied four Hessian-based filters. Besides that, a specific morphological filter has been used as well. Namely, we have used Hessian, Frangi, Meijering, and Sato as the Hessian-based filters, and RORPO as the morphological filter. Figure 4 shows the enhancing effect of these filters on vessels.

6.1.1. Hessian Matrix Computation

The Hessian matrix serves as a fundamental computational method used in all Hessian-based filters. It calculates the local gradation change by performing the second-order partial derivatives of the input image voxel X = (x, y, z) in nine directions. It is defined as

$$\begin{array}{c}H\left(X\right)=\left[\begin{array}{ccc}{f}_{xx}& f_{xy}& f_{xz}\\ f_{yx}& f_{yy}& f_{yz}\\ f_{zx}& f_{zy}& f_{zz}\end{array}\right]=\left[\begin{array}{ccc}\frac{{\partial }^{2}f}{\partial x^2}& \frac{{\partial }^{2}f}{\partial x\partial y}& \frac{{\partial }^{2}f}{\partial x\partial z}\\ \frac{{\partial }^{2}f}{\partial y\partial x}& \frac{{\partial }^{2}f}{\partial y^2}& \frac{{\partial }^{2}f}{\partial y\partial z}\\ \frac{{\partial }^{2}f}{\partial z\partial x}& \frac{{\partial }^{2}f}{\partial z\partial y}& \frac{{\partial }^{2}f}{\partial z^2}\end{array}\right]\end{array}$$

(1)

Eigenanalysis is used to avoid computing derivatives in many directions, and to extract only the principal directions. Let $e_1$ be the eigenvector representing the axial direction, let $e_2$, $e_3$ be the cross-sectional direction of H(X), and the associated ${\lambda }_1$, ${\lambda }_2$, ${\lambda }_3$ are the eigenvalues. The eigenvalues should meet the following condition: $|{\lambda }_1|\le |{\lambda }_2|\le |{\lambda }_3|$. The vessel centerline voxel should satisfy the following: ${\lambda }_2\approx {\lambda }_3\ll 0$, ${\lambda }_1\approx 0$.

6.1.2. Hessian Vesselness Filter

By integrating the directional gradient and the Hessian matrix, Ng et al. [32] suggested a modified multi-scale Hessian filter. It is assumed that the brightness of a hepatic vessel is high in the middle and progressively declines toward the end, allowing it to be described as a Gaussian structure transverse to its axis. The convolution between the image gradient field and the Gaussian kernel is the basis of each approximation in the Hessian matrix H at a specified scale $\sigma$. The eigenvalues ${\lambda }_1$, ${\lambda }_2$ are calculated and utilized to compute the curve derivation R and similarity measure S. The curvilinear likeliness $\mathcal{E}$ is given by

$$\mathcal{E}(x,y,\sigma )=\left\{\begin{array}{cc}0\hfill & if {\lambda }_2<0\hfill \\ e^{\frac{-R}{2{\beta }_1^2}}[1-e^{\frac{-s}{2{\beta }_2^2}}]\hfill & otherwise\hfill \end{array}\right.$$

(2)

The response of the filter L is set as a maximum of different scales that approximate the size of the ridges. It is expressed as

$$L(x,y)=\underset{{\sigma }_{min}⩽\sigma ⩽{\sigma }_{max}}{max}\left[\mathcal{E}(x,y,\sigma )\right]$$

(3)

The parameters ${\beta }_1$, ${\beta }_2$ control the sensitivity of the filter to the measure R and S, respectively.

6.1.3. Frangi Vesselness Filter

The Frangi [33] approach uses all three eigenvalues to discriminate the local orientation pattern. These three eigenvalues serve in the differentiation of blobs $R_b$, and plate-like and line-like structures $R_a$. To decrease the influence of noise S, a Hessian norm measure was developed.

$$R_b=\left|{\lambda }_1\right|/\sqrt{|{\lambda }_2{\lambda }_3|}$$

(4)

$$R_a=|{\lambda }_2|/|{\lambda }_3|$$

(5)

$$S=\sqrt{{\lambda }_1^2+{\lambda }_2^2+{\lambda }_3^2}$$

(6)

They propose the following combination of the components to define a vesselness function,

$$F=\left\{\begin{array}{cc}0\hfill & if {\lambda }_2>0 or {\lambda }_3>0,\hfill \\ (1-exp(-\frac{R_a^2}{2{\alpha }_2}))exp(-\frac{R_b^2}{2{\beta }_2})(1-exp(-\frac{S^2}{2c_2}))\hfill \end{array}\right.$$

(7)

The parameters $\alpha$, $\beta$, $\gamma$ are the thresholds that control the sensitivity of the filter to the measures $R_a$, $R_b$, and S.

6.1.4. Meijering Vesselness Filter

To recognize very elongated structures, Meijering et al. [34] suggested a parameter-free vesselness function. It is based on the following modified Hessian matrix $H^{\prime }\left(\mathit{f}\right)$:

$$H^{\prime }\left(\mathit{f}\right)=\left[\begin{array}{lll}{h}_{11}+\frac{\alpha }{2}(h_{22}+h_{33})& (1-\frac{\alpha }{2})h_{12}& (1-\frac{\alpha }{2})h_{13}\hfill \\ (1-\frac{\alpha }{2})h_{21}& h_{22}+\frac{\alpha }{2}(h_{11}+h_{33})& (1-\frac{\alpha }{2})h_{23}\hfill \\ (1-\frac{\alpha }{2})h_{31}& (1-\frac{\alpha }{2})h_{32}& h_{33}+\frac{\alpha }{2}(h_{11}+h_{22})\hfill \end{array}\right]$$

(8)

In general, $\alpha$ = 1/3. The eigenvalues of $H^{\prime }\left(\mathit{f}\right)$ with respect to $H\left(f\right)$ is expressed as

$${\lambda }_i^{\prime }={\lambda }_i+\alpha {\lambda }_j+\alpha {\lambda }_k$$

(9)

for $i\ne j\ne k$.

The vesselness is defined by,

$$F=\left\{\begin{array}{cc}{\lambda }_{max}/{\lambda }_{min}\hfill & {\lambda }_{max}<0\hfill \\ 0\hfill & {\lambda }_{max}\ge 0\hfill \end{array}\right.$$

(10)

where, ${\lambda }_{max}=max\{{\lambda }_1^{\prime },{\lambda }_2^{\prime },{\lambda }_3^{\prime }\}$, which is computed at each voxel, and ${\lambda }_{min}$ is the minimum of all ${\lambda }_{max}$ of the image.

6.1.5. Sato Vesselness Filter

Sato et al. [35] suggested a line enhancement filter function that is responsive to varied diameter ranges. Sato et al. sorted the eigenvalues as ${\lambda }_i$ as ${\lambda }_1\ge {\lambda }_2\ge {\lambda }_3$. The eigenvector $e_1$ corresponds to the direction of the putative vessel. $|{\lambda }_1|$ < 0 and $|{\lambda }_1|$ < 0 represent the sizes of the cross-section. The Sato vesselness introduces a ratio of the eigenvalues to obtain a high response in tubular structures. It is provided by,

$$F=\left\{\begin{array}{cc}{\lambda }_{c}exp(-\frac{{\lambda }_1^2}{2({\alpha }_1{\lambda }_c{)}^2)})\hfill & {\lambda }_1⩽0,{\lambda }_c\ne 0\hfill \\ {\lambda }_{c}exp(-\frac{{\lambda }_1^2}{2({\alpha }_2{\lambda }_c{)}^2)})\hfill & {\lambda }_1>0,{\lambda }_c\ne 0\hfill \\ 0\hfill & {\lambda }_c=0\hfill \end{array}\right.$$

(11)

where ${\alpha }_1<{\alpha }_2$, ${\lambda }_c=min\{-{\lambda }_2,-{\lambda }_3\}$. The parameters ${\alpha }_1$ and ${\alpha }_2$ control the asymmetrical strength.

6.1.6. RORPO Vesselness Filter

Differential information is not used by the RORPO [1,36] filter. RORPO, on the other hand, is based on the mathematical morphology of the path operators, and is thus defined using adjacency relations. RORPO is semi-global and non-linear. This filter computes the path openings in the seven primary directions, ranks their responses point-by-point, and extracts structures based on low and high responses at each voxel. This noise-resistant filter maintains the intensity of curvilinear shapes while reducing the intensity of other structures.

Path operators with thin voxels oriented along their length are expressed as:

$$L_{n_R}=L_{min}\times f^{n_R-1},$$

(12)

where $L_{n_R}$ is path length to detect, $L_{min}$ is the minimal path length, the geometric sequence of scales is f$ϵR$, and the number of scales is $n_R$.

6.2. Segmentation Pipelines

The three main steps of the segmentation pipelines we have implemented are image data preprocessing, vessel enhancement, and image segmentation. These are explained in the following subsections. First, the image preprocessing is performed, and then specific enhancement filters are applied and either fused before the segmentation or immediately after the segmentation. Figure 5, Figure 6 and Figure 7 show three different approaches.

We have implemented three segmentation pipelines. In the first two pipelines, the fusion of filters are inspired by the approach of Survarachakan et al. [25], and they serve as a comparison. The third pipeline that we propose implements a simple but very effective fusion of raw and filtered images before performing the segmentation.

6.2.1. Data Preprocessing

In our method, the liver area is extracted and cropped according to the liver mask to focus primarily on the liver vascular system. The liver region is extracted in the first step as opposed to the last step in Survarachakan et al. [25]. This helps to remove unnecessary resulting boundaries, and leads to less interference with the background. All scans are converted into $1\times 1\times 1$ mm isotropic resolution. The image intensities are windowed to lie within the $\langle 80,220\rangle$ Hounsfield Unit (HU) range, and are then mapped to the range $\langle 0,1\rangle$. In addition, the 3D anisotropic diffusion is applied to lower the image noise and to still preserve the significant parts such as edges and lines.

6.2.2. Vessel Enhancement

The enhancement of vessels is the preprocessing step used before the segmentation. The preprocessing steps are implemented using the scikit-image library [37]. This step is a key factor in obtaining significantly better segmentation results. Enhanced images acquired by the application of different filters (see Section 6.1) are utilized either before or following the segmentation.

A method that combines the results of individual vessel enhancement filters simultaneously before segmentation, and which is inspired by the approach of Survarachakan et al. [25], is depicted in Figure 5. The final enhanced image is obtained by averaging the respective pixel values from all filters. The resulting image is then used as an input to the segmentation network. Two variants of this approach have been considered. They either use Hessian, similar to [25], or RORPO as one of the four filters. As proven by experiments, the Hessian filter has the weakest enhancing effect on the segmentation results in terms of the dice score. The RORPO filter, on the other hand, has the strongest enhancing effect. Therefore, the initial idea of this method was to use the strongest combination of four filters that would outperform the solution presented in [25]. We indicate this method as FilterAdded, and distinguish whether Hessian or RORPO has been used. The fusion of filters is defined as:

$$FilterAdde{d}_{He}=\frac{He+Fr+Me+Sa}{\mathit{number}\mathit{of}\mathit{filters}}$$

(13)

and

$$FilterAdde{d}_{Ro}=\frac{Ro+Fr+Me+Sa}{\mathit{number}\mathit{of}\mathit{filters}}$$

(14)

where $He$, $Ro$, $Fr$, $Me$, and $Sa$, stand for the Hessian, RORPO, Frangi, Meijering, and Sato filter outputs.

Inspired by the second approach of Survarachakan et al. [25], a better overall result can also be achieved by combining the prediction results from four trained models, each of which used a different vessel improvement filter. Only the pixels predicted from at least two independent filters were taken into account when combining the results, in order to reduce false positives. The approach is depicted in Figure 6. Similar to the previous approach, we opt between two variants of applied filters, either with the Hessian or with the RORPO. Again, the variant with the RORPO filter brings better results, as shown in the experiments. We indicate this method as SegAdded, and distinguish whether Hessian or RORPO has been used. A combination of the enhanced outputs is provided in the following manner:

$$\begin{array}{c}\hfill SegAdde{d}_{He}=(\mathrm{He}\cap \mathrm{Fr})\cup (\mathrm{He}\cap \mathrm{Me})\cup (\mathrm{He}\cap \mathrm{Sa})\\ \hfill \cup (\mathrm{Sa}\cap \mathrm{Me})\cup (\mathrm{Sa}\cap \mathrm{Fr})\cup (\mathrm{Me}\cap \mathrm{Fr})\end{array}$$

(15)

and

$$\begin{array}{c}\hfill SegAdde{d}_{Ro}=(\mathrm{Ro}\cap \mathrm{Fr})\cup (\mathrm{Ro}\cap \mathrm{Me})\cup (\mathrm{Ro}\cap \mathrm{Sa})\\ \hfill \cup (\mathrm{Sa}\cap \mathrm{Me})\cup (\mathrm{Sa}\cap \mathrm{Fr})\cup (\mathrm{Me}\cap \mathrm{Fr})\end{array}$$

(16)

where $\mathrm{He}$, $\mathrm{Ro}$, $\mathrm{Fr}$, $\mathrm{Me}$, and $\mathrm{Sa}$, stand for the Hessian, RORPO, Frangi, Meijering and Sato filtered and segmented outputs, respectively.

The third and most effective approach that we have implemented operates with the RORPO filter and makes a fusion of the enhanced image with the preprocessed raw image before the segmentation. The detailed pipeline is shown in Figure 7. The linear blending fuses the original and enhanced images to create input for the segmentation network.

6.2.3. Linear Blending

The blending method applies a linear combination of two source images, and it is expressed as:

$$g\left(x\right)=\beta ·f_0\left(x\right)+\alpha ·f_1\left(x\right),$$

(17)

where $f_0\left(x\right)$ represents the original image, $f_1\left(x\right)$ stands for RORPO enhanced image, $g\left(x\right)$ provides the blended image, and x represents the pixel coordinate. Both images have to be of the same size and type. Coefficient $\alpha$ lies in the interval $\langle 0,1\rangle$ and determines how much the enhanced image will prevail over the original image. The optimal value of Coefficient $\alpha$ can be determined by running experiments in the stated interval, with an appropriate $\alpha$ increment. Coefficient $\beta$ depends on $\alpha$ and it is expressed as:

$$\beta =(1.0-\alpha )$$

(18)

6.3. Segmentation Model

We have experimented with two types of segmentation architectures: 3D U-Net and 3D V-Net. They are often used to segment tissues from medical images.

The 3D U-Net architecture on which we conducted the experiments is shown in Figure 8. Each layer of the network has a contracting encoding path and an expanding decoding path with a skip connection between them. The encoding path extracts hidden features from the input data, which are then reconstructed by the decoding path, to provide the prediction. The skip connection is used to retain more semantic information, and thus adds the initial feature map to the final feature map. The encoder downsamples the data using strided convolutions of the residual unit in the spatial dimensions by a factor of 2, followed by a Batch Normalization (BN), and a Parametric Rectified Linear Unit (PReLU) as an activation function. During backpropagation, the PReLu learns a small slope parameter, which multiplies with the negative input to better adapt to the other parameters (such as weights and biases). The decoder upsamples using 2 × 2 × 2 up-convolution (transpose convolutions) with stride 2. These convolutions occur at the beginning of each block.

The 3D V-Net implementation is based on Milletari’s [13] design of the 3D V-Net. The 3D V-Net is intended for 3D medical image segmentation and is shown in Figure 9. The network has encoding and decoding blocks, as well as an additional convolutional block in between, similar to the 3D U-Net. The encoder and decoder are also coupled by skip connections, but instead of 2 × 2 × 2 max-pooling, down-convolutions with stride 2 × 2 × 2 and kernel-size 2 × 2 × 2 are used for encoding. The strided deconvolution is used for decoding. The top-level blocks have one convolutional layer, the second-level blocks have two, and the remaining blocks have three convolutional layers. The number of channels in each block is the same as in the U-Net configuration, but all convolutions extract spatial features using 5 × 5 × 5 kernels.

In the case of hepatic vessel segmentation, the network performs better if the image data are preprocessed (see Section 6.2.1).

6.3.1. Data Augmentation

We employ data augmentation in our approach to improve the amount of input data, and to decrease overfitting while training. In each spatial dimension, we apply symmetric padding to the data. We crop random fixed-size sections with a foreground or a background voxel in the center. We rotate the data by ${90}^{\circ }$ in all three spatial planes. We also use a random offset to adjust the intensity.

6.3.2. Parameter Settings and Training

We have used the same parameter settings for all experiments reported in this paper, unless otherwise stated. We have used a batch size of 2, the Adam optimizer with a learning rate of $2.0e^{-4}$, and 1000 of epochs. A learning rate scheduler has been applied to adjust the initial learning rate and to further decrease it. The learning rate scheduler is defined as $lr=init\_lr\times {(1-\frac{epoch}{nEpoch})}^{power}$, where $init\_lr=0.0002$, $nEpoch=5000$ and $power=0.1$. We have used a Dropout of 0 for 3D U-Net, and 0.5 for 3D V-Net. We have used ReLU as the activation function.

6.3.3. Loss Function

During the training process, the loss function assists in minimizing the error for each training sample. Because liver vessels make up a minor percentage of the liver, imbalanced foreground (the liver vessels) and background (the liver) classes frequently induce prediction deviation, biasing the classification to the background with more voxels. Therefore, we have experimented with different loss functions. We have used the Weighted Dice Loss function [38] as a final selection. It uses the dice coefficient to place penalties for the number of misclassified voxels, to solve the high-class imbalance problem and to enhance segmentation accuracy; see

Table 2. Loss Functions: where $g_i$ is the ground truth; $p_i$ is the prediction.

Loss Function	Equation	Definition
Dice [39]	$${\displaystyle \frac{2{\sum }_i^N{p}_i{g}_i}{{\sum }_i^N{p}_i^2+{\sum }_i^N{g}_i^2}}$$	Calculates the overlap of $g_i$ and $p_i$.
Dice-BCE [40]	$${\displaystyle 1-\mathrm{Dice}+{\lambda }_b{L}_{BCE}}$$ $${\displaystyle L_{BCE}=-(p_{i}log\left(g_i\right)+(1-p_i)log(1-g_i))}$$	Evaluates the class prediction of each pixel vector, and then averages all pixels. ${\lambda }_b$ is 0.5.
Log Cosh Dice [40]	$${\displaystyle log(cosh(p_i-g_i))+{\lambda }_l\ast Dice}$$	For smoothing variations, can be used for skewed dataset. ${\lambda }_l$ is 0.5.
Focal [40]	$${\displaystyle -{\alpha }_t{\left(1-L_{BCE}\right)}^{\gamma }log\left(L_{BCE}\right)}$$ $${\displaystyle L_{BCE}=-(p_{i}log\left(g_i\right)+(1-p_i)log(1-g_i))}$$	It up-weights the contributions of difficult examples. Here, $\gamma$ is 0.5, ${\alpha }_t$ is 0.5.
Dice Focal [40]	$${\displaystyle Dice+{\lambda }_f\ast Focal}$$	Variant of Dice loss, with focus on difficult examples. ${\lambda }_f$ is 0.5.
Tversky [40]	$${\displaystyle \frac{{\sum }_i^N{p}_i{g}_i}{{\sum }_i^N{p}_i{g}_i+{\beta }_t{\sum }_i^N(1-p_i)g_i+{\alpha }_t{\sum }_i^N{p}_i(1-g_i)}}$$	Adds weight to False positives and False negatives, where ${\alpha }_t$ is 0.5 and ${\beta }_t$ is (1 − ${\alpha }_t$).
Weighted Dice [38]	$${\displaystyle 1-\frac{{\sum }_i^N{p}_i{g}_i}{{\sum }_i^N{p}_i{g}_i+{\alpha }_l{\beta }_l{\sum }_i^N(1-p_i^2){\sum }_i^N(1-g_i^2)}}$$	Data with unbalanced classes, where ${\alpha }_l$ is 0.5 and ${\beta }_l$ is 6 [38].

.

7. Experiments and Results

We have tested the proposed algorithms on the 3D-IRCADb dataset. If not otherwise stated, all of the experiments were performed using improved labels in the 3D-IRCADb dataset. We have used typical metrics to compare the implemented methods. Specifically, these include the Dice Score, the Accuracy, the Sensitivity, and the Precision. Specific equations for the metrics are expressed in

Table 3. Measures of segmentation quality. TP and TN stand for True Positive and True Negative (correctly segmented pixels), while FP and FN stand for False Positive and False Negative (incorrectly segmented pixels).

Metrics	Equation	Explanation
Dice Score	${\displaystyle \frac{2TP}{2TP+FP+FN}}$	The overlap of the two segmentations, divided by the total size
Accuracy	${\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$	Correctly predicted data points out of all the data points
Sensitivity	${\displaystyle \frac{TP}{TP+FN}}$	When identifying TP and the cost of an FN is high
Precision	${\displaystyle \frac{TP}{TP+FP}}$	When identifying TP and the cost of an FP is high

.

7.1. Experiment 1: Comparison of Different Vessel Enhancement Filters

First, the five vascular enhancement filters were compared: Hessian, Frangi, Meijering, RORPO, and Sato. The five different neural networks were trained to compare the filters in terms of their effect on vessel segmentation. In this experiment, the 3D U-Net architecture was used. The processing workflow from the raw input to the segmented output is shown in Figure 10.

Table 4. Comparison of different enhancement filters on the 3D-IRCADb dataset and 3D U-Net as a segmentation network.

Filter	Original Annotations
	DSC	ACC	SEN	PRC
Unenhanced	$56.4\pm 13.8$	$98.3\pm 2.3$	$66.0\pm 14.2$	$56.5\pm 23.9$
Hessian	$60.0\pm 12.1$	$99.0\pm 0.3$	$62.9\pm 6.2$	$62.4\pm 21.6$
Frangi	$62.1\pm 6.6$	$99.0\pm 0.4$	74.6 ± 6.6	$53.9\pm 8.6$
Meijering	$62.4\pm 8.9$	$99.0\pm 0.7$	$69.1\pm 9.2$	$57.6\pm 11.9$
Sato	$58.3\pm 9.1$	$98.7\pm 0.9$	$68.4\pm 7.3$	$51.5\pm 11.4$
RORPO	62.7 ± 6.6	99.2 ± 0.2	60.7 ± 6.7	69.4 ± 19.1
Filter	Improved annotations
	DSC	ACC	SEN	PRC
Unenhanced	$62.7\pm 15.4$	$98.2\pm 2.3$	$72.5\pm 13.7$	$63.2\pm 25.5$
Hessian	$66.1\pm 8.0$	$99.0\pm 0.3$	$68.3\pm 5.3$	$65.4\pm 13.0$
Frangi	$69.6\pm 3.9$	$99.1\pm 0.1$	$75.7\pm 6.1$	$64.6\pm 3.7$
Meijering	$66.7\pm 10.1$	$98.7\pm 1.0$	$76.8\pm 6.2$	$61.2\pm 16.2$
Sato	$67.0\pm 4.8$	$98.9\pm 0.4$	77.7 ± 7.1	$59.2\pm 4.9$
RORPO	69.7 ± 4.8	99.2 ± 0.1	62.9 ± 6.1	79.1 ± 9.0

provides the comparison of different filters concerning segmentation quality measures. The table is divided into two sections. The first one provides results using original annotations, and the second one uses improved annotations.

7.2. Experiment 2: Fusion of Vessel Enhancement Filters

As described in Section 6.2.2, we have used three different segmentation pipelines that leverage different vessel enhancement filters to improve segmentation results. Specifically, the pipelines named as $FilterAdde{d}_{He}$, $FilterAdde{d}_{Ro}$, $SegAdde{d}_{He}$, $SegAdde{d}_{Ro}$, and our method (the RORPO filter with a linearly blended input image) were implemented and compared. The dice scores of different approaches performed on the test data are shown in

Table 5. Comparison of segmentation pipelines with enhancement filters applied to improved 3D-IRCADb with standard Dice Loss function and different networks.

Filter	3D U-Net
	DSC	ACC	SEN	PRC
Unprocessed	$62.7\pm 15.4$	$98.2\pm 2.3$	$72.5\pm 13.7$	$63.2\pm 25.5$
FilterAdded${}_{He}$	67.8 ± 4.8	99.0 ± 0.4	72.8 ± 5.6	64.4 ± 9.0
FilterAdded${}_{Ro}$	71.7 ± 3.2	$99.1\pm 0.2$	$77.3\pm 5.0$	$67.1\pm 3.7$
SegAdded${}_{He}$	70.2 ± 4.2	99.0 ± 0.3	79.5 ± 6.1	63.1 ± 5.0
SegAdded${}_{Ro}$	70.9 ± 3.8	$99.0\pm 0.3$	79.3 ± 6.1	$64.3\pm 4.2$
Our method	74.4 ± 4.6	99.3 ± 0.1	72.7 ± 5.8	76.5 ± 6.4
Filter	3D V-Net
	DSC	ACC	SEN	PRC
Unprocessed	$68.4\pm 12.2$	$98.7\pm 1.1$	$80.7\pm 3.6$	$61.3\pm 17.3$
FilterAdded${}_{He}$	$68.8\pm 4.4$	$99.1\pm 0.2$	$69.5\pm 4.9$	$68.3\pm 6.0$
FilterAdded${}_{Ro}$	$73.5\pm 3.8$	$99.2\pm 0.1$	$75.9\pm 4.8$	$71.5\pm 5.3$
SegAdded${}_{He}$	$72.9\pm 3.5$	$99.1\pm 0.1$	$81.5\pm 4.4$	$66.1\pm 4.4$
SegAdded${}_{Ro}$	$72.7\pm 3.7$	$99.1\pm 0.1$	82.1 ± 3.8	$65.5\pm 5.0$
Our method	75.4 ± 4.3	99.3 ± 0.1	$76.9\pm 4.1$	74.4 ± 7.8

.

Comparison of Segmentation Pipelines

We have compared all three segmentation pipelines and have used either 3D U-Net or 3D V-Net as the segmentation architecture. The Segmentation results are shown in Table 5. Our approach provides the best performance in this comparison. As for the specific segmentation network, the 3D V-Net leads to better results than the 3D U-Net if applied in our method.

7.3. Experiment 3: Linear Blending with the RORPO Filter

Based on the results from the previous experiment, it is obvious that the linear blending with just the RORPO filter outperforms approaches where multiple filters are combined. Therefore, the focus was aimed on using only this method and elaborating whether the blending between the original and enhanced image was set to its optimum, or whether it could improve the results even further by setting it to a different value. As described by Equations (17) and (18), the blending is controlled by setting the $\alpha$ value from the interval $<0,1>$. The effect of different $\alpha$ values is shown in Figure 11.

7.3.1. Comparison of Segmentation Models with Different Blending Values

We have compared our method of linear blending with the RORPO filterm using either 3D U-Net or 3D V-Net as the segmentation architecture. The highest dice score is achieved by blending with $60\%$ ($\alpha =0.6$) of the image enhanced usin the RORPO, and $40\%$ ($\beta =0.4$) of the original image in both 3D U-Net- and 3D V-Net-based architectures; see Figure 11. It also shows that segmentation with 3D V-Net provides better dice scores and more consistent results over a wider spectrum of $\alpha$ values than 3D U-Net. As an acceptable region for blending value, the region between 40 and 60% is set.

7.3.2. Comparison of Different Loss Functions

We have evaluated different types of loss functions and their effects on the segmentation quality. The evaluation was performed on the best-performing setup with the 3D V-Net as a segmenter. The results for all metrics are shown in

Table 6. Comparison of different loss functions tested on the improved 3D-IRCADb dataset and the 3D V-Net architecture.

Loss Functions	3D V-Net
	DSC	ACC	SEN	PRC
Dice Loss	$75.4\pm 4.3$	$99.3\pm 0.1$	$76.9\pm 4.1$	$74.4\pm 7.8$
Dice-BCE Loss	$75.9\pm 3.5$	$99.3\pm 0.1$	$75.5\pm 3.1$	$76.8\pm 7.1$
Log Cosh Dice Loss	$74.6\pm 3.3$	$99.3\pm 0.1$	$73.5\pm 4.2$	$76.3\pm 7.4$
Focal Loss	$74.4\pm 3.7$	$99.3\pm 0.1$	$70.2\pm 4.0$	$79.6\pm 8.0$
Dice Focal Loss	$75.2\pm 4.3$	$99.3\pm 0.1$	$75.2\pm 4.3$	$75.6\pm 7.0$
Tversky Loss	$76.0\pm 4.2$	$99.3\pm 0.1$	75.7 ± 3.9	$76.8\pm 7.1$
Weighted Dice Loss ${\beta }_l$ = 6	76.2 ± 3.7	99.3 ± 0.1	75.0 ± 3.4	77.7 ± 6.1

.

The best-performing loss function in our model is the Weighted Dice Loss. It operates with the parameter ${\beta }_l$ (see Table 2) to tackle the problem of class imbalance. The higher the value of ${\beta }_l$, the greater the penalty for misclassified voxels. Unfortunately, this also slows down the gradient flow and increases the risk of sticking in the local minima. When ${\beta }_l$ is in the range of 5–7, optimal outcomes are obtained. We have found ${\beta }_l=6$ to be the best in our study.

7.4. Experiment 4: Ablation Study on the Final Segmentation Pipeline

We have performed the ablation of a varying number of convolutional filters at all stages of the 3D V-Net architecture after training. We have found that certain stages are more significant than others for the quality of veins segmentation.

The V-Net is divided into several stages (the stages are shown in Figure 9) that operate at different resolutions during the down- as well as upsampling phases. Each stage consists of one to three convolutional layers, with each layer having a specific number of channels (filters). We ablated groups of similar filters at every stage by increasing the percentage ($25\%$, $50\%$, and $75\%$) of similar filters. Since the number of filters varies from stage to stage, it means that the same percentage may correspond to a different number of ablated filters. The similarity between filters within a group was calculated based on the absolute Euclidean distance of the normalized filter weights. Ablations were performed by manually setting the weights and biases of all incoming connections of a filter to 0, thus eliminating any activation of that filter.

Figure 12 shows the dice scores for ablations of $25\%$, $50\%$, and $75\%$ at every stage of the V-Net. Each data point shows the effect of a specific stage ablation. The dice score is affected noticeably more in stages ’in 16’, ’down 64’, and ’up 64’, than in the other stages. Furthermore, the effect of the largest ablation has a stronger influence on some stages than on others. For instance, stage ’up 32’ shows a noticeably larger drop in dice score for $75\%$ of the ablated filters, compared to $50\%$, while the stages ’down 256’ and ’up 256’ are largely unaffected.

7.5. Segmentation Results

Figure 13 shows the segmentation results of the proposed algorithm on the improved 3D-IRCADb dataset. The provided method can predict the vascular structure well, and the results are close to the ground truth.

7.6. Experiment 5: Comparison of Performance between the Proposed Algorithm and Other Algorithms

We have compared our segmentation approach with the work of others. Specifically, with works [4,8,14,15,16,18,23], which combine different approaches to provide liver vessel segmentation.

In the case of [4,8,14,15,18], it was impossible to make a fair comparison. The source code for these methods was not available, so it was not possible to train and test these methods on the same enhanced 3D-IRCADb dataset that we used in our work. We provide only an indirect comparison with the numerical results as presented in those papers. A comparison is provided in the first part of

Table 7. Comparison of the proposed method, with related works.

Filters	Published	Dice score	Accuracy	Sensitivity	Specificity
Indirect comparison
Zhang et al. [4]	2018	67.3	96.4	73.7	97.4
Huang et al. [8]	2018	75.3	97.6	76.7	98.8
Su et al. [14]	2021	75.46	-	76.9	-
Yang et al. [15]	2021	71.6	98.5	75.4	99.5
Xu et al. [18]	2020	68.7	99.8	78.6	99.2
Direct comparison (re-implemented related works)
Altini et al. [16]	2020	65.0 ± 9.7	99.8 ± 0.4	52.3 ± 10.8	88.9 ± 6.2
Lamy et al. [23]	2020	59.6 ± 5.8	98.9 ± 0.3	55.9 ± 4.2	64.8 ± 12.0
Our method	2022	76.2 ± 3.7	99.3 ± 0.1	75.0 ± 3.4	99.7 ± 0.1

.

We were able to objectively evaluate the performance of methods from [16,23] since their code is available. The authors of the paper [23] published the code on GitHub and have also created an online demonstration tool for using the filters. We tested the RORPO filter with default parameter settings. Then, the result was thresholded. The parameter $nbThresholds=200$ means that each patient is thresholded with a $200x$ different value. This value is increased by a very small step of $0.005$ and lies in the range of $<0,1>$. As opposed to that, the authors of the paper [16] implemented 2.5D V-Net with Tversky loss function. They combined patients from the 3D-IRCADb dataset, and CT scans from the Polyclinic of Bari. The 2.5D V-Net approach processes five slices as five channels, since the network uses only 2D convolutional layers. The random patches of five consecutive slices are sent to the network for training. To test the network, the sliding window is used. However, we infer sub-volumes of dimensions $512\times 512\times 5$, and only the middle slice is used for prediction on the middle slice. The results of these methods and our proposed method are provided in the second part of Table 7.

8. Discussion

In the first experiment, different vascular enhancement filters were compared. The task was to find out which of the five filters was the best, in terms of the segmentation quality measures. From the visualizations of the filters’ enhancing effects in Figure 4, it can be seen that the RORPO filter performs very well. There is a minimum amount of image noise. In addition, there are no visible boundaries between the liver and the background that could affect the result. Although the image enhanced by the Frangi filter seems promising as well, the vessels are relatively thin. Visible boundaries and some noise are typical for the Meijering filter and the Sato filter. In terms of the segmentation quality, the dice scores for the original and the improved annotations prove that the RORPO is the most effective filter, and has permitted us to improve the dice score from 62.7% to 69.7% and to reduce the mean standard deviation from 13.8% to 4.8%. The improvement in dice score and reduction in standard deviation shows that the model can segment vessels more efficiently in different test patients.

The second experiment focused on leveraging the vessel enhancement effects of different filters in various segmentation pipelines. In FilerAdded, four enhancement filters were combined, and the model was trained on this combined data. In SegAdded, four models were trained with distinct enhancement filters, and the results were later combined when at least two models were able to predict the vessel pixel. In linear blending, the original image was combined with the RORPO enhanced image. By comparing the different pipelines, the linear blending provides the best results for all observed metrics except sensitivity. Here, the SegAdded approach performs better. Results were also obtained from different segmentation models (3D U-Net and 3D V-Net).

In the third experiment, a detailed analysis of the proposed method was provided. We have explored the effect of RORPO on the original image. We have set the mixing ratio between RORPO and the original image, $\alpha$, from $0\%$ to $100\%$, with a step of $10\%$. The RORPO filter has a significant positive effect on the result. The best segmentation is achieved by using $40\%$ of the original image and $60\%$ of RORPO. The acceptable $\alpha$ range is 40–60%. This proves that the role of background information is also helpful in segmenting the region of interest. The results are also performed on 3D U-Net and 3D V-Net, and 3D V-Net performs better than 3D U-Net. This shows that for volumetric input, 3D V-Net performs better. Hence, we used V-Net in all further tests. However, 3D V-Net is more expensive in terms of computing resources, and the time for training is twice as long as that of 3D U-Net. Another evaluation of the proposed method focused on the use of different dice loss functions. The Weighted Dice Loss ${\beta }_l=6$ outperformed other loss functions, achieving a dice score of 76.2% and a mean standard deviation of 3.7%. This loss function can deal with unbalanced classes and enhance segmentation accuracy and sensitivity. Parameter ${\beta }_l=6$ was used to introduce a penalty for misclassified pixels, and also to avoid trapping the loss function in local minima. The ${\beta }_l$ parameter can be in the range (0, 9). If the ${\beta }_l$ parameter is set to a lower or higher value than 6, the results are worse overall, since the coefficient penalizes too few or too many false negatives. This approach set the final version of the method and led to the highest dice score, as well as increased precision.

The fourth experiment examined the effects of varying percentages of ablations on the segmentation performance of the 3D V-Net architecture trained on the 3D-IRCADb dataset. As anticipated, the performance typically declined as the number of ablated filters increased. The results of the experiment showed that some filters contributed more to the performance than others. The effect of ablation is much stronger for stages (’down 64’, ’up 64’) for $50\%$ ablation, and stages ’up 32’ for $75\%$ ablation. The stages (’down 256’, ’up 256’) showed negligible performance loss, as these stages may have redundant feature representation. This experiment determines the influence of different layers on the segmentation results. By identifying the network’s sensitive areas, we want to modify the model’s architecture in future research.

In the last experiment, we compared the performance of the proposed method with some other algorithms described in the related work. We compared indirectly with [4,8,14,15,18] because it was not possible to re-implement or evaluate these methods with our dataset. A direct comparison was made with [16,23]. These methods were re-implemented based on the available source code and evaluated with our dataset. The authors in [23] just used a thresholding approach that did not provide satisfactory vessel segmentation. The authors of the paper [16] used 2.5D V-Net with a Tversky loss function, yielding results with high variability and a high mean standard deviation. We can state that our method prevails in dice score and specificity value over all of the other methods. In comparison with the two reimplemented methods, we dominate in terms of sensitivity, meaning that there are fewer missing and fragmented vessels. In terms of accuracy, our method lags just slightly behind.

Overall, the method outlined in this paper can automatically segment enough liver vessels to correctly extract the vessels, and it can be further manually corrected using Slicer tools. The automatic segmentation results are visually presented in Section 7.5. The method works on CT obtained from different machines with different acquisition parameters. Our method is also able to segment images with different liver shapes, livers with thinner vessels, and livers with vessels atrophied. The specific combination of the original image with the vessel-enhanced image is the originality of our work, and it outperforms other methods proposed in the literature. The method outlined in this paper have some limitations. The results in some cases have misclassified vessel endings, variability in vessel edges, and some regions are not connected to the main vessels. Finally, we need more data to make the model more robust, test the vessel segmentation, and lower the variability in our experiments.

9. Conclusions and Future Works

The proposed algorithm effectively segments liver vessels with a dice score of $76.2\%$ and a precision of $77.7\%$. The network can automatically segment labeled or even other unmarked liver vessels (that should be labeled) from the CT images. The RORPO filter used for vessel enhancement in conjunction with blending its output with the original CT image has successfully been proven to give better results compared to other state-of-the-art techniques. Thus, adding this information to the enhanced vessels has helped to improve the robustness of the segmentation model. The Weighted Dice Loss metric based on the dice coefficient is used for the loss function computation to improve segmentation accuracy, and to deal with unbalanced foreground and background class voxels. The Weighted Dice Loss function with ${\beta }_l=6$ has been proven to perform the best if it is combined with the 3D V-Net.

This best performing model has been made available to the local hospital’s doctors, who can use it for the initial annotation of new data using the tool described in Section 3.

In the future, we plan to cooperate with medical doctors more closely, extend the training set with new validated data, and continue our research in the area of hepatic vessel segmentation. Additionally, in our future studies, we will focus on experiment pruning and computational cost reduction by understanding the significance of ablated filters.