Segmentation of Liver Tumor in CT Scan Using ResU-Net

Abstract

Segmentation of images is a common task within medical image analysis and a necessary component of medical image segmentation. The segmentation of the liver and liver tumors is an important but challenging stage in screening and diagnosing liver diseases. Although many automated techniques have been developed for liver and tumor segmentation; however, segmentation of the liver is still challenging due to the fuzzy & complex background of the liver position with other organs. As a result, creating a considerable automated liver and tumour division from CT scans is critical for identifying liver cancer. In this article, deeply dense-network ResU-Net architecture is implemented on CT scan using the 3D-IRCADb01 dataset. An essential feature of ResU-Net is the residual block and U-Net architecture, which extract additional information from the input data compared to the traditional U-Net network. Before being fed to the deep neural network, image pre-processing techniques are applied, including data augmentation, Hounsfield windowing unit, and histogram equalization. The ResU-Net network performance is evaluated using the dice similarity coefficient (DSC) metric. The ResU-Net system with residual connections outperformed state-of-the-art approaches for liver tumour identification, with a DSC value of 0.97% for organ recognition and 0.83% for segmentation methods.

1. Introduction

Malignant liver neoplasm is one of the most common types of human cancer and is responsible for the high death rate due to its late diagnosis. Liver disease had the sixth-highest occurrence of all cancers, and the fourth-highest mortality rate in the world, with around 841,000 new cases at a rate of 93 cases per million people and about 782,000 deaths at a rate of 85 cases per million people in 2020 [1]. As tumors can be detected in medical images, the focus of recent research has been shifted towards the segmentation of liver and tumor. It is evident from the fact that the number of studies based on segmentation of liver and tumor has doubled recently [2]. Although the windowed Hounsfield unit values may provide a range of values for the marking of different regions in medical images, identifying voxels that portray liver and lesion regions is challenging due to the low contrast between the liver and the neighbouring organs [3]. Owing to the different sizes, form, and locations of lesions, liver tumor segmentation becomes more complex and challenging in different patients [4]. Furthermore, the margins of some lesions are imprecise, such as hazy, making it hard to detect using edge-based segmentation algorithms. It warrants the need for an algorithm using artificial intelligence that may help medical physicians segment organs and malignancies. In addition, computed tomography (CT) has become one of the most commonly used imaging technologies for identifying and detecting liver tumors because of its excellent spatial precision and speedy laser power. In typical clinical operations, segmentation can be done manually by operators with sufficient competence, but it is time-consuming, and results from different operators are sometimes inconsistent. As a result, developing an automatic segmentation method is extremely difficult, because of the variable forms of tumours, the vast range of tumor intensities, and the uncertainty of tumor and adjacent normal hepatic tissues borders. Over the last few years, semantic segmentation of CT images has become a hot topic of study. Deep learning has significantly enhanced artificial intelligence performance in recent years. In medical imaging, deep learning methods, especially fully convolutional neural network, have surpassed their competitors significantly. In this article, we have implemented ResU-Net architecture for liver and liver tumor segmentation in CT Scan images. The training and testing of the ResU-Net are conducted with selected image pre-processing techniques, Housnfiled windowing unit, histogram equalization and data augmentation methods.

Our contribution to this work includes the implementation of U-Net architecture along with residual blocks. The ResU-Net architecture was trained using selected image pre-processing techniques, Hounsfield Windowing, methods for adjusting the brightness level of CT scan, data augmentation for increasing the number of images and their variations, and a method to tackle the issue of class imbalance. The weighted cross-entropy loss function was also used during network training to address the class imbalance issue. The proposed methodology has obtained better results for liver and liver tumor as assessed by DSC, accuracy, precision, specificity, VOE and RVD values.

The paper is outlined as follows. In Section 2, we have analysed state-of-the-artwork related to the liver and liver tumor segmentation. This section also includes a summary of our contribution. Next, Section 3 presents materials and methods, and Section 4 presents result and discussion. Lastly, Section 5 concludes the paper and suggests further work in this area.

2. Related Work

Over the past decade, researchers have used different machine learning strategies to separate liver and tumors. Li et al. [5] proposed an autonomous 3D liver segmentation approach that uses a total variation with the L1 norm (TV-L1) to detect an initial liver border, followed by a level set method that uses both local and global energy functions. A texture analysis method based on the grey level co-occurrence matrix (GLCM) is used for refined segmentation. Xu et al. [6] proposed a multi-atlas segmentation (MAS) based performance level estimation (SIMPLE) method for automatic segmentation of abdominal organs, including the liver. Song et al. [7] developed an adaptive FMM-based fully automatic liver segmentation algorithm. Self-adaptive parameter adjustment is used in the proposed adaptive FMM. In FMM, the arrival time is adjusted based on the intensity statistics of the potential liver region, which can be used to estimate the size of the liver region on the corresponding computed tomography (CT) slices. Maklad et al. [8] developed a semi-automated method for fragmenting the hepatic low-intensity tumor using CT images that utilized abdominal arterial channels in the entrance stage. Peng et al. [9] proposed a finite difference energy technique that incorporates brightness, region attractiveness, or surface smoothing.

Automatic and robust liver segmentation from CT volumes is challenging due to the low-intensity contrast between the liver and neighbouring organs. Deep neural networks are used extensively in present healthcare image segmentation frameworks [10,11].Tianfei Zhou et al. [12] introduced quality-aware memory network for interactive segmentation of 3D medical images. The memory-augmented network can quickly encode and retrieve segments from the past for segmentation of new slices. Elshaer et al. [13] used two trained deep CNN models to reduce the computational time of a large number of slices. After obtaining the liver segmentation with the first model, the second model was used to avoid the impact of image resampling by removing small missing lesions. Chen Li et al. [14] employed a convolutional neural network (CNN) that used image patches. Each pixel is considered as a patch of the image, with the pixel of interest at its center. It divides the patches into normal or tumor liver tissues. The patch is considered positive if it contains at least 50 percent or more tumor tissue. Hu et al. [15] merged a 2D DenseNet network that combined the extracted intra-slice features and the 3D counterpart for hierarchically aggregating volumetric contexts for liver and lesion segmentation. The liver tumour was segmented by Li et al. [16] using a cascading architecture in which soft and harsh focus techniques and long or short skip linkages were integrated. False positives were reduced by using a joint dice loss function. Jiang et al. [17] developed an edge system by incorporating spatial stream convolution into the U-Net network’s modules. Furthermore, current liver segment techniques are designed for segmenting diagnostic CT pictures and may not even be suitable for segmenting interventional CT images. In addition, CT scans frequently show low gentle brightness, roughness, and other abnormalities. Current tumor segmentation approaches have had mixed results in resolving these challenging problems. As a result, U-Net was used by Chen et al. [18] in conjunction with a recurrent neural block [19] to resolve the limitation of the state-of-the-art models and schemes. The U-Net will use the recurrent neural network to gain relevant data about the hepatic area, aiding in improved liver and hepatic segmentation methods. Hao Xiong et al. [20] implemented probabilistic graphical model for segmentation for fundus images and obtained accuracy of 0.99. Karthik et al. [21] implemented a Fully Convolutional Network (FCN) to better segment the brain MRI having a varying size and shape ischemic lesion and obtained the DSC score of 0.75. Manjunath et al. [22] implemented modified ResU-Net to segment livers and their tumors. The modified system obtained 96.35% DSC and 89.28% accuracy 99.71% and 99.72% for liver and tumour segmentation. Javaria Amin et al. [23] generated synthetic images utilizing a generative adversarial network (GAN). YOLOv3 detector and Resnet block localize the liver in the synthesized images. Moreover, DeeplabV3+ having Inceptionresnetv2 model as a base is implemented for better liver segmentation. Qiangguo Jin et al. [24] implemented a novel network called residual attention-aware U-Net (RA-U-Net) to segment 3D images of liver 3DIRCADb dataset. Moreover, Yodit Abebe Ayalew et al. [25] implemented U-Net to segment liver and liver tumor and obtained a DSC value of 0.96 and 0.74. Sultan Almotairi et al. [26] applied a semantic pixel-wise classification network called SegNet for tumor classification of the liver. The implemented network obtained an accuracy of 99.9% for liver tumor. The liver is an essential organ in the abdominal area, and overlapping the tumor region on the liver may cause trouble automatically segmenting the liver. Some surrounding tissues may also cause boundary problems for the automated liver segmentation task. The residual block in the segmentation model would provide efficient backpropagation gradient capability to train the deep learning model well and produce efficient segmentation results.

However, due to the heterogeneity and variation in shapes of the liver and its potential tumors, segmenting CT images remains challenging, and many efforts have been made to address it.

3. Material and Method

3.1. Liver CT Scan Dataset

The ResU-Net network is evaluated on the 3D-IRCADb01 dataset [27] publicly provided by the IRCAD Digestive Cancer Institute, which included 3D CT scans of 10 females and 10 males with liver cancer among 75 female and male cases. The 3D CT scan is in the shape of DICOM and is divided into 2D slices, and each has 2800 slices with masks for liver, tumor, bone, arteries, kidneys, and lungs. The general pipeline of the method utilized in this study for liver tumor segmentation as shown in Figure 1.

3.2. Image Pre-Processing

In order to distinguish the liver from neighbouring organs, pre-processing of each slice was performed as described in subsequent sub-sections.

3.2.1. Hounsfield Windowing ($HU$)

In CT scans, the tissue density is measured in the Hounsfield unit ($HU$), which is defined as −1000 for air, zero for water and +1000 for bone, while the HU values for different organs and fluids lies in between this range as shown in

Table 1. Hounsfield unit values define for body organ.

HU Value for Body Organ
Bone	1000
Liver	40 to 60
White Matter	46
Grey Matter	43
Blood	40
Muscle	10 to 40
Kidney	30
Cerebrospinal	15
Water	0
Fat	−50 to −100
Air	−1000

. In order to observe specific tissues of interest, a windowing operation is usually performed in which the CT image greyscale component of an image is manipulated via the CT numbers. The brightness of the image is adjusted via the window level. The contrast is adjusted via the window width.

$$HU=1000\times \frac{\mu tissue-\mu H_{2}O}{\mu H_{2}O}$$

(1)

The $HU$ value assigned to each pixel is calculated by giving grayscale intensity to each value, with higher values indicating brighter pixels. Using Hounsfield Windowing, read the slides in DICOM format and set the range [−100, 400] [−100, 400] [28].

In order to enhance the visualization of the liver, HU windowing is performed at each slice where the HU range of −100 to 400 is selected, as shown in Figure 2. It shows that most organs are occluded in CT image slices without windowing operation, while the image and organs became clearer after HU windowing. Therefore, this HU windowing is performed over all slices before passing them to the next process.

3.2.2. Histogram Equalization

Although HU windowing provides good visualization of organs, it was still difficult to differentiate between the liver and adjacent tissues. Therefore, histogram equalization was applied to the image obtained after window processing and then normalized in the range of [0, 1] as shown in Figure 3. It can be seen that as compared with Figure 2A,B, the organs boundaries are more evident in Figure 3C,D after histogram equalization.

3.3. Data Augmentation

In the 3D IRCADb01 dataset, each scan has its own tumor mask that represents the location of the liver and tumor. However, the number of slices containing the tumor is not sufficient for the training process of deep learning. Therefore, data augmentation such as reflection or flip and rotation of combined slice and its mask was performed. Data augmentation will essentially increase the spatial variation of the images and improve the network accuracy [29]. The two data augmentation techniques used are as follows:

3.3.1. Reflection

Reflection is a data augmentation technique in which the image can be flipped across the x or y-axis to generate more samples for network training. In order to avoid the computational expense, the liver images along with its tumor masks are reflected as shown in Figure 4A–F.

3.3.2. Rotation

The rotation method of data augmentation involves rotating the image to a certain degree that can be considered a new image.Here, a 90${}^{\circ }$ rotation of liver images along with its masks are performed as shown in Figure 5A–F.

3.4. Image Normalization

Image intensity normalisation is a technique for reducing inter-patient variability in the intensity distribution of images in a dataset [30]. It will also increase patient image comparability and result in improved segmentation findings. As a result, liver CT scan results are subjected to intensity normalisation.

3.5. ResU-Net Architecture for Liver CT Scan Segmentation

A deep residual U-Net is called the ResU-NET, which is a semantic segmentation encoder-decoder architecture. ResU-Net integrates three-way coding into the encoder-decoder architecture. ResU-Net makes use of both the U-Net and Deep Residual Learning architectures. ResU-Net is a fully convolutional neural network with fewer parameters that aims to achieve excellent performance. It is an improvement over the existing U-Net architecture. The ResU-Net consists of an encoding network, decoding network and a bridge connecting these networks. Like U-Net, the ResU-Net has a gateway, an encoding connection, and a decoding system connecting these networks. The U-Net employs two 3 × 3 convolutions, each with a ReLU activation function. In the case of ResU-NET, these layers are substituted by a residual block that has been pre-activated, as shown in Figure 6. Similarly, the ResU-NET has an encoding network, a decoding network, and a bridge that connects these networks. The U-Net employs two 3 × 3 convolutions, each with a ReLU activation function. In the case of ResU-NET, these layers are substituted by a residual block that has already been activated.

3.6. Loss Function

In this work, the weighted cross-entropy (WCE) loss function is selected for the CNNs because it helps the network to better differentiate between the background and liver pixels and is especially handy when there is class imbalance. The WCE loss function penalizes each class based on the median frequency of each class, which is formulated as follows [31].

$$WCE=-\frac{1}{n}\sum _{i=1}^n{W}_{c,i}\left[T_{i}log{P}_i+(1-T_i)log(1-P_i)\right],$$

(2)

where the sum is run over all training images, n. The variable $P_i$ is the predicted segmentation class, $T_i$ is the target or the ground truth segmentation label and $W_{c,i}$ is the class weight calculated from Equation (2).

3.7. Evaluation Metric

Dice similarity coefficient, accuracy, precision, specificity,and volume overlap error values are used to evaluate the results of liver and tumor segmentation.

3.7.1. Dice Similarity Coefficient ($DSC$)

The dice similarity coefficient (DSC), which evaluates the overlapped results from two independent regions [32,33], was used to examine the liver or liver tumor segmentation of CT scan images. If true positive $TP$ is real positive, $FP$ for false positive, and $FN$ for negative result, the $DSC$ can be represents as,

$$DSC=\frac{\left|2TP\right|}{\left|2TP\right|+\left|FP\right|+\left|FN\right|},$$

(3)

3.7.2. Accuracy

It measures how many positive and negative observations are correctly classified.

$$Accuracy=(TP+TN)/(TP+TN+FP+FN)$$

(4)

3.7.3. Specificity

Specificity measures the rate of liver normal tissue.

$$Accuracy=TN/(TN+FP)$$

(5)

3.7.4. Precision

Precision predicts the the liver tumor tissue.

$$Accuracy=TP/(TP+FP)$$

(6)

3.7.5. Relative Volume Difference (RVD)

$RVD$ represents the volume difference between the segmented image and the ground truth.

$$RVD(A,B)=\left|B\right|-\left|A\right|/\left|B\right|$$

(7)

3.7.6. Volume Overlap Error ($VOE$)

The $VOE$ can be calculated by subtracting the Jaccard coefficient from the value of a unit based on the dissimilarities between the two volumes [34].

$$VOE=100(1-\frac{\left|2TP\right|}{\left|2TP\right|+\left|FP\right|+\left|FN\right|})$$

(8)

where |T| and |P| are the number of elements in the target and prediction sets, respectively.

4. Result and Discussion

4.1. Experimental Set-Up

The code for training and testing the ResU-Net model is developed on python 3.9.1, i7 @ 2.25 GHz 16 core CPU with Ubuntu 20.04 LTC and 18.04 LTS operating system. During the training, the network weight is updated by Adam optimizer [35] with a learning rate selected at 0.0001 and a minibatch size equal to 16. Training of ResU-Net is performed over NVIDIA Tesla T4 GPU Colab powered by Google using TensorFlow and Keras deep learning libraries. The proposed method was tested and validated over synthetic and real-world images.

4.2. Liver Segmentation Results

The qualitative and quantitative result of liver CT scan segmentation is shown in Figure 7 and in

Table 2. ResU-Net model results on liver segmentation.

S.No	Evaluation Metrics	ResU-Net Network
1	DSC	0.9781
2	Accuracy	0.991
3	Precision	0.982
4	Specificity	0.961
5	VOE	0.13
6	RVD	0.018

. The proposed technique has been obtained regarding liver segmentation as assessed by DSC, accuracy, precision, specificity VOE, and RVD values. The Figure 7A,B and Table 2 highlight the better performance of proposed approach.

4.3. Tumor Segmentation Results

The qualitative and quantitative result of liver Tumor CT scan segmentation is shown in Figure 8 and in

Table 3. ResU-Net model results on liver tumor segmentation.

S.No	Evaluation Metrics	ResU-Net Network
1	DSC	0.893
2	Accuracy	0.97
3	Precision	0.950
4	Specificity	0.957
5	VOE	13.15
6	RVD	7.23

. The proposed technique has been obtained in terms of liver segmentation as assessed by DSC, accuracy, precision, specificity VOE, and RVD values. The Figure 8A,B and Table 3 highlight the better performance of proposed approach.

4.4. Comparative Analysis

This article segments liver and liver tumor CT scans using ResU-Net. The localization results demonstrate that the proposed method more accurately localized the very minute liver tumor. The Liver CT scan segmentation is quantitatively evaluated in terms of DSC, accuracy, precision, specificity, VOE, and RVD values averaged over all test images. Due to differences in the dataset, scan processes, and image quantities, performance-based evaluation of the ResU-Net network consisting of DSC, accuracy, precision, specificity VOE, and RVD scores for Liver and tumor detection with some other methods. To evaluate the method, we compare our results with other state-of-art methods.

Table 4. Result comparison of ResU-Net based liver segmentation with state-of-art techniques.

Approach	DSC	Accuracy	Precision	Specificity	VOE	RVD
Christ et al. [31]	0.94	-	-	-	10.7	−1.4
Rafiei et al. [36]	0.935	-	-	-	56.47	-
Li et al. [14]	0.80	-	0.826	-	29.04	-
Qiangguo Jin et al. [24]	0.971	-	-	-	0.074	0.002
Sultan Almotairi et al.  [26]	-	0.988	-	-	-	-
Yodit Abebe Ayalew et al. [25]	0.961	0.993	-	-	-
Proposed ResU-Net	0.9781	0.991	0.982	0.9618	0.13	0.018

and

Table 5. Result comparison of ResU-Net based liver tumor segmentation with state-of-art techniques.

Approach	DSC	Accuracy	Precision	Specificity	VOE	RVD
Christ et al. [13]	0.823	-	-	-	-	-
Sun et al. [39]	-	-	-	-	15.6	5.8
Wu et al. [38]	0.83	-	-	-	29.04	−2.20
Qiangguo Jin et al.  [24]	0.595	-	-	-	0.380	−0.152
Han et al. [37]	0.67	-	-	-	-	0.40
Yodit Abebe Ayalew et al. [25]	0.74	0.995	-	-	-
Proposed ResU-Net	0.893	0.97	0.950	0.957	13.15	7.23

show rough comparability with various state-of-art segmentation methods. The comparison results show that our method gives the best DSC, accuracy, precision, recall, specificity VOE, and RVD scores.

4.4.1. Segmentation of Liver

Table 4 highlight the overall comparison of our network with the state-of-art techniques. Christ et al. [31] utilized spiralled convolutionary neural networks and proposed a method for autonomously segmenting liver and tumors in CT and MRI abdomen scans (CFCNs). During training, the spiralled FCN will divide the hepatic as an ROI, then tumors inside the expected liver ROIs. For the hepatic and liver tumor, the spiralled FCN produced DSC values of 0.943 and 0.56, respectively. Rafiei at al. [36] developed a unique 3D to 2D fully convolutional network (3D-2D-FCN) to improve liver segment. The dice score for the 3D-2D-FCN system was 0.935. Moreover, Li et al. [14] applied CNN to liver segmentation and obtained a DSC score of 0.80. Qiangguo Jin et al. [24] implemented 3D hybrid residual attention-aware (RA-UNet) and obtained a DSC score of 0.971 for liver segmentation. Sultan Almotairi et al. [26] utilized the SegNet network for segmentation of Liver CT scan and obtained an accuracy of 0.988. Yodit Abebe Ayalew et al. [25] introduced a modified U-Net to segment the liver CT scan and obtained DSC scores of 0.961 for liver segmentation.

4.4.2. Segmentation of Liver Tumor

The tumor is separated from the segmented liver in our method. If no liver is in any slice of CT volume, then that slice will not account for the further procedure in tumor segmentation. The comparison of the proposed method with state-of-art techniques is highlighted in Table 5. Christ et al. [13] utilized FCN to segment liver and tumors in CT and MRI abdomen scans (CFCNs). The spiralled FCN produced VOE values of 0.823. The model proposed by Han et al. [37] operate in a 2.5D context, in which the deep learning model based on the DCNN technique yielded a DSC of 0.67. Wu et al. [38] suggested an automated technique for liver tumor delineation in CT volumes based on adaptive fuzzy C-means (FCM) and graph cuts, which resulted in a DSC of 0.83. Sun et al. [39] used a multi-channel convolutionary network (MC-FCN) to separate liver tumors from a CT scan and achieved a VOE of 15.6. Qiangguo Jin et al. [24] implemented 3D hybrid residual attention-aware (RA-UNet) and obtained DSC score of 0.59 for liver tumor segmentation. Yodit Abebe Ayalew et al. [25] introduced a modified U-Net to segment the liver CT scan and obtained DSC scores of 0.74 for liver tumor segmentation. Our method demonstrated high extension and generalization abilities in another tumor segmentation dataset and achieved competitive performance in the liver tumor challenge. The ResU-Net and recurrent neural block performed better segmentation of the hepatic and liver tumour. The U-Net network collects more data with recurrent neural blocks, allowing for a better liver CT scan segment than other state-of-the-art studies. The proposed model has great potential for application to other modalities of medical images. Additionally, it may assist surgeons in finding treatments for novel tumors.

5. Conclusions

In this work, we utilized the ResU-Net model to automatically segment the liver and its tumors into pixels from the liver CT scan images. The proposed technique is one of the best techniques that can be used in the segmentation of the liver and its tumors. It can also be used even though this technique can enhance the result for other research work related to tumors like lung and brain tumors, etc. Researchers are working on a performance-based study of various networks specifically employed for liver and tumours. A wide range of data sets can be counted and used extensively by testing and implementing this approach and enhancing the result time by time. The proposed ResU-Net network in this study outperforms state-of-the-art techniques in the liver and its tumor segmentation.

Future Work

The deep learning-based techniques can improve performance when using large numbers of liver datasets. Additionally, data preparation techniques such as the generative adversarial network (GAN) can be utilised to increase the size of liver images. Although the ResU-Net demonstrated promising results, the authors should try alternative deep learning-based networks in the future to see if they can increase performance further.