A Novel Bio-Inspired Deep Learning Approach for Liver Cancer Diagnosis

Abstract

Current research on computer-aided diagnosis (CAD) of liver cancer is based on traditional feature engineering methods, which have several drawbacks including redundant features and high computational cost. Recent deep learning models overcome these problems by implicitly capturing intricate structures from large-scale medical image data. However, they are still affected by network hyperparameters and topology. Hence, the state of the art in this area can be further optimized by integrating bio-inspired concepts into deep learning models. This work proposes a novel bio-inspired deep learning approach for optimizing predictive results of liver cancer. This approach contributes to the literature in two ways. Firstly, a novel hybrid segmentation algorithm is proposed to extract liver lesions from computed tomography (CT) images using SegNet network, UNet network, and artificial bee colony optimization (ABC), namely, SegNet-UNet-ABC. This algorithm uses the SegNet for separating liver from the abdominal CT scan, then the UNet is used to extract lesions from the liver. In parallel, the ABC algorithm is hybridized with each network to tune its hyperparameters, as they highly affect the segmentation performance. Secondly, a hybrid algorithm of the LeNet-5 model and ABC algorithm, namely, LeNet-5/ABC, is proposed as feature extractor and classifier of liver lesions. The LeNet-5/ABC algorithm uses the ABC to select the optimal topology for constructing the LeNet-5 network, as network structure affects learning time and classification accuracy. For assessing performance of the two proposed algorithms, comparisons have been made to the state-of-the-art algorithms on liver lesion segmentation and classification. The results reveal that the SegNet-UNet-ABC is superior to other compared algorithms regarding Jaccard index, Dice index, correlation coefficient, and convergence time. Moreover, the LeNet-5/ABC algorithm outperforms other algorithms regarding specificity, F1-score, accuracy, and computational time.

1. Introduction

Liver cancer is among the most common causes of death worldwide [1]. In order to raise opportunities for survival by supplying optimal treatments, detecting the presence of liver cancer early is of significant importance. At the current time, biopsy is considered golden standard to detect cancer, although it is uncomfortable, invasive, and does not always represent a viable option, depending on the tumor location [2]. Noninvasive diagnosis of liver lesions could be evaluated by using medical imaging modalities. Computed tomography (CT) is among the most commonly used modalities for detecting, diagnosing, and following up the status of liver lesions, specifically metastases [3]. The images are acquired before and after intravenous injection of a contrast agent with optimal detection of lesions in the portal phase (60–80 s post injection) images. However, current radiological practice is to visually inspect the image of the liver. Visual inspection for an enormous number of medical images can be tedious and time consuming. This task requires the radiologist to search through a three-dimensional CT scan which may include hundreds of slices and multiple lesions, causing human bias and mistakes [4].

For this reason, computer-aided diagnosis (CAD) through computer vision and machine learning [5,6,7,8] has been developed to assist doctors in decision-making through facilitating the interpretation of liver CT scans [3,4,9,10,11,12,13]. However, little work has been conducted on liver cancer diagnosis using CT images. Moreover, most of the proposed solutions in the literature are based on manually crafted features [3,4,9,10,11,12,13]. Different visual descriptors have been tested, including texture [3,9,11,12], shape [9], and a combination of them [3]. The grey level co-occurrence matrix (GLCM) features [3,9,11,12], wavelet coefficient statistics [12], and first-order statistics [9,11] have been used frequently for texture and shape description, while support vector machine (SVM) [9,11,13] and artificial neural networks (ANNs) [11,12] have been used in the classification stage.

Despite the success of these methods, it is usually challenging to design handcrafted features that are optimal for a specific classification task. Furthermore, these methods cannot present discriminative hierarchical feature representations from image data effectively. In recent years, the importance of representation learning in liver cancer diagnosis has been emphasized instead of feature engineering [14,15]. Deep learning [16,17,18] is one type of representational learning technique that can learn abstract mid-level and high-level features from image data. One advantage of deep learning is that it can learn extremely complex patterns. Deep learning algorithms, especially convolutional neural networks (CNNs), use "hidden layers" between inputs and outputs in order to model intermediary representations of image data that other algorithms cannot easily learn. Hence, they can generate high-level feature representations directly from raw medical images.

CNNs [18,19], which are biologically inspired networks, have led to significant contributions in medical image analysis tasks, including organ segmentation [20], texture analysis [19], and disease classification [21]. Several studies have also emphasized that CNNs achieve promising performance in cancer detection and diagnosis [14,22,23]. However, the accuracy of segmenting and classifying tumors using CNN models depends on the network hyperparameters and topology, which also have an impact on the overall performance of the CAD system. Since these parameters affect performance directly, algorithms that take inspiration from natural phenomena [24,25,26,27,28,29,30] can be integrated with deep learning models to select optimal hyperparameters by searching the solution space in a global manner.

Particle swarm optimization (PSO) [24], artificial bee colony optimization (ABC) [25], differential evolution (DE) [26], harmony search (HS) [27], gravitational search (GS) [28], grey wolf optimization (GWO) [9], antlion optimization (ALO) [29], and ant colony optimization (ACO) [30] are a few of the popular algorithms of this class. Other works have demonstrated that each of these algorithms can be considered an effective solver of complex optimization problems in the medical domain [31,32]. The ABC optimization algorithm as an instance represents a prominent candidate among effective optimization algorithms. It is inspired by the intelligent foraging behavior of honey bees and is able to share information [32].

Contrary to state-of-the-art systems which are based on using feature engineering methods, or hybrids of feature engineering and deep learning algorithms, this work presents a fully bio-inspired deep learning approach for liver cancer diagnosis using CT images. The effect of hybridizing multiple deep learning models with ABC bio-inspired optimization is investigated in the segmentation, feature extraction, and classification of liver lesions. The main contributions of the paper include the following:

• An extensive survey is introduced to discuss current state-of-the-art methods for diagnosing both other cancers and liver cancer. Also, recent applications of bio-inspired methods in the optimization of medical domain problems are reviewed.
• A novel hybrid segmentation algorithm, namely, SegNet-UNet-ABC, is proposed for extracting liver lesions from CT images using the SegNet network [33], the UNet network [20], and ABC. In this algorithm, the SegNet network is used for extracting liver from the abdominal CT scan, while the UNet network is used to extract lesions from the liver. In parallel, the components of ABC bio-inspired optimization are integrated with each deep learning network to adjust its hyperparameters, as they highly affect the segmentation performance [34]. These parameters include the learning rate, minibatch size, momentum, maximum epochs, shuffle, and regularization. Hence, this hybridization can provide near-optimal segmentation results in comparison to state-of-the-art algorithms for liver lesion segmentation.
• Furthermore, to investigate the efficiency of the ABC algorithm in optimizing segmentation of liver lesions appearing on CT images when it is used as a hybrid with SegNet and UNet architectures, extensive comparisons are made to other bio-inspired optimization algorithms, including GWO, ALO, and ACO. Therefore, this work compares the performance of the proposed SegNet-UNet-ABC algorithm with that obtained by hybridization of SegNet-UNet with GWO (SegNet-UNet-GWO), SegNet-UNet with ALO (SegNet-UNet-ALO), and SegNet-UNet with ACO (SegNet-UNet-ACO). A detailed performance comparison is reported.
• Moreover, a hybrid algorithm of the LeNet-5 deep learning model [35] and the ABC algorithm, namely, LeNet-5/ABC, is proposed as a feature extractor and classifier of liver lesions. The reason for this hybridization is that the hyperparameters mainly determine the layer architecture, i.e., the size of resulting feature map, in the feature extraction step of the LeNet-5 network, which affects the learning time and classification accuracy. Therefore, the ABC algorithm is used to determine the optimal topology for constructing the LeNet-5 model by selecting the best values of kernel size, padding, stride, and number of filters applied at each convolution and pooling layer. This, in turn, can optimize the classification part in the LeNet-5 model by reducing classification error and minimizing the probability of being trapped in local optima.

2. Related Work

By reviewing the state of the art of cancer diagnosis using image modalities, we find that the contributions can generally be divided into feature-engineering-based CAD methods and deep-learning-based CAD methods. Hence, this section presents an overview on utilizing these methods in diagnosing cancers, including liver cancer. Furthermore, this section sheds light on using bio-inspired methods in optimizing medical diagnosis.

2.1. Feature Engineering Methods for Diagnosis of Cancers Generally and Liver Cancer Specifically

For feature engineering approaches, several studies have been introduced to diagnose either other cancers or liver cancer. In [36], a texture descriptor was introduced for representing rich texture features through the integration of multiscale Gabor filters with local binary pattern histograms for the classification of lung tissue. In [37], the authors introduced a CAD system for thyroid cancer using internal and external characteristics, where geometric and textural features were extracted. Further, multilayer perceptron was utilized for classifying internal characteristics, whereas SVM was utilized for classifying external characteristics.

As for liver cancer diagnosis, many studies have proposed feature engineering methods embedded into CAD systems by using CT scan images [3,9,10,11,12], as illustrated in

Table 1. Review on feature engineering methods utilized in liver cancer diagnosis from computed tomography (CT) images.

Reference	Year of Publication	Approach	Performance Measure
[12]	2013	Algorithm of confidence connected region growing is utilized for liver extraction, and clustering algorithm of alternative fuzzy C-means (FCM) is utilized for segmenting tumor. Feature extraction is based on four feature sets: wavelet coefficient statistics, grey level co-occurrence, original gray level, and contourlet coefficient statistics. Probabilistic neural network is employed for tumor classification.	Highest accuracy = 96.7%. Highest specificity = 97.3%.
[9]	2016	Hybrid algorithm integrating fuzzy clustering with grey wolf optimization is used for liver segmentation. 16-dimensional vector of shape statistical features (comprising median, area, mean, kurtosis, standard deviation and skewness) together with texture features taken by GLCM is extracted. SVM is employed for tumor classification.	Highest accuracy = 97%.
[3]	2017	Region growing algorithm is employed for tumor segmentation. Texture, shape, and kinetic curve are then extracted from tumor. Three-dimensional (3D) texture is represented by GLCM. The 3D shape is described by margin, compactness, and elliptic model. From every tumor phase, a kinetic curve is taken to represent density variations between phases. Binary logistic regression analysis is employed for tumor classification.	Highest accuracy = 81.69%. Highest sensitivity = 81.82%. Highest specificity = 81.63%.
[10]	2017	14 high-level local and global features are extracted from CT images to describe focal liver regions (such as center location and Intensity diversity of liver lesion). Three-way rules are used for CT image classification.	Highest accuracy = 91.71%. Highest precision = 100%. Highest recall = 88.52%. Highest F1-score = 93.84.
[11]	2018	Statistical features comprising first-order statistics together with 13 GLCM features are estimated from the intended region of interest. Binary particle multi-swarm heterogeneous optimization using the win-win approach is used for feature selection. Probabilistic neural network and SVM are employed as classifiers.	Accuracy = 82.86%, for both probabilistic neural network and SVM.

. In the segmentation phase, the region growing algorithm and fuzzy C-means (FCM) are popular algorithms that have been frequently employed for either liver or lesion segmentation [3,12]. For feature extraction, the majority of liver CAD systems have used statistical features for describing texture and shape [3,9,11,12], including GLCM features; wavelet coefficient statistics; and statistical measures of mean, skewness, variance, standard deviation, and kurtosis. In this context, feature extraction has played a crucial role in the liver CAD system as it heavily affects overall performance. In the classification phase, conventional linear and nonlinear machine learning algorithms have been used, including probabilistic neural networks [11,12], SVM [9,11], and binary logistic regression [3].

2.2. Deep Learning Methods for Diagnosis of Cancers Generally and Liver Cancer Specifically

Recently, applications of the deep learning have emerged generally in medical image analysis using a variety of image modalities. Segmentation, feature extraction, and classification are the three most basic tasks that deep learning algorithms have been investigated in. For instance, these algorithms have been widely investigated with different anatomical structures (organs or body locations) for medical image analysis, including breast [38], prostate [39], heart/cardiac [40], carotid [41], thyroid [42], intravascular [43], fetus [44], lymph node [45], spine [46], bone [47], muscle [48], tongue [49], and more. Different kinds of deep networks have been adopted to do these tasks.

For state-of-the-art systems using deep learning approaches in the diagnosis of cancers [14,50,51], including liver cancer [15,52,53,54], demonstrated in

Table 2. The current state-of-the-art deep learning approaches for diagnosing cancers, including liver cancer.

Reference	Year of Publication	Cancer Type	Approach	Performance Measure
[52]	2014	Liver cancer	Authors used contrast-enhanced ultrasound (CEUS) imaging, taken from an unpublished dataset for Huazhong University of Science and Technology. Feature exaction is based on sparse non-negative matrix factorizations. A deep belief network is used for classification purpose.	Results of accuracy, sensitivity, and specificity are 86.36%, 83.33%, and 87.50%, respectively.
[50]	2017	Breast cancer	Authors used the Histopathological image database, from Motic (Xiamen) Medical Diagnostic Systems Co. Ltd. For extracting features, nucleus-guided method is used. A CNN with three hierarchy structures is used for classification of breast cancer.	Accuracy = 96.4%
[53]	2018	Liver cancer	Deep multilayered group algorithm of data handling (GMDH)-type neural network based on revised heuristic self-organization.	Prediction error criterion, defined as (Akaike’s information criterion, AIC), is used. Resulting values of AIC are decreased in comparison to ordinary deep network architecture with several hidden layers.
[51]	2019	Lung cancer	Authors used CT images, taken from a publicly available database for lung nodules: the LIDC-IDRI. An object detection framework is presented, which is based upon faster region-based CNN, namely, R-CNN. 2D CNN is used for lung cancer classification.	Sensitivity of detecting nodule candidates is 86.42%, whereas sensitivity of reducing false positives (FPs) is 73.4% for 1/8 FPs/scan, and 74.4% for 1/4 FPs/scan.
[14]	2019	Breast cancer	Authors used mammogram images from the Mammographic Image Analysis Society (MIAS). Feature-wise data augmentation is used with CNN for breast cancer classification.	Results of accuracy, sensitivity, specificity, and area under receiver operating characteristic curve (AUC) are 90.50%, 89.47%, 90.71%, and 0.901 ± 0.0314, respectively.
[15]	2019	Liver cancer	Some geometrical, statistical, and textural features are extracted from images, segmented by a Gaussian mixture model. Deep neural network is utilized as the classifier.	Accuracy = 99.38%
[54]	2019	Liver cancer	End-to-end approach of deep learning incorporating feature extraction of the InceptionV3 integrated with residual connections, and pretrained weights of ImageNet. Fully connected layers are integrated as a classifier to provide a probabilistic output of liver lesion type.	Accuracy = 0.96 F1-score = 0.92

, the majority of works have applied CNN models to learn from the image modalities and hierarchical abstract representations, followed by a softmax layer or other linear classifier (such as SVM) that is used to provide one or more probabilities or class labels. To date, the majority of these works have neglected the effect of network hyperparameters on overall performance. Until now, few solutions have been introduced to optimize the performance of segmentation, feature extraction, and classification using ordinary deep learning architectures [55].

2.3. Bio-Inspired Optimization in Medical Diagnosis

As with most neural networks, deep learning architectures are susceptible to problems such as lack of hyperparameter tuning, multiple local optima, and increased computational time. To avoid these problems, optimization of the network topology and hyperparameters has become a crucial task. The bio-inspired optimization algorithms are broadly utilized in general optimization problems in the medical domain, as demonstrated in

Table 3. Recent state-of-the-art bio-inspired optimization approaches utilized in medical diagnosis.

Reference	Publication Year	Medical Application Domain	Bio-Inspired Optimization Approach	Performance Results
[32]	2018	Ultrasonic echo estimation utilizing ultrasonic and simulated signals.	ABC-optimized matching pursuit approach, referred to as ABC-MP.	Results of energy error, amplitude error, coefficient error, and residue signal energy are 10.94%, 1.93%, 0.52%, and 12.05%, respectively.
[56]	2018	Selecting cancer progression pathway genes from distinct cancer datasets.	A gene selection approach, namely, MFDPSO-BLABC, utilizes bi-stage hierarchical swarm and integrates (1) a feature selection procedure with discrete particle swarm optimization of multiple fitness functions (MFDPSO) and a multi filtering-enabled gene selection technique, and (2) the blended Laplacian ABC algorithm (BLABC) for clustering genes selected by the first procedure.	Classification accuracies of the MFDPSO-BLABC approach with SVM are 0.99, 0.79, 0.97, 0.94, 0.93, and 0.86 for the following cancer datasets: Leukemia, Colon, Gastric, DLBCL, Prostate, and Child_ALL (related to childhood cancer), respectively.
[57]	2017	Microarray cancer classification.	An approach encompassing genetic algorithm (GA) with ABC algorithm. The proposed algorithm is executed on the microarray gene expression profile to choose the most informative and predictive genes for classifying cancer.	Classification accuracies obtained using SVM are 90.32%, 93.05%, 97.91%, 77.11%, 86.36%, and 84.72%, respectively, for the following datasets: Colon, Leukemia1, Lung, SRBCT, Lymphoma, and Leukemia2.
[31]	2016	Classification of DNA microarrays by identifying distinct classes associated with a specific disease.	The ABC algorithm is used to select gene sets from a DNA microarray characterizing a specific disease. Three classifiers are trained with the resulting information to classify DNA microarrays associated with disease: multilayer perceptron network (MLP), SVM, and radial basis function (RBF).	The optimized MLP and SVM outperformed the optimized RBF in terms of classification accuracy.
[58]	2015	Retinal blood vessel localization from retinal images.	An approach based on two levels of clustering: (1) the ABC optimization together with a fuzzy clustering compactness fitness function, used to determine coarse vessels, and (2) pattern search, employed to optimize the segmentation outcomes.	The results of sensitivity, specificity, and accuracy are 0.721, 0.971, and 0.9388, respectively.
[59]	2013	Reducing bioinformatics data dimension for solving classification problems.	The ABC is used for selecting an optimal subset of dimensions among high-dimensional data while keeping a subset which achieves the defined objective. Further, the fitness of ABC is assessed by k-nearest neighbor.	Average accuracy = 93.75%.
[60]	2013	Diagnosing diabetes disease using a diabetes dataset.	A modified version of ABC is introduced, which is different from the ordinary ABC in one point, if no optimization in fitness function is occurred, blended crossover operator for GA is used for further exploration and exploitation. This version is used as a tool to build a fuzzy-rule-based classifier with no prior knowledge.	Classification rate = 84.21%.

[31,32,56,57,58,59,60], including ultrasonic echo estimation [32], selecting cancer progression pathway genes [56], microarray cancer classification [57], classification of DNA microarrays [31], retinal blood vessel localization [58], bioinformatics data dimension reduction for solving classification problems [59], and diabetes disease diagnosis [60]. However, these algorithms have been rarely used to optimize the results of segmentation, feature extraction, and classification obtained using a CNN. Therefore, this work investigates the effect of hybridizing multiple deep learning networks with bio-inspired optimization on the segmentation, feature extraction, and classification of liver lesions from CT images.

3. Materials and Methods

A brief explanation is given in this section on materials and methods employed in current work including the dataset, performance measures, CNN, bio-inspired ABC algorithm, and the proposed approach.

3.1. Datasets of Liver CT Images

The approach proposed in this work was tested using two publicly available datasets. Firstly, we used the dataset tested in [20,61], namely, LiTS, which comprises 131 CT scan images along with their ground truths (clinical annotation). The LiTS dataset also includes a set of 70 CT images for testing purposes, but this does not have any accompanying annotations. Hence, only the 131 annotated CT images were considered in this work. Secondly, we used the Liver CT dataset that was tested in [62], namely, Radiopaedia. This dataset is a complex one which includes abdominal CT images for the liver taken from more than 105 patients. Furthermore, more than 150 slices for each patient are included. The images are all available in JPEG format, obtained from a DICOM file of dimension 630 × 630 and bit depth of 24 bits.

3.2. Performance Measures

For confirming adequacy of liver lesion segmentation by the proposed hybrid algorithm and also to compare it to the state-of-the-art algorithms, this work implements a quantitative analysis using three indices: the Jaccard index [15,62], Dice coefficient [15,20,61,62], and correlation coefficient [62,63]. Furthermore, convergence time [64] is another criterion used to evaluate the quality of segmentation using hybrid methods.

Jaccard index: This similarity index is a popular measure used for binary data as given below, where $AOO$ represents the area of overlapping, $M$ is a binary image, and $K$ is a ground truth image.

$$J(AOO)=\frac{M\cap K}{M\cup K}$$

(1)

Dice Index: This coefficient is utilized to measure segmentation performance. The value of the Dice coefficient describes the percentage of pixels in the predicted image which exactly match the ground truth. This measure is computed by Equation (2).

$$D(M,K)=\frac{2\left|M\cap K\right|}{\left|M\right|+\left|K\right|}$$

(2)

Correlation coefficient: This measure is used to compute similarity of segmented image to ground truth, in terms of their respective pixels’ intensity. This coefficient is defined by Equation (3), where the indices $a$ and $b$ represent the locations of pixels in liver CT image.

$$CC=\frac{{\displaystyle {\sum }_a{\displaystyle {\sum }_b\left(M_{ab}-\mathrm{mean}(M)\right)\left(K_{ab}-\mathrm{mean}(K)\right)}}}{\sqrt{\left({\displaystyle {\sum }_a{\displaystyle {\sum }_b{\left(M_{ab}-\mathrm{mean}(M)\right)}^2}}\right)\left({\displaystyle {\sum }_a{\displaystyle {\sum }_b{\left(K_{ab}-\mathrm{mean}(K)\right)}^2}}\right)}}$$

(3)

Convergence time: The convergence time of each hybrid segmentation algorithm is expressed as the time taken by the algorithm versus the false alarm rate (FAR), computed by Equation (4).

$$FAR=\frac{FalsePositive}{FalsePositive+TrueNegative}$$

(4)

On the other side, three measures were used as validation metrics for testing performance of proposed LeNet-5/ABC liver cancer classification algorithm.

Specificity: The specificity represents the true negative rate that is given using Equation (5) [65].

$$SP=\frac{TrueNegative}{TrueNegative+FalsePositive}$$

(5)

F1-score: This measure expresses the harmonic average of both precision and recall [64], which is computed by Equation (6).

$$F1-score=2\frac{(Precision\ast Recall)}{(Precision+Recall)}$$

(6)

Accuracy: The accuracy is expressed as the probability of obtaining a true prediction [65], which can be computed using Equation (7).

$$Accuracy=\frac{TruePositive+TrueNegative}{TruePositive+TrueNegative+FalsePositive+FalseNegative}$$

(7)

Computational time (in seconds): This is used for assessing quality of classification obtained by each classifier.

3.3. Convolutional Neural Networks

To demonstrate how the CNNs work [14,15,16,17,18,19,20,21,22,23], it is required to understand the architecture of the basic ANNs, which represent human-brain-inspired architectures widely utilized in machine learning. ANN comprises three basic layers: the first is input one, the middle is hidden, and the final is output layer. The set of characteristics which represents the class that the ANN has to learn is received by the input layer. Further, input data processing is performed by the hidden layer through recognizing patterns to give identical or approximate value for the class that has to be recognized by the output layer. As depicted in Figure 1 [19], this process is expressed as feed-forward. If the output is not matched to the correct class, a back-propagation process is performed by the ANN for adjusting the connection weights of the corresponding hidden layers according to the calculated error, allowing correct class recognition based on repetitive learning iterations.

A recent alternative to ANN for big data, including images, is CNN. The basic difference between CNNs and ANNs is the convolution and pooling layers adopted in the former to extract image characteristics more effectively using fewer dimensions. A CNN passes a given image through its layers and outputs the decision class. The network may comprise tens or hundreds of layers, where each layer learns to detect different feature kinds. Each training image is subjected to filters at different resolutions, then the output of every convolved image is given as input to subsequent layer. The basic architecture of CNNs is depicted in Figure 2 and includes the following layers.

3.3.1. Convolutional Layer

The convolution represents particular kind of linear operation, in which the image matrix is subjected to a kernel. Figure 3 presents an instance on convolutions, e.g., a filter application to the given image. Figure 3a presents the matrix of image that will be filtered, the kernel or filter is presented in Figure 3b, and the convolution result is presented in Figure 3c. The filter depicted in Figure 3b successively reads from the left to right, besides to, top to bottom, where all pixels in the area of kernel action are within the gray area of the matrix shown in Figure 3a. Subsequent to this operation, pixel value 16 in the array of Figure 3a becomes 17 in the matrix of Figure 3c due to the convolution. This operation is computed as follows: $19\times 0+12\times 0+11\times 0+17\times 1+16\times 0+18\times 0+13\times 0+17\times 0+14\times 0=17$.

3.3.2. Rectified Linear Unit Layer

Rectified linear unit, referred as ReLU, comes after the convolution layers where feature maps are fed to nonlinear activation functions. Accordingly, the whole neural network becomes able to approximate nonlinear functions [14]. The activation function generally represents a simple ReLU, defined as in Equation (8). The ReLU function swaps all negative states with zeros. At the same time, feeding the resulting feature maps to the activation function generates new tensors, termed as feature maps. Figure 4 demonstrates an example of ReLU operation.

$$A_{xyz}=\max \{0,A_{xyz}\}$$

(8)

3.3.3. Pooling Layer

The pooling layers aim at reducing the parameter number of big image data. To this end, each feature map generated through feeding the data to single or multiple convolutional layers is then pooled within a pooling layer. The pooling operations obtain small grid segment as input and generate singular number for every segment. This is known as subsampling or downsampling, in which dimensionality of every map is minimized while retaining important information. There are different types of spatial pooling, comprising (1) max-pooling; (2) average-pooling; and (3) sum-pooling. Max-pooling obtains the largest value of the considered rectified feature map. Obtaining the average of elements in the feature map is referred as average pooling, while taking their sum is named sum-pooling. Figure 5 demonstrates the max-pooling operation used in this work with different filters and stride values. The stride refers to number of shifts in pixels over the input image matrix. When the stride value is 1, filters are shifted one pixel at a time. When the value is 2, filters are shifted two pixels at a time, and so on.

3.3.4. Fully Connected Layer

Fully connected layer (FCL) represents the decision layer in a CNN. The softmax function is utilized to compress the outputs of every neuron to be between 0 and 1. It acts similarly to sigmoid function. The FCL divides every output as if the total output sum is equal to one. The produced output represents the categorical probability distribution. The FCL computes a probability that a class is true, as follows, where $o$ denotes the input vector to the output layer. If the number of output units is equal to ten, then there will be ten elements in $o$. $h$ indexes the units of outputs, so that $h=1,2,\dots ,K$.

$$\sigma {(o)}_h=\frac{e^{o_h}}{{\displaystyle {\sum }_{k=1}^K{e}^{o_k}}}$$

(9)

3.4. Artificial Bee Colony Optimization

In ABC [31,32,56,57,58,59,60] meta-heuristics, artificial bees of the colony cooperate to find optimal solutions to the optimization problem. One important feature of ABC is it being inspired by nature, exactly, by honey bees’ behavior seeking a good-quality food source. The essential components of ABC which are modeled after bees’ foraging process are demonstrated as follows: (1) food source, which refers to a feasible solution for the optimization problem; (2) fitness value, which represents food source quality and is expressed as single quantity associating to objective function for the feasible solution; and (3) the bee agents, which represent a group of computational agents. For the algorithm of ABC, the colony is divided equally into three kinds of honey bees: employed bees, onlooker bees, and scout bees. The solution within the search space encompasses parameter set that represents food source location. The count of employed bees equals to the count of food sources, where one employed bee is specified for one food source. Basic steps of ABC optimization are illustrated below.

3.4.1. Initialization

The ABC algorithm begins with the random choice of a food source corresponding to a potential solution. Equation (10) is used to produce initial solutions for employed bees, where $S_{ij}$ represents the $jth$ dimension for the $ith$ food source’s employed bee; $S_j^{\min }$ and $S_j^{\max }$ denote the lower and upper bounds of the $jth$ parameter, respectively; $\mu$ is a random number that falls in $[0;1]$. $I$ is the food source number, i.e., the swarm size, while $J$ refers to the dimensionality of the problem considered for optimization. Resetting of abandonment counter (AC) is also done for every employed bee in the initialization phase.

$$S_{ij}=S_j^{\min }+\mu (S_j^{\max }-S_j^{\min }),i=1,2,\dots ,I,j=1,2,\dots ,J$$

(10)

3.4.2. Employed Bees Phase

For every employed bee, a new solution is immediately produced in this phase. Firstly, the employed bee solution is copied to the candidate new solution ($D_i$ = $S_i$). Then, Equation (11) is used to update the solution parameters:

$$D_{ij}=\psi S_{ij}+\varphi (S_{ij}-S_{mj}),i,m\in \{1,2,3,\dots ,E\},j\in \{1,2,3,\dots ,Z\}andi\ne m$$

(11)

where the $jth$ parameter is randomly selected to be updated and the $\psi$ coefficient is obtained as unity in the basic ABC algorithm. Such process is done through randomly choosing a candidate $S_i$ in the neighborhood of the $ith$ candidate. Also, $\varphi$ is a random number falling in the interval $[-1,1]$, $E$ indicates the count of employed bees, and $Z$ represents the dimensionality of the problem considered for optimization. After locating a candidate solution and computing the objective function value, the fitness values for candidate solutions and employed bees’ solutions are computed as shown below:

$$\{\begin{array}{ll}{F}_i=\frac{1}{1+B_i}\hfill & If{B}_i\ge 0\hfill \\ F_i=1+abs(B_i)\hfill & Otherwise\hfill \end{array}$$

(12)

where $F_i$ indicates fitness value for $ith$ candidate solution and $B_i$ represents the objective function value for the $ith$ employed bee. Subsequently, if the resulting fitness value of an updated solution is superior to that of the present solution, then the present solution is replaced by the candidate solution, while the AC of an employed bee is readjusted to zero; otherwise, the abandonment counter is immediately increased by 1.

3.4.3. Onlooker Bees Phase

To enhance the solution, every onlooker bee chooses an employed bee. In this context, the Roulette wheel is utilized to compute the selection probability for the $ith$ employed bee as follows:

$$R_i=\frac{F_i}{{\displaystyle {\sum }_{j=1}^Z{F}_j}}$$

(13)

where $P{R}_i$ symbolizes the selection probability of the $ith$ employed bee. Accordingly, the solution of the chosen employed bee is optimized by the onlooker bee, according to Equation (11). If the resulting fitness value for the new solution, located by onlooker bee, is superior to that by the employed bee, then the latter replaces the onlooker bee and the AC of an employed bee is immediately readjusted to zero; otherwise, the abandonment counter is incremented by 1.

3.4.4. Scout Bee Phase

A predefined limit is used in this phase to check the AC of every employed bee. Any employed bee that fails to optimize the solution prior to limit is met, will be considered a scout bee. Thereafter, the solution of the scout bee is produced using Equation (10) and the AC is immediately reset. Accordingly, the scout bee is considered an employed bee. Hence, scout bees also prevent the employed bees from stagnating.

3.5. The Proposed Approach

A new methodology for liver cancer diagnosis using CT images is proposed in this section. Instead of using the conventional feature engineering methods which are designed to be suitable for specific medical pattern recognition problems, the approach proposed in this work is a fully deep learning one. However, the more complex a deep learning method is, the more computational time it demands in order to perform at an acceptable pace. To overcome this problem, in this work we investigate the effect of hybridizing multiple deep learning networks with bio-inspired algorithms to optimize the segmentation, feature extraction, and classification of liver lesions. The proposed liver cancer diagnosis approach is demonstrated in Figure 6 and includes three stages. The first stage is preprocessing of liver CT images. A proposed hybrid algorithm is used in the second stage to segment liver lesions from the CT images, based on the SegNet network, UNet network, and ABC algorithm. In the third stage, a proposed hybrid LeNet-5/ABC algorithm is used as a feature extractor and classifier of the liver lesions into benign and malignant.

3.5.1. Preprocessing of CT Images

In this phase, the liver CT image is firstly converted into grayscale then resized to a size of $128\times 128$ as the SegNet network receives inputs of this size. The noise is removed using a median filter, whereas contrast is enhanced by histogram equalization. Then grayscale image of the liver is smoothed, enhanced, and denoised by a median filter algorithm of $3\times 3$ window size. The filter runs over every element of the CT image and replaces every pixel by the median value of its neighborhood pixels located in the square neighborhood surrounding the evaluated pixel. Equation (14) demonstrates using the histogram equalization for modifying the dynamic range of each intensity value and increasing CT contrast without an effect on structure of information included therein. $\mathrm{CDF}$ is the cumulative distribution function of unique pixel value $a$, and $M$ represents the grey level number used for an image of size $k\times l$. Figure 7 shows preprocessing results on some CT slices.

$$Hist(a)=round\left(\frac{\mathrm{CD}\overline{\mathrm{F}}(a)-\mathrm{CD}{\overline{\mathrm{F}}}_{\min }}{(k\times l)-\mathrm{CD}{\overline{\mathrm{F}}}_{\min }}\times (M-2)\right)$$

(14)

3.5.2. The Proposed Hybrid SegNet-UNet-ABC Algorithm for Liver Tumor Segmentation

CNN has different architectures for segmentation, feature extraction, and classification problems. SegNet and UNet have recently been used for semantic segmentation purposes. However, the network hyperparameters have a direct effect on the segmentation accuracy. This requires the optimization of hyperparameters in order to obtain near-optimal segmentation results. Hence, hyperparameter selection through recent effective optimization algorithms is necessary. These algorithms can search the solution space efficiently in a global way. For optimizing liver lesion segmentation from CT images, we propose a hybrid algorithm, namely, SegNet-UNet-ABC, which integrates SegNet and UNet deep learning architectures with ABC optimization for segmenting livers from abdominal CT images and lesions from liver tissue. The proposed hybrid segmentation algorithm is depicted in Figure 8 and includes the following:

A. Liver segmentation from the abdominal CT image using the SegNet network.

The abdominal CT scan includes other organs in addition to the liver. Therefore, liver extraction is a critical task to achieve accurate cancer diagnosis. In this work, the CNN is used to accomplish this task, wherein the SegNet architecture is employed. This architecture has shown robustness in pixel-wise semantic segmentation tasks [33]. The SegNet network encompasses an encoder–decoder architectural engine that is ended at a pixel-wise classification layer, as shown in Figure 8.

The encoder section of the SegNet architecture comprises a repeating group of layers. Each group is constituted of some convolutional layers that are followed by a layer of max-pooling. This portion reveals the first 13 convolutional layers in the VGG16 [65] architecture. The role of the convolutional layer is to produce the required number of feature maps through the convolution process of input by a filter bank. Thereafter, resulting feature maps are patch normalized [20]. Next, the ReLU process, namely, the pixel-wise operation, is implemented, where the output represents $\max (0,k)$. In this context, a max-pooling layer is used to perform downsampling by 2 through defining a window with size $2\times 2$ and using a stride value of 2. The max-pooling is crucial for the SegNet to fulfill translational invariance. However, it exhibits loss of the boundary details, which is unfavorable in the segmentation process. To overcome this issue, the boundary information is sorted within feature maps of the encoder before implementing the max-pooling process. Practically, the index of the pixel with the maximum value is kept within the feature maps of every pooling window.

For the decoder section of the SegNet architecture, the layers are organized in parallel to the encoder but in reverse order. The memorized max-pooling locations are firstly used to upsample the input maps and to present a sparse feature map. The convolutions are then used to produce a dense feature map using the filter banks of the decoder. In the same manner as the encoder section, the batch normalization is implemented after the convolution operation. At the final layer of the decoder, the pixel information is fed into the output layer using the Softmax activation function. The predicted segmentation is achieved through classification of every pixel to a corresponding class.

B. Lesion segmentation from the liver tissue using the UNet network.

This step is crucial to extracting lesions from liver tissue to be analyzed later. For this purpose, the UNet architecture is used. The UNet architecture has demonstrated good results when applied to biomedical images [66]. As depicted in Figure 8, the input layer receives the liver image in the form $128\times 128\times 1$. Furthermore, the UNet architecture comprises three parts. The downsampling path is the first part, in which we can find two convolutional layers which are followed by a max-pooling layer of $2\times 2$ window size and a stride value of 2. The input liver image is convolved twice using a filter of size $3\times 3$ followed by a ReLU activation function. Padding value is retained the same because the output image will have the same size as the input image. The filter number in the convolution layer of the first group is set to eight and continues doubling until reaching the fifth layer group.

Thereafter, an upsampling path is followed where the feature maps of each group are halved [16]. In this regard, the UNet architecture uses a concatenation layer to concatenate the features from both the previous layer and the downsampling layer, which has the same number of filters as the current layer group. Following this, there are two layers with a convolutional filter of $3\times 3$, followed by a ReLU activation function. This group of layers is repeated starting from group six to group nine. The output layer is the tenth, which is a convolutional one with $1\times 1$ filter [66] and has eight feature channels. To sum up, 27 layers [20] are involved in this architecture (18 convolutional + ReLU, 4 pooling, 4 up-convolutional, and 1 softmax layer).

C. Optimization of segmentation performance using the ABC algorithm.

Solution vectors are firstly generated by the ABC algorithm. Each generated vector comprises all possible values of hyperparameters to be optimized. These values are then employed as training parameters during training process of SegNet network. The fitness value for each hyperparameter vector produced by the ABC is evaluated using Equation (15). This is implemented by computing the contour matching score ($C-score$) between the predicted image $P$ and ground truth image $G$. Accordingly, the optimal solution for liver segmentation from the abdominal CT scan will be the one that increases the F1-score, precision, and recall. The three weights $w_f$, $w_p$, and $w_r$ were used to define the F1-score, precision, and recall, respectively.

$$Fitness=C-score(P,G)$$

(15)

$$C-score(P,G)=w_f(F1-score)+w_p(Precision)+w_r(Recall)$$

(16)

The precision and recall are computed as follows, while F1-score is computed by Equation (6).

$$Precision=\frac{TruePositive}{TruePositive+FalsePositive}$$

(17)

$$Recall=\frac{TruePositive}{TruePositive+FalseNegative}$$

(18)

As demonstrated in Algorithm 1, the optimal hyperparameters for liver segmentation are computed using Algorithm 2. The best solution is returned by Algorithm 2 when the termination criterion is satisfied ($cycle\le MCN$), which is achieved when the number of iterations reaches the threshold value. Thereafter, the resulting solution is passed to Algorithm 1, where training is implemented afresh using the produced hyperparameter values, the SegNet deep learning architecture, and the training set ($SegNe{t}_{Training}$), as shown in Steps 6 and 7. After that, the trained SegNet network is tested using the testing set ($SegNe{t}_{Testing}$) to validate the liver segmentation performance, as shown in Step 8.

The previous steps are done for the UNet network as demonstrated in lines 10 to 19 of Algorithm 1, where the resulting set of segmented liver images is divided to training ($UNe{t}_{Training}$) and testing ($UNe{t}_{Testing}$). The training set of UNet ($UNe{t}_{Training}$) is then divided to training ($Trainin{g}_{FP}$) and validation ($Validation{}_{FP}$). Thereafter, the $Trainin{g}_{FP}$, $Validation{}_{FP}$, and UNet structure are sent as functional parameters of Algorithm 2 of the ABC, which computes the hyperparameters that optimize lesion segmentation using UNet. The resulting optimized parameters are embedded into the UNet network. Then, the network is trained using the original training set ($UNe{t}_{Training}$) and tested on ($UNe{t}_{Testing}$), as demonstrated in Steps 17 and 18 of Algorithm 1. Figure 9 demonstrates the liver segmentation results using SegNet and the UNet predictions of liver lesions. The optimized hyperparameter values selected by the ABC algorithm for segmentation using the SegNet and UNet networks, are shown in

Table 4. The optimized hyperparameters for segmenting livers and liver lesions using SegNet and UNet, respectively.

Hyperparameter	Optimized Values
	SegNet	UNet
Initial learning rate	0.01	0.05
Minibatch size	11	16
Momentum	0.9	0.9
Maximum epochs	30	150
Shuffle	Once	Every epoch
$l_2$ regularization	0.0006019928	0.0004795852

.

Algorithm 1: The proposed hybrid SegNet-UNet-ABC for liver lesion segmentation.

Inputs:$Dee{p}_{Training}\to$ training set of abdominal CT images.

$Dee{p}_{Testing}\to$ testing set of abdominal CT images.

Output: Optimized performance of liver lesion segmentation from CT scan image.

Load $Dee{p}_{Training}$ and further split it into training set $Trainin{g}_{FP}$ and validation set $Validation{}_{FP}$. Load the SegNet model. Send the $Trainin{g}_{FP}$, $Validation{}_{FP}$, and SegNet structure as functional parameters of Algorithm 2. Initialize the SegNet hyperparameter vectors including learning rate $Learnin{g}_{rate}$, minibatch size $M$, momentum $T$, maximum epochs $E$, shuffle $S$, and regularization $l_2$. Get the optimized hyperparameter values of SegNet network generated by Algorithm 2. Embed the resulting optimized parameters $Optima{l}_{hyp}$ into the SegNet architecture. Train the network on the original training set $SegNe{t}_{Training}$. Evaluate performance of liver segmentation from abdominal CT image using $SegNe{t}_{Testing}$. Load the resulting dataset of liver image after it has been segmented using SegNet. Split the set of segmented images into training $UNe{t}_{Training}$ and testing $UNe{t}_{Testing}$. Split the training set $UNe{t}_{Training}$ into $Trainin{g}_{FP}$ and $Validation{}_{FP}$. Load the UNet model. Send $Trainin{g}_{FP}$, $Validation{}_{FP}$, and UNet structure as functional parameters to Algorithm 2. Initialize the UNet hyperparameter vectors including learning rate $Learnin{g}_{rate}$, minibatch size $M$, momentum $T$, maximum epochs $E$, shuffle $S$, and regularization $l_2$. Get the optimized hyperparameter set of the UNet network using Algorithm 2. Embed the resulting optimized parameters $P_2$ into the UNet architecture. Train the network on the segmented training set $UNe{t}_{Training}$. Evaluate the performance of liver lesion segmentation from the liver tissue using $UNe{t}_{Testing}$.

Algorithm 2: The ABC algorithm for selecting the optimal hyperparameter values that optimize the performance of segmentation using the deep network.

Inputs:$FS\to$ the food source number.

$L\to$ the limit or maximum number of trials for abandoning a source.

$MCN\to$ the maximum cycle number.

$Trainin{g}_{FP}\to$ training set.

$Validation{}_{FP}\to$ validation set.

Deep learning architecture (SegNet, UNet, or LeNet-5).

Output: Optimal hyperparameters optimizing segmentation performance using the deep network.

Initialize the parameter set of ABC algorithm. Randomly generate an initial population $P_k$ from the hyperparameter vectors, where $k=\{1.2,\mathrm{.....},K\}$. Randomly select the food sources that correspond to potential solutions using Equation (10). Set $cycle=1$ Do    For each employed bee     Produce a new candidate solution $c_i$ from $P_k$.     Update one parameter in the solution according to Equation (11).     IF the loaded deep learning model is SegNet OR UNet THEN        Evaluate fitness $F_i$ of all candidates and employed bees using Equation (15).       IF the loaded deep learning model is LeNet-5 THEN          Evaluate fitness $F_i$ of all candidates and employed bees using Equation (19).       END IF     END IF     Do the greedy selection as follows:     IF a fitness of an updated candidate solution $>$ fitness of the present solution     THEN        Swap the present solution with the candidate one.     ELSE        Reset abandonment counter for the employed bee to $0$.     END IF        Keep the optimal solution found so far that has the highest fitness value.    END For    For every onlooker bee Do     Select the employed bee by computing its probability using Equation (13).     Randomly select new neighbor solution $P_{neighbor}$ by Equation (11), through updating the solution of the selected employed bee using the onlooker bees.     IF the loaded deep learning model is SegNet OR UNet THEN        Evaluate fitness of all candidates and employed bees using Equation (15).        IF the loaded deep learning model is LeNet-5 THEN          Evaluate fitness of all candidates and employed bees using Equation (19).        END IF     END IF     IF fitness of the new solution discovered by onlooker bee $>$ fitness of employed bee     THEN        Swap the onlooker bee by the employed one.     ELSE        Reset abandonment counter for employed bee to $0$.     END IF     Keep the optimal solution $P_{best}$ found so far that has the highest fitness value.    END For    For each scout bee       Set $L=5$.       IF the abandonment counter of a bee is >$L$       THEN       Reset abandonment counter for the employed bee to $0$.       Produce new solution for scout bee using Equation (10).       END IF     END For Until$cycle\le MCN$ Get the last found sources. Return the solution $k$ with an optimal hyperparameter set optimizing segmentation performance. $\}$

3.5.3. The Proposed Hybrid LeNet-5/ABC Algorithm as Feature Extractor and Classifier of Liver Lesions

One advantage of the CNN is that it can operate directly on the raw data without extraction of data characteristics. This is due to the feature extraction step embedded inside. When constructing CNN architecture, the hyperparameters including convolutional kernel size, number of filters, padding, and stride can affect the network performance as they determine structure of layers, comprising size of resulting feature map at the layer level. Contrarily, the pretrained deep learning networks such as LeNet-5 and AlexNet use static predefined hyperparameters to extract features from images. In LeNet-5 as an example, all convolutional kernels are set to size 5, while in the AlexNet architecture, the sizes of kernels are 11, 5, and 3. Hence, to get optimal feature extraction results using CNN, hyperparameter setting has to be appropriately done. However, there are no standard rules for optimizing CNN hyperparameters that influence the feature extraction process. This has depended mostly on the designer’s intuition [35,67].

Contrary to the ordinary work on CNNs as feature extractors, the ABC optimization algorithm is proposed in this work for tuning the hyperparameters of feature extraction step; this is hypothesized to optimize the predictive results of liver lesion classification, as depicted in the optimized LeNet-5 of Figure 10. The steps of the hybrid LeNet-5/ABC algorithm are shown in Algorithm 3, in which the ABC is used for optimizing the ordinary LeNet-5 topology, which is considered to be the first architecture for CNNs. The ABC algorithm generates an initial population with potential solutions for LeNet-5 construction, where each solution vector comprises the kernel size, stride, and padding, in addition to the number of filters at each convolution and pooling layer, as presented in Step 3 of Algorithm 3. These parameters are supposed to be the solutions for employed bees. The classification error computed at the LeNet-5 classification step is used for evaluating the classification quality of liver lesions, which is used for representing the fitness of solutions computed by ABC, as demonstrated in Step 30 of Algorithm 2. In other words, the fitness value of the hyperparameter vector generated by Algorithm 2 is computed using the following cost function that determines the new solution generated in each iteration step.

$$Error=1-Accuracy$$

(19)

Thereafter, the onlooker bees choose the solutions which return higher fitness values, update these solutions, and compute the fitness values of their solutions once again. Such process is repeated till the termination criterion of Algorithm 2 is met, $cycle\le MCN$. Solutions which cannot optimize the fitness value through a certain time period are considered as abandoned, while new solutions are generated using the scout bees. Further, the LeNet-5 architecture of the CNN is trained using the new optimized hyperparameter values of each convolution and pooling layer. Since size of input layer in the LeNet-5 architecture is $32\times 32\times 1$, the segmented lesion image is resized using MATLAB from $128\times 128\times 1$ to $32\times 32\times 1$. Then, the 2D kernels of LeNet-5 are exploited to extract the local information of liver lesions. On the other side, setting the hyperparameters using the ABC algorithm will influence the feature map size resulting at each subsequent layer of the feature extraction step in the CNN. Thus, the feature map size is computed at each layer using Equation (20), as demonstrated in Step 6 of Algorithm 3.

$$Output=\frac{Size+2\ast Padding-Kernel}{Stride}+1$$

(20)

Here, $Output$ denotes output layer size, $Size$ is input layer size, $Padding$ represents the padding value, $Stride$ is the stride value, and $Kernel$ denotes kernel size. In the proposed algorithm, the neuron numbers of the fully connected layer equal 120 and 84, while the neuron number at output layer was set to 2, which indicates the number of liver lesion classes, i.e., malignant and benign.

The LeNet-5 network is then constructed according to the new topology, as demonstrated in Step 8 of Algorithm 3. Accordingly, the new LeNet-5 architecture is trained using the original training set $LeNet-5_{Testing}$ and tested on $LeNet-5_{Testing}$, as shown in Steps 9 and 10 of Algorithm 3.

Table 5. The optimal values selected by the ABC algorithm to construct the LeNet-5 topology.

Layer	Kernel Size	Stride	Padding	Number of Filters	Size of Output Feature Map
Convolutional ($Con{v}_1$)	$k_1=9$	$s_1=1$	$p_1=6$	$f_1=5$	$36\times 36\times 5$
Pooling ($Poo{l}_1$)	$k_2=5$	$s_2=1$	$p_2=0$	$f_2=19$	$32\times 32\times 19$
Convolutional ($Con{v}_2$)	$k_3=19$	$s_3=1$	$p_3=0$	$f_3=8$	$14\times 14\times 8$
Pooling ($Poo{l}_2$)	$k_4=4$	$s_4=2$	$p_4=0$	$f_4=8$	$6\times 6\times 8$

presents the optimal hyperparameter set selected by the ABC algorithm to construct the LeNet-5 architecture. From the table, the best values of convolutional kernels are $k_1=9$, $k_2=5$, $k_3=19$, and $k_4=4$, respectively, for the first convolutional layer ($Con{v}_1$), first pooling layer ($Poo{l}_1$), second convolutional layer ($Con{v}_2$), and second pooling layer ($Poo{l}_2$). The best stride values are also $s_1=1$, $s_2=1$, $s_3=1$, and $s_4=2$, while the best padding values are $p_1=6$, $p_2=0$, $p_3=0$, and $p_4=0$, respectively, for $Con{v}_1$, $Poo{l}_1$, $Con{v}_2$, and $Poo{l}_2$. The new sizes of feature maps are $36\times 36\times 5$, $32\times 32\times 19$, $14\times 14\times 8$, $6\times 6\times 8$, respectively, at the same layers.

Algorithm 3: The proposed hybrid LeNet-5/ABC for liver lesion classification.

Inputs:$LeNet-5_{Training}\to$ training set of CT images after segmenting the liver using SegNet.

$LeNet-5_{Testing}\to$ testing set of CT images after segmenting the liver using SegNet.

Output: Optimal topology of the LeNet-5 network which optimizes the predictive results of liver lesion classification.

Load the input $LeNet-5_{Training}$ and split it into $Trainin{g}_{FP}$ and $Validation{}_{FP}$. Send the $Trainin{g}_{FP}$, $Validation{}_{FP}$, and LeNet-5 structure as functional parameters of Algorithm 2. Initialize the hyperparameter values of each convolution and pooling layer in the LeNet-5 topology including the kernel size $k$, number of filters $f$, stride $s$, and padding $p$. Get the optimal parameters for constructing the CNN topology using Algorithm 1 of ABC. For each layer $L$ in LeNet-5 architecture       Compute size of resulting feature map at $L$ using Equation (20). End Reconstruct the LeNet-5 network according to the new optimized topology. Train the new LeNet-5 network on the original training set $LeNet-5_{Testing}$. Evaluate the performance of liver lesion classification using testing set $LeNet-5_{Testing}$.

4. Experimental Results

This section clarifies a performance evaluation of the segmentation and classification algorithms proposed in this work. Results and discussion along with comparisons to the other work are demonstrated as follows.

4.1. Experimental Setup

In this work, each dataset was split into a 7:3 ratio [15]; hence, 70% of each was allocated as a training set and 30% was allocated for testing. The original training set was split further to create a validation set: training (35%) and validation (35%). The validation set was used to tune the hyperparameters. More specifically, a model was trained with various hyperparameters on the reduced training set (i.e., the full training set minus the validation set), and the values that performed best on the validation set were returned by the bio-inspired algorithm. Once the bio-inspired algorithm selected the best-performing parameters on the validation set, the best model was trained on the full training set (including the validation set), and this gave the final model. Eventually, the final model was evaluated on the test set to get an estimate of the performance measures and report results. The results reported in this paper were taken using a testing set.

4.2. Results and Discussion

For validating the proposed approach for liver cancer diagnosis, this section tests its main phases, which include lesion segmentation from the CT images using the proposed SegNet-UNet-ABC algorithm, and lesion classification using the proposed hybrid LeNet-5/ABC algorithm. Furthermore, comparisons to the previously published approaches for liver cancer diagnosis are made.

4.2.1. Validation of the Liver Lesion Segmentation Algorithm

To test the efficiency of the proposed SegNet-UNet-ABC algorithm in liver tumor segmentation, this study compares its performance to that of two recent segmentation methods that have been proposed in the literature in this regard. The first method was proposed in [62], which is a hybrid of watershed algorithm (WA), neutrosophic sets (NS), besides to fast fuzzy c-mean-based clustering (FFCM). The authors tested their method, named NS-WS-FFCM, using the Radiopaedia dataset. The obtained results were 92.88%, 86.84%, and 91.66%, respectively, for Jaccard index, Dice index, and correlation coefficient. On the other side, the second compared method was introduced in [20] and utilizes cascaded CNNs optimized using GA to perform liver lesion segmentation using the LiTS dataset. The authors reported that their deep learning method achieved 0.9557 in terms of Dice score. For comparison purposes,

Table 6. The application results of the SegNet-UNet-ABC algorithm for liver lesion segmentation.

Image	Proposed Liver Tumor Segmentation Method
	Radiopaedia			LiTS
	Jaccard Index	Dice Index	Correlation Coefficient	Jaccard Index	Dice Index	Correlation Coefficient
1	0.954	0.96	0.951	0.945	0.95	0.94
2	0.95	0.964	0.957	0.965	0.979	0.968
3	0.959	0.961	0.95	0.948	0.955	0.944
4	0.942	0.962	0.958	0.959	0.971	0.967
5	0.945	0.959	0.95	0.95	0.975	0.96
6	0.942	0.971	0.97	0.95	0.954	0.952
7	0.955	0.965	0.961	0.944	0.965	0.953
8	0.95	0.971	0.969	0.962	0.978	0.969
9	0.958	0.964	0.954	0.954	0.963	0.951
10	0.947	0.956	0.943	0.95	0.957	0.95
11	0.933	0.948	0.93	0.951	0.963	0.959
12	0.958	0.967	0.96	0.971	0.978	0.97
13	0.952	0.968	0.966	0.958	0.975	0.966
14	0.963	0.978	0.974	0.966	0.988	0.961
15	0.961	0.953	0.95	0.966	0.974	0.967
16	0.968	0.964	0.95	0.945	0.977	0.94
17	0.971	0.978	0.976	0.966	0.978	0.964
18	0.977	0.982	0.98	0.98	0.984	0.95
19	0.963	0.979	0.975	0.959	0.96	0.952
20	0.962	0.971	0.97	0.979	0.983	0.949
21	0.963	0.97	0.966	0.97	0.954	0.96
22	0.975	0.979	0.969	0.987	0.954	0.956
23	0.969	0.974	0.97	0.964	0.961	0.955
24	0.97	0.975	0.966	0.97	0.959	0.95
25	0.964	0.969	0.961	0.96	0.977	0.952
26	0.954	0.96	0.95	0.977	0.986	0.958
27	0.971	0.976	0.97	0.971	0.965	0.96
28	0.965	0.97	0.968	0.985	0.97	0.977
29	0.956	0.963	0.952	0.98	0.988	0.952
30	0.977	0.979	0.974	0.987	0.962	0.96
Total	0.96	0.968	0.962	0.964	0.97	0.958

demonstrates the results of liver lesion segmentation obtained in this work using the proposed SegNet-UNet-ABC method, over the Radiopaedia and LiTS datasets.

The results demonstrate that the application of proposed SegNet-UNet-ABC algorithm over the Radiopaedia dataset achieved 0.96, 0.968, and 0.962 in terms of the Jaccard index, Dice index, and correlation coefficient, respectively, while it achieved 0.964, 0.97, and 0.958, respectively, for the three measures when it was tested on the LiTS dataset. It is obvious that the proposed method outperformed the NS-WS-FFCM method and the other method when each of them was applied to one of the datasets tested in this work. These results are due to the robustness of the SegNet and UNet in segmenting liver parenchyma and liver lesions from the CT images, respectively, which perform well in the case of absent clear edges, the definite shape of the liver parenchyma, in addition to the near connection between the liver tissue and the adjacent organs, as illustrated in the cases in Figure 9. In addition, the ABC bio-inspired optimization algorithm optimized the segmentation results of the liver parenchyma and liver lesions by selecting the best hyperparameters for SegNet and UNet which achieved the highest fitness in each step of segmentation.

To investigate the effect of the ABC algorithm on optimizing liver lesion segmentation from CT images when it is used as a hybrid with the SegNet and UNet architectures, some comparisons were made to other bio-inspired optimization algorithms: grey wolf optimization (GWO) [9], antlion optimization (ALO) [29], and ant colony optimization (ACO) [30]. Therefore, in this work we compare the performance of SegNet-UNet-ABC algorithm with that obtained by hybridization of SegNet-UNet with each one of these three other bio-inspired algorithms. Figure 11 presents comparisons of the segmentation performance using SegNet-UNet-ABC, SegNet-UNet-GWO, SegNet-UNet-ALO, and SegNet-UNet-ACO, over the datasets (a) Radiobaedia and (b) LiTS. From the figure, the proposed hybrid SegNet-UNet-ABC outperformed the other hybrid segmentation algorithms in terms of liver lesion segmentation according to the Jaccard index, Dice index, and correlation coefficient.

Furthermore, the convergence time of the proposed SegNet-UNet-ABC algorithm was computed and then compared to those of SegNet-UNet-GWO, SegNet-UNet-ALO, and SegNet-UNet-ACO. Figure 12 presents the time taken by all hybrid algorithms to segment the lesions from CT images using the Radiopaedia and LiTS datasets, respectively. As depicted in Figure 12, when following the proposed SegNet-UNet-ABC algorithm across the two datasets, we can see that it obtained lower convergence time than the other compared algorithms. Hence, it is superior to SegNet-UNet-GWO, SegNet-UNet-ALO, and SegNet-UNet-ACO in terms of convergence time, Jaccard index, Dice index, and correlation coefficient.

Table 7. The parameter lists which achieved the best segmentation results for each bio-inspired algorithm.

ABC	ALO	GWO	ACO
Colony size = 50	Search agents size = 20	Search agents = 90	Number of ants = 100
Maximum iterations = 30	Maximum iterations = 50	Maximum iterations = 30	Maximum iterations = 20
Number of food source, onlooker and employed bees = 25		Lower bound = −50	Evaporation rate = 0.05
Number of solutions = 50		Upper bound = 50	Initial pheromone and the heuristic value = 0.1

demonstrates the parameter list that achieved the best segmentation results for each bio-inspired optimization algorithm.

These results verify that the ABC is the most successful bio-inspired algorithm among those examined when it is used to tune the hyperparameters of SegNet and UNet. High flexibility, broad applicability, population of solutions, capability for handling an objective cost, capability to effectively explore the local solutions, ease of implementation, and robustness properties played a crucial role in optimizing the segmentation of liver and lesions. This result agrees with [68], where the ABC algorithm showed high ability to tune CNN hyperparameters when it was used to optimize hand gesture recognition performance.

4.2.2. Validation of the LeNet-5/ABC Algorithm

For validating performance of proposed LeNet-5/ABC algorithm as a feature extractor and classifier of liver cancer, we compared the solution results obtained by it against two other algorithms used in the literature for the same purpose, which are the single CNN [69] and traditional feature-based SVM [69]. In the first compared algorithm, the single CNN was employed as a feature extractor and classifier of liver cancer. In the second compared algorithm, a 114-dimensional feature vector was extracted from CT images including gray level statistics, GLCM features, and Gabor features, then principal component analysis (PCA) was used to reduce the feature space into a 25-dimensional vector.

For comparison, the number of runs used for validating each algorithm was set to 10. The three algorithms were validated using the two datasets, LiTS and Radiopaedia. The total averages of specificity, F1-score, and classification accuracy at each run of LeNet-5/ABC are presented in Figure 13. The same averages were likewise computed at each run of the single CNN algorithm and traditional feature-based SVM, as demonstrated in Figure 14 and Figure 15, respectively. The overall averages at all runs of each algorithm were also computed to give a clear view of the optimization occurring across the three algorithms.

For 10 runs of the LeNet-5/ABC algorithm, it is obvious from Figure 13 that the overall averages of specificity, F1-score, and accuracy were 0.986, 0.98, and 0.99, respectively, over the Radiopaedia dataset, whereas they were 0.982, 0.976, and 0.985, respectively, over the LiTS dataset. By following the 10 runs of the single CNN algorithm, it is evident that the averages were 0.963, 0.967, and 0.956 over the Radiopaedia dataset, while they were 0.958, 967, and 0.961 over the LiTS dataset. Eventually, the averages of specificity, F1-score, and accuracy obtained through 10 runs of traditional feature-based SVM were 0.932, 0.919, and 0.904 over the Radiopaedia dataset, whereas they were 0.926, 0.914, and 0.893 over the LiTS dataset.

As computational time is a necessary measure to evaluate the quality of classification, Figure 16 presents a comparison of the computational time (in seconds) required to test each CT image in the LiTS and Radiopaedia sets using LeNet-5/ABC, single CNN, and traditional feature-based SVM. For the two datasets, it is vivid that LeNet-5/ABC achieved the lowest computational time (4 s) in comparison to the two other algorithms. Therefore, LeNet-5/ABC outperforms them regarding specificity, F1-score, accuracy, and computational time.

4.2.3. Comparisons to Other Work

To investigate the effectiveness of the overall proposed approach for liver cancer diagnosis, a comparison between it and the most recent works was made.

Table 8. Comparison to other works on liver cancer diagnosis using CT images.

Reference	Year	Dataset	Approach	Performance Measure
[61]	2018	LiTS	Liver segmentation using Morphological Snake and Felzenszwalb, and liver centroid prediction using ANN.	Morphological Snake outperformed the other algorithm in terms of Dice index and accuracy, by achieving 0.88 and 98.11%, respectively. The result of $F1-score$ was 0.8771.
[70]	2018	Private dataset	Deep learning model of generative adversarial networks.	Specificity = 92.4%
[15]	2019	Private dataset	Lesion segmentation and classification using watershed with GMM and GLCM with DNN, respectively.	Results obtained using testing set were 98.38%, 95%, and 97.72% for accuracy, Jaccard index, and specificity, respectively.
[71]	2019	Radiopaedia	Crow search, integrated with chaos theory and FCM algorithm.	Accuracy = 0.880
[72]	2019	Private dataset (hospital data)	CNN and DWT-SVD-based perceptual hash function.	Accuracy = 97.3%
Proposed	2019	Radiopaedia and LiTS	Lesion segmentation and classification using SegNet-UNet-ABC and LeNet-5/ABC, respectively.	Results of the Radiopaedia dataset were 0.99, 0.96, 0.968, 0.986, and 0.98 for accuracy, Jaccard index, Dice index, specificity, and $F1-score$. Results of the LiTS dataset were 0.985, 0.964, 0.97, 0.982, and 0.976, respectively, for the same measures.

[15,61,70,71] demonstrates the comparison, which includes the following: (1) dataset used; (2) the approach which comprises segmentation method, feature extraction, and classification algorithm; and (3) the performance measures used. As can be observed, the majority of segmentation methods proposed in the literature [61,70] depend on region-based and clustering approaches, which check the low contrast between the liver and the surrounding tissues and organs. However, the noise of CT images, together with the large difference in liver shapes of patients, makes the state-of-the-art algorithms for liver segmentation incapable of giving optimal results. For instance, in comparison to current work, the two segmentation algorithms proposed in [61] achieved lower performance regarding Dice index and $F1-score$. The results reported for Dice index are 0.8602 and 0.8803, while those reported for $F1-score$ are 0.8771 and 0.8556.

On the other side, classification of liver cancer depends on traditional feature-based SVM, ANN, and CNN, and hybrid approaches [15,69]. For instance, 0.880 was reported in [71] as the classification accuracy obtained on the Radiopaedia dataset, which is lower than the accuracies reported in the current work. In [70], 92.4% was reported in terms of specificity, which is lower than the specificity results achieved using the proposed approach. In [15], the authors attained their results using two datasets: training and testing. The results reported using the testing set were 98.38%, 95%, and 97.72% for accuracy, Jaccard index, and specificity, respectively, while the results reported using the training set (samples seen by the system) were 99.38%, 98.18%, and 99.09%, respectively, for the same measures. Therefore, the results reported in the current work using the testing set (samples unseen by the system) outperform those reported in [15] in terms of accuracy, Jaccard index, and specificity. Hence, it is evident from Table 8 that the proposed hybrid bio-inspired deep learning approach outperforms state-of-the-art contributions that have used the same and different CT datasets in terms of accuracy, Jaccard index, Dice index, specificity, and $F1-score$.

The main thought to justify why the proposed hybrid model performs better than the other works is that the deep learning networks achieve robust performance in terms of liver lesion segmentation and classification when they are hybridized with the ABC bio-inspired optimization algorithm. This hybridization helped to increase the segmentation performance by minimizing over-segmentation. It also helped to handle the indeterminacy and uncertainty in CT images in a more effective way. Furthermore, it improved the classification performance of the LeNet-5 network by providing an optimal topology of the network and reducing over-fitting and the probability of being trapped in local optima. This eventually achieved high classification accuracy that led to better diagnostic results. Likewise, this reduced the computational time needed by the deep learning algorithms either to segment the liver lesions from the CT image or to classify the lesions into the corresponding cancer types.

5. Conclusions

This work proposed a new approach for liver cancer diagnosis from CT images, based on the hybridization of different deep learning models with the ABC bio-inspired optimization algorithm. Firstly, a novel hybrid segmentation algorithm was proposed for extracting liver lesions from CT images using SegNet, UNet, and ABC, named SegNet-UNet-ABC. The ABC algorithm was used in this regard to tune the hyperparameters of the SegNet and UNet deep learning architectures, in such a way as to optimize the performance of liver lesion segmentation. Secondly, a proposed hybrid LeNet-5/ABC algorithm was introduced; this uses the LeNet-5 architecture of CNN as a feature extractor and classifier in a different way to other works on liver cancer diagnosis which employ the traditional feature-based classification methods. Furthermore, the ABC algorithm was used to select the optimal topology for constructing the LeNet-5 network, with the aim to improve the predictive results of liver cancer diagnosis. Two publicly available datasets, namely, Radiopaedia and LiTS, were tested. To investigate the efficacy of the proposed SegNet-UNet-ABC algorithm in liver lesion segmentation from CT images, this work firstly compared its performance to that of two other segmentation methods from the state-of-the-art. The results demonstrate that the SegNet-UNet-ABC algorithm outperformed the two other algorithms when it was applied to the two datasets. The results obtained using the Radiopaedia dataset were 0.96, 0.968, and 0.962, while those obtained using the LiTS dataset were 0.964, 0.97, and 0.958 for Jaccard index, Dice index, and correlation coefficient, respectively. Furthermore, extensive comparisons were made to investigate the efficiency of the ABC algorithm in selecting hyperparameters that improve segmentation accuracy when used in combination with the SegNet and UNet architectures. The other bio-inspired optimization algorithms used in the comparison were GWO, ALO, and ACO. Hence, this work compared performance of SegNet-UNet-ABC algorithm to that of SegNet-UNet-GWO, SegNet-UNet-ALO, and SegNet-UNet-ACO. The results demonstrate that the SegNet-UNet-ABC algorithm outperformed the other algorithms regarding Jaccard index, Dice index, correlation coefficient, and convergence time. Moreover, the hybridization of deep learning networks with the ABC bio-inspired concept provides optimal hyperparameters which minimize over-segmentation and overcome indeterminacy and uncertainty in CT images in a more effective way. Further, validation of the hybrid LeNet-5/ABC algorithm was done by comparing the solution results obtained by it to those obtained by two other algorithms used in the literature for feature extraction and classification of liver cancer: the single CNN and traditional feature-based SVM. Results obtained across 10 runs of each algorithm revealed optimization in overall averages of specificity, F1-score, and accuracy, in favor of the LeNet-5/ABC algorithm. The optimization ratios obtained over the Radiopaedia dataset were 2.3%, 1.3%, and 3.4%, respectively, for the aforementioned algorithms, while the ratios were 2.4%, 1.3%, and 2.4%, respectively, for the same algorithms, over the LiTS dataset. Moreover, the LeNet-5/ABC algorithm is superior than other algorithms in terms of computational time. For future work on liver cancer diagnosis, multiple modalities such as ultrasound and CT images are intended to be fused as a multimodal diagnostic approach including deep learning. This approach is hypothesized to increase diagnosis confidence through the merits of deep multimodal fusion of medical images.