Ischemic Stroke Lesion Segmentation Using Mutation Model and Generative Adversarial Network

Abstract

Ischemic stroke lesion segmentation using different types of images, such as Computed Tomography Perfusion (CTP), is important for medical and Artificial intelligence fields. These images are potential resources to enhance machine learning and deep learning models. However, collecting these types of images is a considerable challenge. Therefore, new augmentation techniques are required to handle the lack of collected images presenting Ischemic strokes. In this paper, the proposed model of mutation model using a distance map is integrated into the generative adversarial network (GAN) to generate a synthetic dataset. The Euclidean distance is used to compute the average distance of each pixel with its neighbor in the right and bottom directions. Then a threshold is used to select the adjacent locations with similar intensities for the mutation process. Furthermore, semi-supervised GAN is enhanced and transformed into supervised GAN, where the segmentation and discriminator are shared the same convolution neural network to reduce the computation process. The mutation and GAN models are trained as an end-to-end model. The results show that the mutation model enhances the dice coefficient of the proposed GAN model by 2.54%. Furthermore, it slightly enhances the recall of the proposed GAN model compared to other GAN models.

1. Introduction

Medical imaging is a domain that uses methods to create images for the human body that can be used in medical decisions [1]. Deep learning is the most commonly used in computer vision [2]. Automated segmentation in medical images is considered an important task and research in both fields: artificial intelligence (AI) and medical [3,4,5,6,7]. AI models are used to determine the location of the damaged tissues and structures in the images of the human body, such as tumors and ischemic stroke (IS) in the brain [8]. In contrast, these types of images are considered potential resources for machine learning and deep learning model enhancements. On the other hand, these images suffer heterogenous in the intensity of each voxel. They are low contrast, low frequency, and wide variety because they belong to different low contrast in medical scans [8,9,10,11,12,13,14]. Furthermore, medical image annotation requires expert doctors and is considered to be a time-consuming task. Therefore, the number of existing images for specific diseases is a significant challenge for deep learning models [4,8,11,15,16,17]. Another challenge in medical-image segmentation, especially for 3D images, is the sturdy imbalance between damaged and normal tissues. The size of damaged tissues is minimal compared to normal tissues, and some images did not contain damaged tissues [18,19,20,21,22,23,24,25].

Ischemic stroke lesion segmentation is one of the essential segmentation tasks where traditional machine learning and deep learning approaches are used to automatically determine the location of the damaged tissues in the brain. The computed tomography perfusion (CTP) is the most popular computed tomography (CT) modality for ischemic stroke (IS). CTP is used to assess the magnitude of infarct core by hypoperfused in the damaged tissues to decide whether there is an ability to treat the patient [3,8].

Table 1. Shows brief descriptions for all modalities and their abbreviations that are used in the proposed model [3,27,28].

Abbreviations	Description
CT	Computed tomography
CTP	Computed tomography perfusion
DWI	Diffusion-weighted imaging
CBF	Cerebral blood flow
CBV	Cerebral blood volume
Tmax	Time-to-maximum flow
MTT	Mean transit time
OT	Semantic segmentation label for IS lesions

shows brief descriptions for all abbreviations of the modalities that used in the proposed model, as well as the low non-uniformity contrast of voxel intensities in the brain CTP images and a limited number of available images, the lesions in the brain have a different shape, magnitude, and positions in the brain [3,4,10,11,17,26].

One of the most popular datasets for segmenting the IS lesions in the brain is ischemic stroke lesion segmentation (ISLES-2018) [3,27]. It consists of 94 and 62 cases for training and test sets, respectively. Therefore, many researchers have used different machine learning models and different augmentation techniques to expand the training set in the ISLES-2018 dataset. For instance, the rotation, flipping, and re-scaling images are used in the [16,29,30,31,32,33] models. Furthermore, the [12] model uses mirroring and permuting the input axis technique for augmentation. While the [31] model shifts forward or backward and adds Gaussian noise. However, the performance using the test set of the ISLES-2018 dataset is not satisfactory using these techniques.

The [33] model uses the generative adversarial network (GAN) to translate the CTP images into DWI modality. In contrast, other models use the GAN approach to generate synthetic images to increase the number of the training set [10,16,32]. The [16,32] models use the Gaussian noise in the generator method, where their performance is evaluated using the training set.

The [10] model uses synthesized pseudo-DWI as a generator method to generate lesion regions based on the DWI label. Additionally, the input consists of six channels: low-level and high-level features of trained CTP image, and the original images of the rest modalities. However, it depends on the DWI label to generate lesion locations without taking into account the normal regions. Additionally, it ignores the slice level where a single channel for each modality is used in the generator.

Thus, more new techniques are required to handle the small number issue of medical images, particularly brain images in the ISLES-2018 dataset. In this paper, the proposed mutation model inspired by the genetic algorithm is used to generate a synthetic dataset to increase the number of training samples. Consequently, the performance of identifying the IS tissues increases by training various generated samples called mutated images. The mutation model uses the distance map to determine the adjacent locations related to a region in the CTP image. These locations are used to mutate the corresponding regions of all modalities and slices in two inputs to generate synthetic inputs.

Furthermore, a semi-supervised GAN model, called a few-shot 3D multi-model [8], was enhanced and transformed into a supervised GAN model to exploit its generator while gaining more meaningful information using labels. Therefore, four main contributions are used in the proposed model, as illustrated below.

• The CTP image and the euclidean distance method are used to generate a distance map by computing distance horizontally and vertically for every two adjacent pixels.
• The distance map and mutation model are used to produce new synthetic samples. A set of adjacent pixels, locations of a region, are selected from one CTP image to assign the locations values of the modalities into different corresponding modalities while preserving the shape of these locations.
• A semi-supervised GAN model was enhanced and modified to supervised GAN model to exploit the entire knowledge and gain more meaningful information using labels.
• A shared module between segmentation and discriminator are used to reduce complexity of the GAN model and evaluate the proposed mutation method as an end-to-end model.

The rest of this paper is organized as follows. Section 2 presents the literature review. Section 3 illustrates the proposed end-to-end model. Section 4 presents the experiment setting. Section 5 presents applications and future directions. Finally, Section 6 concludes the paper.

2. Literature Review

Similarly to the detection tasks, such as ADHD detection [34] and schizophrenia patients detection [35], the segmentation task also has a significant role in the medical field, such as IS lesion segmentation. Although the ISLES-2018 dataset was created for IS lesions segmentation, authors in [9] have used it for the classification task. The lesion segmentation is utilized as a pre-processing step to classify the brain image into healthy or unhealthy. The DWI, CBV, and CBF modalities are only used in the model where the most noisier modalities are ignored. Based on the [10], the Tmax and MTT modalities are the most noise modalities in the ISLES-2018 dataset and have a lower spatial resolution. In contrast, many researchers have utilized the ISLES-2018 dataset and machine learning for IS lesions segmentation. Authors in [30,31] have considered the imbalance issue and use encoder–decoder convolutional neural network to train data. The Clera2 model [30] uses the balanced patches and a category weighting loss function. While the [31] model uses weighted binary cross-entropy to equalize the previous category probabilities during training.

State-of-the-art models are used widely in the IS lesions segmentation. Both [28] and [29] models uses the ResNet [36] as a segmentation task. Researchers in [28] have transferred the pre-trained ResNet and integrated its output with the output of the random forest (RF) model. Researchers in [29] have combined a fully convolutional neural network and dilated convolutional network to introduce a pyramid pooling. Furthermore, the dilated convolutional network is integrated with a different state-of-the-art model called U-shape [37] to encode features of the brain images [12]. While the MS-DCNN model [38] integrates the U-shape [37] with dense block for more robust extract features and to mitigate overfitting.

Moreover, the GAN approach is used to expand the training set of the ISLES-2018 dataset [10,16,33]. The [10] model uses the extracted features of CTP images to create new segments that are similar to lesions located in the determined set of the lesion DWI label. The mean square error (MSE) and L1 normalization are used as loss functions for training the generator. While the [33] model uses the generator module to generate DWI modality from the CTP modality by exploiting the U-net model [37]. The [16] model uses the generator module to increase the dataset by generating noisy images, and the discriminator module to provide GAN the ability to distinguish between original and generated samples. On the other hand, the [17] model inputs the error of the discriminator into the segmentation module for second back-propagation. However, it excludes the generator module, which does not provide a synthetic dataset.

In contrast to the previous frameworks that either utilize the DWI modality or add noise to create a synthetic dataset of the brain images, the proposed model uses a new technique to generate a synthetic dataset using mutation operation and distance map. Moreover, the mutated images were integrated into the supervised GAN model to utilize its generative model.

Table 2. Summary of the existing GAN and proposed models that use the ISLES-2018 dataset for evaluation.

References	Augmentation Method	Finding	Limitation
Wang et al., 2020 [10]	Synthesized pseudo DWI module	The Synthesized pseudo DWI module uses the ISLES-2018 to generate a synthetic dataset	The DWI label is used to switch lesion regions with normal regions without considering other possible choices for switching regions
Rezaei et al., 2019 [16]	General augmentation methods	The general augmentation techniques are used to increase the training set	The traditional augmentation processes are used to increase the training set, such as flipping and adding Gaussian noise.
Yang et al., 2018 [17]	-	The error of the discriminator is fed into the segmentation module for second back-propagation	The generator module is excluded, and a synthetic dataset is not provided.
Rezaei et al., 2018 [32]	Gaussian noise method	The Gaussian noise method is used to increase the training set	The Gaussian noise is used only for augmentation.
Liu and Pengbo 2018 [33]	General augmentation methods and translation method	General augmentation methods are used to increase the dataset, and the generator is used to translate the CTP modality into DWI modality	Traditional processes are used for augmentation, such as flipping and scaling images.
Proposed model	Mutation model based on distance map that integrated into proposed GAN model	Presents a mutation model based on a distance map that randomly selects normal or damaged regions to generate a synthetic dataset and integrate it into the GAN model. Furthermore, a supervised GAN model is proposed to exploit the generator and gain information from labels. Finally, utilize a shared network for segmentation and discriminator to reduce GAN complexity	The proposed mutation model is not adaptive, and the proposed end-to-end model suffers from overfitting

summarized the previous works, which used the benchmark ISLES-2018 dataset [3,27], and noted their limitations and findings.

3. Materials and Methods

The proposed model is an end-to-end model that uses two stages of generating a synthetic dataset: a mutation model using a distance map and a supervised GAN model. The first stage is an augmentation technique to generate new images that preserve the spatial dimension of the original images. In the second stage, a set of patches of each image are fed to the GAN model. However, the mutation model is used during training, where the test set is introduced directly to the second stage.

This section is organized as follows. Section 3.1 presents the pre-processing methods that are used to enhance and normalize data. Section 3.2 illustrates the integrated mutation model using a distance map. Section 3.3 presents the baseline semi-supervised GAN model and proposed supervised GAN model.

3.1. Data Pre-Processing

Two major pre-processing techniques are used to handle the low-frequency and the variety of the intensities values in the different modalities of the ISLES-2018 dataset: bias correction [12], and image enhancement [9]. The bias correction method corrects the non-uniformity low-frequency intensity in the input modalities: CTP, CBF, CBV, MTT, and Tmax [12]. The image enhancement method enhances CTP images and normalizes their intensity values to provide better contrast and information [9]. The first step of the image enhancement is converting each image of dimension $height\left(H\right)\times width\left(W\right)\times number$ $of$ $slices\left(D\right)$ into a single channel as presented in Equation (1), where $(i,j)$ is the location of the intensity value in spatial dimension $H\times W$, and D is the number of slices. Then, each image is normalized into gray-scale as demonstrated in Equation (2), where $minimu{m}_{value}$ and $maximu{m}_{value}$ are the minimum and maximum intensity values of the $H\times W$ image. Consequently, the range of intensity values is between 0 and 255.

$$imag{e}_{i,j}=\frac{{\sum }_{d=1}^{D}imag{e}_{i,j,d}}{D}$$

(1)

$$imag{e}_{i,j}=\frac{(imag{e}_{i,j}-minimu{m}_{value})\ast 255}{maximu{m}_{value}-minimu{m}_{value}}$$

(2)

Finally, linear transformation, log transformation, smooth, and edge enhancement are used for image enhancement [9]. Figure 1. shows an example of the image enhancement technique. However, the image enhancement technique is used as a pre-processing of the mutation model to determine mutated locations, where the CTP images are prepared to have a clear brain structure that is observed by visualization, as presented in Figure 1. The 32 slices for each input modality are fed to the GAN model to mitigate losing information.

3.2. Mutation Model Using Distance Map

Two proposed modules use the CTP modality to generate synthetic images: distance map and mutation. The comprehensive model of this stage is illustrated in Algorithm 1, and Figure 2 shows an example of generating a new CTP image. Two generated distance maps are used to compute the average distance map: horizontal and vertical distance maps. The horizontal map computes the average distance values between each pixel and its right neighbor, while the vertical map computes the average distance values between each pixel and its bottom neighbor. Then, a threshold is used to select all adjacent pixels related to a single region, where these pixels have similar intensity values.

Algorithm 1 Generate synthetic data using mutation model

procedure Mutation_Model(data1, label1, data2, label2, center_point)     ▷ Input $data1$: The first input consists of five modalities with dimension $H\times W\times D1\times N$     ▷ Input $label1$: The semantic label of the $data1$ with dimension $H\times W\times D1$     ▷ Input $data2$: The second input consists of five modalities with dimension $\times H\times W\times D2\times N$     ▷ Input $label2$: The semantic label of the $data2$ with dimension $H\times W\times D2$     ▷ Input $center\_point$: The determined center point $(y,x)$ of the selected region            $ctp\_image\_1\leftarrow$ return the CTP modality from $data1$            $distance\_map\_1\leftarrow DISTANCE\_MAP\_METHOD(ctp\_image\_1)$ as illustrated in Algorithm 2            $ctp\_image\_2\leftarrow$ return the CTP modality from $data2$            $distance\_map\_2\leftarrow DISTANCE\_MAP\_METHOD(ctp\_image\_2)$ as illustrated in Algorithm 2            $angle\leftarrow ROTATION\_METHOD(distance\_map\_1,distance\_map\_2)$ as illustrated in Algorithm 3            $rotated\_image\_2\leftarrow$ rotate $ctp\_image\_2$ by $angle$            $locations\leftarrow SELECT\_LOCATIONS\_METHOD(rotated\_image\_2,center\_point)$ as illustrated in Algorithm 4            $newData1,newLabel1\leftarrow MUTATION\_METHOD(data1,data2,$                          $label1,label2,locations,angle,center\_point)$ illustrated in Algorithm 5            $\mathbf{RETURN}$ $newData1$, $newLabel1$ end procedure

Algorithm 2 demonstrates the method of the computing distance map. The Euclidean distance is used to create the horizontal and vertical distance maps. First, a 2-dimensional (2D) array locates the intensity value of the right neighbor pixel for each CTP image pixel. The second 2D array is used to modify the spatial dimension of the CTP image to the same dimension of the first 2D array while preserving the location of each pixel. Then, the Euclidean distance computes the distance between every two corresponding pixels in these two 2D arrays. Second, the same scenario is used to compute the vertical distance map, where the first 2D array is used to locate the intensity values of the bottom neighbor for each CTP image pixel. Finally, the distance map is the average of these two maps, where it is normalized to the gray-scale.

Algorithm 2 Distance map method

procedure Distance_Map_Method(ctp_image)     ▷ Input $ctp\_image$: CTP image of dimension $h\times w\times d$            $ctp\_image\leftarrow$ enhance the image as illustrated in Section 3.1            $im1\leftarrow ctp\_image[:,1:]$            $im2\leftarrow ctp\_image[:,:-1]$            $horizontal\_distance\_map\leftarrow \sqrt{{(im1-im2)}^2}$            $im1\leftarrow ctp\_image[1:,:]$            $im2\leftarrow ctp\_image[:-1,:]$            $vertical\_distance\_map\leftarrow \sqrt{{(im1-im2)}^2}$            $distance\_map\leftarrow \frac{horizontal\_distance\_map+vertical\_distance\_map}{2}$            $distance\_map\leftarrow$ normalize $distance\_map$ to gray-scale as illustrated in Equation (2)            $\mathbf{RETURN}$ $distance\_map$ of dimension $h-1\times w-1$     end procedure

Algorithm 3 explains the image rotation method that ensures two images have approximately the same structure rotation. The distance map has the shape of a brain structure, as presented in Figure 2. Therefore, the front and back regions of the brain structure are used to rotate the distance map of the second image to have a similar structure rotation to the first distance map. The Euclidean distance is used to find the best rotation angle, which finds the minimum distance between two distance maps using both front and back parts. Then, the optimal angle is computed by finding the average of these two angles. However, it is different for every two inputs.

Algorithm 4 illustrates the method of selecting a set of adjacent locations for the mutation process. A single point ($y,x$) is selected randomly from determined regions of the brain structure to prevent generating arbitrary regions: center, top left, and top right. Then, regions are cropped from the distance map where the point ($y,x$) is the center. Algorithm 2 is used to compute the distance map of the cropped region, where a threshold is used to select a set of adjacent locations (pixels). These locations have approximately similar intensity values.

Finally, Algorithm 5 illustrates the mutation process. The selected adjacent locations are used to mutate the intensity values in all corresponding modalities of two inputs. It is performed for each slice separately, where the intensity equal to zero is ignored. Although the rotation method is used to achieve similar structure rotation for the two images, they have not achieved exactly the same rotation. Therefore, the Euclidean distance is used to determine the best new locations where the intensity values of the first input have a minimum distance from the intensity values of the second one.

Algorithm 3 Rotation method

procedure Rotation_Method(img1, img2)     ▷ Input $img1$: Image of dimension $h\times w$     ▷ Input $img2$: Image of dimension $h\times w$            $best\_front\_angle\leftarrow 0$            $best\_back\_angle\leftarrow 0$            $mini\_front\_distance\leftarrow {10}^6$            $mini\_back\_distance\leftarrow {10}^6$            $img1\_front\leftarrow img1[h/2-16:h/2+16,0:w/10]$            $img1\_back\leftarrow img1[h/2-16:h/2+16,w/10:]$            For a ← 1 to 90                For s ∈ [1, −1]                   $rotated\_img2\leftarrow$ rotate $img2$ by angle equals to $a\ast s$                   $img2\_front\leftarrow rotated\_img2[h/2-16:h/2+16,0:w/10]$                   $img2\_back\leftarrow rotated\_img2[h/2-16:h/2+16,w/10:]$                   $front\_distance\leftarrow \sqrt{{\sum }_{i=0}^h{\sum }_{j=0}^w{(img1\_front[i,j]-img2\_front[i,j])}^2}$                   $back\_distance\leftarrow \sqrt{{\sum }_{i=0}^h{\sum }_{j=0}^w{(img1\_back[i,j]-img2\_back[i,j])}^2}$                   $best\_front\_angle\leftarrow$ a*s, if the $front\_distance$ is less than $mini\_front\_distance$                   $best\_back\_angle\leftarrow$ a*s, if the $back\_distance$ is less than $mini\_back\_distance$            $angle\leftarrow \frac{best\_front\_angle+best\_back\_angle}{2}$            $\mathbf{RETURN}$ $angle$     end procedure

Algorithm 4 Select adjacent locations from the distance map for mutation process

procedure Select_Locations_Method(img, center_point)     ▷ Input $img$: Image of dimension $h\times w$     ▷ Input $center\_point$: The determined center point (y, x) of the selected region     ▷L: Length used for cropped image     ▷T: Threshold used to select adjacent locations           $cropped\_img\leftarrow img[y-L:y+L,x-L:x+L]$           $cropped\_distance\_map\leftarrow DISTANCE\_MAP\_METHOD(cropped\_img)$ as illustrated in Algorithm 2           $locations\leftarrow$ list of all locations in $cropped\_distance\_map$ where their distance values are less than T           $\mathbf{RETURN}$ $locations$     end procedure

Algorithm 5 Mutate regions to generate new input

procedure Mutation_Method(modalities_1, modalities_2, locations, angle, center_point)     ▷ Input $modalities\_1$: First input consists of five modalities with dimensions $H\times W\times D1\times N$     ▷ Input $modalities\_2$: Second input consists of five modalities with dimensions $H\times W\times D2\times N$     ▷ Input $label1$: The semantic label of the $modalities\_1$ that its dimension $H\times W\times D2$     ▷ Input $label2$: The semantic label of the $modalities\_2$ that its dimension $H\times W\times D2$     ▷ Input $locations$: Set of the adjacent locations used for mutation model     ▷ Input $angle$: The rotated angle that used to rotate images of the $modalities\_2$     ▷ Input $center\_point$: The center points of the selected locations in the $modalities\_2$            $min\_slices\_num\leftarrow$ MIN($D1$, $D2$)            For i ← 1 to N                $M1\leftarrow modalities\_1[:,:,:,i]$                $M2\leftarrow modalities\_2[:,:,:,i]$                $M2\leftarrow$ rotate $M2$ by $angel$                IF i ← 1                   $region\leftarrow$$M2[center\_point-l:center\_point+l,center\_point-l:center\_point+l,:,i]\left[locations\right]$                       $region\leftarrow$ convert the CTP image to single channel and normalized it using Equitation (1) and Equitation (2), respectively                    $new\_center\leftarrow$ return shifted center based on the new region in CTP image of the M1 and $region$                    $label1[new\_center-l:new\_center+l,new\_center:new\_center,:,i]\left[locations\right]\leftarrow$                            $label2[center\_point-l:center\_point+l,center\_point-l:center\_point+l,:,i]\left[locations\right]$                END IF                $M1[new\_center-l:new\_center+l,new\_center:new\_center,:,i]\left[locations\right]\leftarrow$                              $M2[center\_point-l:center\_point+l,center\_point-l:center\_point+l,:,i]\left[locations\right]$                $newData1\leftarrow$ append $M1$ in axis zero            END FOR,            $\mathbf{RETURN}$ $newData1$, $label1$     end procedure

3.3. Supervised GAN model

The semi-supervised GAN model called few-shot 3D multi-model [8] was enhanced and transformed into a supervised GAN model to exploit its generator module to generate patches with similar distributions to the mutated patches. More details of the few-shot 3D multi-model [8] and its generator are illustrated in Section 3.3.1. Furthermore, the proposed supervised GAN model is presented in Section 3.3.2.

3.3.1. Semi-Supervised GAN Model

A set of labeled patches X and unlabeled patches U are generated from the original images and introduced into the few-shot 3D multi-model [8]. The model consists of two modules: discriminator D and generator G. The discriminator module differs from the standard GAN approach by using a single output layer to segment the $K+1$ classes. The first K probabilities are for segmentation, while the last probability $K+1$ predicts whether each voxel of input is fake. The discriminator has three inputs: labeled patches, unlabeled patches, and fake patches $\stackrel{-}{X}$ generated by the generator module. The discriminator loss function consists of three loss functions for each input where the cross-entropy of the segmentation task uses labeled patches, while both unlabeled patches and fake patches are used to achieve the discriminator purpose using $k+1$ output. In the generator module, vectors of random noises Z are fed to the deconvolution network to generate a set of fake patches $y\{\stackrel{-}{X}|G\left(Z\right)\}$ have a similar distribution to the unlabeled patches. The feature matching ($FM$) loss is used to optimize the generator module that generates features matching the features of the unlabeled patches in the intermediate layers of discriminator $ϝ$.

$$FM=\left|\right|{\mathbb{E}}_Uϝ\left(U\right)-{\mathbb{E}}_Zϝ\left(G\left(Z\right)\right){\left|\right|}_2^2$$

(3)

3.3.2. Proposed Supervised GAN Model

In contrast to the few-shot 3D multi-model [8], the proposed GAN model uses patches generated from the mutated images and ignores the unlabeled patches to exploit the entire knowledge and gain more meaningful information using labels. Furthermore, the few-shot 3D multi-model [8] was designed for semi-supervised datasets, while the proposed model uses a supervised ISLES-2018 dataset. The proposed model consists of two modules: shared neural network $SNN$ and generator G, as illustrated in Figure 3. Inspired by the multi-channel segmentation (MCS) [39] model, the segmentation S and discriminator D have a shared neural network architecture $SNN$ to utilize shared feature map and reduce computation processes. Each of S and G has a separate output and loss function, where both of these two loss functions optimize the weights of the $SNN$ module. However, the $SNN$ architecture is similar to the autoencoder module that is used in the few-shot 3D multi-model [8].

The $SNN$ module consists of two inputs and three output layers. The inputs are labeled patches X and generated patches $y\{\stackrel{-}{X}|G\left(Z\right)\}$. The outputs are the IS segmentation $y\{y_{IS}|SNND\left(X\right)\}$, and two binary classification outputs: $y\{y_X|SNNE\left(X\right)\}$ and $y\{y_{\stackrel{-}{X}}|SNNE\left(G\left(Z\right)\right)\}$. The $SNNE$ and $SNND$ indicate to encoder and decoder, respectively, where the fully connected layer with soft-max activation function is performed on latent vector $y\{\stackrel{-}{Z}|SNNE(·)\}$ to predict the $y_X$ and $y_{\stackrel{-}{X}}$ into fake or real patches. The cross-entropy loss function is used for each output, where the discriminator loss function is the summation of the binary classification loss functions. The discriminator loss function optimizes the SNNE module. The proposed generator is similar to the generator in the few-shot 3D multi-model [8]. However, it uses the labeled patches X to compute the $FM$ loss function.

$$FM=\left|\right|{\mathbb{E}}_Xϝ\left(X\right)-{\mathbb{E}}_Zϝ\left(G\left(Z\right)\right){\left|\right|}_2^2$$

(4)

4. Experiment Result

The proposed supervised GAN model and the ISLES-2018 dataset are used to evaluate our main contribution, which is the mutation model. This section is organized as follows. Section 4.1 presents the model setting used for the experiment. Section 4.2 shows the performance results of the proposed model.

4.1. Experimental Settings

The experiments were conducted using Google Colab and Tensorflow with Keras for implementation. The initial values of the selected parameters are similar to the [8] study. The Adam optimizer is used with an initial learning rate of $0.0001$, 128 batch size. However, the number of epochs used to train the proposed model is 20, where the model is converged. The training images of the ISLES-2018 dataset are split into training and valid by a ratio of 80:10. The dimensions of stride to generate patches is $(8,8,8)$, and the dimension of the random noise Z is 200. In the mutation model, the threshold is 44, and the max length used to crop images in all directions is 64. Therefore, the spatial dimension of the cropped region is $128\times 128$, where locations that have intensity values less than the threshold are used for the mutation process.

In the training phase, each input image is mutated randomly for each batch, where each image has the probability of $10\%$ not being augmented. Therefore, the mutation model generates new synthetic training images at each epoch. In contrast, the original valid images are fed to the proposed GAN model without augmentation to evaluate the end-to-end model.

4.2. Results

Since the mutation and supervised GAN models are trained as an end-to-end model to generate and train a variety of images, the performance increases by 2.54% using a valid set, as presented in

Table 3. Shows the performance enhancement by integrating the proposed mutation model into the proposed supervised GAN model. The valid set of the ISLES-2018 dataset is used for evaluation.

Model	Dice
Supervised GAN without integrated mutated model	40.68%
Supervised GAN with integrated mutated model	43.22%

. Since the valid set is not mutated, the mutation model succeeds in increasing and diversifying the training images by generating new images with no different distribution than the original one. Consequently, it enhances the proposed model performance using the original valid set.

Table 4. The training set of the ISLES-2018 dataset is used to present the average loss value of training the SNN module for 20 epochs with and without an integrated mutation model.

Model	loss
Supervised GAN without mutated images	42.99%
Supervised GAN with mutated images	42.86%

presents the average loss value of training the SNN module over 20 epochs. It is reduced slightly during the training of the mutated images. The proposed model has an extremely high loss value in the first epoch, then it reduces significantly in the second epoch, as illustrated in Figure 4. However, it suffers overfitting in the early epochs, and the loss cost is relatively high due to the SNN loss function combining both segmentation and discriminator loss functions. Nevertheless, the segmentation performance is satisfactory despite many obstacles in the ISLES-2018 dataset.

Table 5. The performance of the proposed model using the training set of the ISLES-2018 dataset.

Model	Recall
Proposed model	79.46%
3DJoinGANs [16]	79.00%
Voxel-GAN [32]	78.00%

presents that the proposed model outperforms the recall of other GAN models that use general augmentation techniques, such as the Gaussian noise technique, by utilizing the Euclidean distance to select the adjacent locations that have similar intensities for the mutation process. Where the recall has significant importance over the precision as it represents a ratio of the true positive samples to all test data [39].

5. Applications and Future Direction

The CT scans have a significant issue that is vital to be obtained and the number of existing images for specific diseases is significant challenge for deep learning models. The conventional methods use CT images due to is considered a challenges datasets and a potential resources for machine learning and deep learning models enhancements. Therefore, this section provides a future direction by suggesting exploiting the proposed augmentation technique in various applications.

5.1. Augmentation Technique

In addition to increasing the images related to ischemic stroke lesion segmentation, the proposed mutation process could be used for different wide image process applications and different types of images, such as breast lesion segmentation using dynamic contrast enhanced magnetic resonance imaging (DCE-MRI) [40].

5.2. Feature Extraction

The proposed distance map model that finds the adjusting regions has the ability to be used and enhanced to extract further features aiding in classification and segmentation tasks. For instance, extract further features using the proposed distance map for detecting coronary artery lesions [41], diagnosing neurodegenerative diseases, such as Alzheimer’s disease [42] and retinal vessels segmentation [43]. Furthermore, it has the ability to enhance for extracting features related to the volume of tissues, such as hepatic tumors [44] and clinical evaluation of muscle volume [45]. These features aiding to study the brain anatomy in neuroscience and neuroanatomy [46].

In addition, it has the ability to integrated to different machine learning models for different objects that not related to medical images such as Q-learning RL-based optimization algorithm (ROA) that proposed for natural scene image classification [47].

6. Conclusions

A new synthetic dataset to handle the limited number of images in the ISLES-2018 dataset for semantic segmentation using a model-based mutation and distance map has been generated, presented, and evaluated in this paper. The distance map preserves the structure shape of the human brain in the images. It prevents duplication of intensity values in the selected locations used for the mutation process. Furthermore, a semi-supervised GAN model is modified to a supervised GAN and enhanced using a shared module for both segmentation and discriminator to reduce the complexity of the end-to-end model. The proposed mutation model improves the dice coefficient of the proposed supervised GAN model by 2.54% using the original valid set of the ISLES-2018 dataset. Moreover, it enhances its recall compared to the existing GAN models, where the recall has significant importance over the precision as it represents a ratio of the true positive samples to all test data. However, it suffers overfitting, and the mutation model lacks adaptation. The self-organizing map (SOM) approach will be integrated into the proposed mutation model to achieve adaptation in future work.