Multi-Modal Rigid Image Registration and Segmentation Using Multi-Stage Forward Path Regenerative Genetic Algorithm

Abstract

Medical image diagnosis and delineation of lesions in the human brain require information to combine from different imaging sensors. Image registration is considered to be an essential pre-processing technique of aligning images of different modalities. The brain is a naturally bilateral symmetrical organ, where the left half lobe resembles the right half lobe around the symmetrical axis. The identified symmetry axis in one MRI image can identify symmetry axes in multi-modal registered MRI images instantly. MRI sensors may induce different levels of noise and Intensity Non-Uniformity (INU) in images. These image degradations may cause difficulty in finding true transformation parameters for an optimization technique. We will be investigating the new variant of evolution strategy of genetic algorithm as an optimization technique that performs well even for the high level of noise and INU, compared to Nesterov, Limited-memory Broyden–Fletcher–Goldfarb–Shanno algorithm (LBFGS), Simulated Annealing (SA), and Single-Stage Genetic Algorithm (SSGA). The proposed new multi-modal image registration technique based on a genetic algorithm with increasing precision levels and decreasing search spaces in successive stages is called the Multi-Stage Forward Path Regenerative Genetic Algorithm (MFRGA). Our proposed algorithm is better in terms of overall registration error as compared to the standard genetic algorithm. MFRGA results in a mean registration error of 0.492 in case of the same level of noise (1–9)% and INU (0–40)% in both reference and template image, and 0.317 in case of a noise-free template and reference with noise levels (1–9)% and INU (0–40)%. Accurate registration results in good segmentation, and we apply registration transformations to segment normal brain structures for evaluating registration accuracy. The brain segmentation via registration with our proposed algorithm is better even in cases of high levels of noise and INU as compared to GA and LBFGS. The mean dice similarity coefficient of brain structures CSF, GM, and WM is 0.701, 0.792, and 0.913, respectively.

1. Introduction

Medical images may be of the same or different types, such as Magnetic Resonance Imaging (MRI), Computerized Tomography (CT), and Positron Emission Tomography (PET). No single image modality provides complete information. Images of different modalities can provide more comprehensive information than those provided by a single modality. However, target and source images of the same object can be different and may have different alignments due to images taken at different time instants, different sources such as MRI, CT, and PET, or having different angles in order to have a 2D or 3D perspective [1]. Moreover, it is almost impossible to have the affected person’s head placed inside the scanner exactly the same way as the first time, so the information in the acquired images will not be spatially aligned [2]. Alignment of images is important to obtain complementary anatomic and functional information from multiple modalities for precise diagnosis and treatment of the patient [3]. Image registration is a fundamental problem in medical imaging and is often extensively used as a preliminary step to establishing correspondence between two images [4]. This enables us to compare common features in different images. In registration problems, one image is considered fixed and the other is considered a moving image. Aligning a moving image with any fixed image is the goal of registration. The alignment is done through the computation of spatial transformation [5,6]. Registration can be divided into four main components: feature space, search space, search strategy, and similarity. Each component provides essential information to decide which registration technique is to be used in a particular scenario. The area of interest used as the basis for registration is referred to as feature space, such as outlines, tumors, and edges. The transformation of moving an image to align to the source image is referred to as search space. Search strategy is the determination of the transformation technique to be chosen based on previous transformation results. The similarity is a metric of comparison between the source and target images that are being aligned. This forms the basis of how a registration problem can be framed [7]. This process has been rendered less experimental and more relevant recently by advances in image registration research. Registration methods have traditionally focused on the matching correspondence features in the images, but the research community has recently gained interest in similarity measures of a global correspondence nature. In this case, image registration performance has a direct dependency on the effectiveness of similarity measure for computing the similarity among images [8]. Multi-modality image registration based on comparing intensity values is not trivial because of the difference in intensity values of the same tissues in CT and MR images. Among the brain imaging modalities, MRI is able to scan tumor borders with a great level of detail and provides the best discrimination between soft tissues inside the brain [9]. An image can be geometrically represented and transformed in several ways, each with its own pros and cons. For accurate comparison and study, it is important that key biological landmarks are in the same place. Each approach does not work for all cases due to a number of different types of problems occurring while registering images [10]. Image registration is crucial in situations involving brain tumors, particularly well-defined glioblastoma multiforme, as the extraction of accurate morphological features depends on the correct alignment of the tumor zone. Image registration of multi-modal MRI scans of the brain is useful in the diagnosis of abnormalities. The accurate alignment of MRI sequences [11] results in accurate feature extraction and segmentation of the brain tumor. Moreover, deformable image registration is used in atlas-based image registration and segmentation [12]. In this strategy, an atlas (grayscale and labeled image) is registered to the patient’s brain MRI using its grayscale image. The deformation found using the non-rigid image registration technique is then applied to the counterpart of the atlas labeled image. The labels from registered labeled data of the atlas are then transferred to the patients’ data that directly segment brain structures along with anomalies. Image registration and segmentation can be used interchangeably. These processes can be used one after the other. The image registration aids in image segmentation while the results of image segmentation can be further used to improve image registration results, and hence image segmentation. Moreover, joint image registration and segmentation algorithms [13] also improve image registration and segmentation results. Furthermore, the segmentation results can be used to assess the registration accuracy as the image registration ground truth values are difficult to find as compared to the manual segmentation of brain structures by medical experts.

The main challenges in medical image registration are the presence of noise and INU. The performance of image registration may reduce as the pixels get affected by noise and INU, creating false local contours. The optimization techniques that are used to find a global optimum solution may be stuck in the local optimum due to noise and INU.

The main contribution of our research work is to introduce an evolutionary-based algorithm called MFRGA that performs well and finds a global optimum solution even for an increased level of noise and intensity INU. Our proposed methodology is not only successful in providing a global optimum solution but also the constituent multiple stages improve the accuracy of results.

2. Literature Review

Multi-modal image registration focuses on finding correspondence between images of different modalities and provides in depth visual information by fusion of multi-modal images. It is an important task in medical image analysis. Different studies are made in order to fully utilize its potential. A complete software framework based on multiresolution deformable transform, tackling elastic deformations possibly occurring during the surgical procedure, is presented [14] that enables the registration of MR and ultrasound (US) scans of the brain. The methods that involve the image registration technique for segmentation use the location-based information of the anatomical features. The atlas is used and transformed to segment the target image. It also preserves the brain structures information along with the detection of brain abnormalities [15]. However, the clustering methods [16,17] applied for the segmentation of both normal and abnormal tissues simultaneously are rare. The MRI sequence FLAIR is used for the segmentation of the brain tumor using sliding window and fuzzy c-means clustering to find the exact location of the tumor [18]. The segmentation of brain tumors, known as glioma, is done by applying a unified algorithm on multi-modal images from the BRATS 2015 dataset. This is comprised of estimation of the region of interest using fuzzy c-means clustering and region growing and then refining the glioma border using region merging and improved distance regularization level set method [19]. The MRI sequences T1, T2, and FSPRG T1C are used for brain tumor segmentation using a computational algorithm [20]. The algorithm unifies the enhancement of the region of interest by disconnecting adjacent brain structures and segmentation using detection by coordinates and arithmetic mean. Moreover, morphological operators are used to further improve the results.

Moreover, Razlighi et al. [8] have shown that the measure of similarity with high robustness is more efficient in the registration of degraded images. In their analysis, five different brain image modalities (T1, T2, PD, EPI, PET) and four different forms of misregistration (translation, rotation, scaling, and B-spline) are used to compute the robustness of selected similarity measures (SMs). They have shown that images with higher robustness are not only more tolerant to brain image degradation, but also more effective for intermodal image registration. Uss et al. [21] demonstrated that single complex combined SM based on five existing SMs, namely, Modality Independent Neighborhood Descriptor (MIND), Logarithmic-Likelihood Ratio (logLR), scale-invariant feature transform-octave (SIFT-OCT), phase correlation (PC), histogram of orientated phase congruency (HOPC), can be applied to general cases as well to a particular case under consideration for registration. In comparison to existing multi-modal SMs, the proposed complex combined SM increases the area under the curve by 1% to 21%. Haskins et al. [22] proposed a deep convolutional neural network to learn the similarity metric for MR–TRUS (trans-rectal ultrasound) registration. The learned similarity metric using a deep convolutional neural network outperformed the existing classic mutual information and feature-based methods [23,24]. A dual supervised deep learning model called BIRNet is proposed for image registration by prediction of deformation of the image from its appearance. The proposed model achieved state-of-the-art performance without the need for tuning parameters [25]. Mutual information as a similarity measure was first introduced in [26,27]. These studies were based on the assumption that the similar tissue region in one image would correspond to a similar region in the other image and will have similar grey values. Wood’s measure was adapted by [28]. They constructed a feature space by combining grey values for all corresponding points in each of the two images using a two-dimensional plot. However, they defined the regions in feature space with the difference to Wood’s method. The region in their method was based on clustering of registered images in feature space. Mutual information is a good choice for multi-modal image registration; however, local intensity variation in volumes and consideration of only statistical information are its limitations. To overcome these limitations, Ref. [29] proposed a method of adding spatial and geometrical information about the voxels via a 3D Harris operator. They focused on the registration of the low- and high-resolution images. Their proposed method came up with accurate registration and high performance with respect to other standard registration methods. Moreover, our previous work is related to the mono-modal image registration for brain MRI using MFRGA [30] with Structural Similarity Index Measure (SSIM) and performed better even for higher noise and INU.

3. Materials and Methods

Multi-modal image registration requires aligning images from different modalities so that the similarity measure is maximized between them. The dataset used here is a BrainWeb dataset [31]—an online simulator of the brain. The multi-modal images are taken from the mentioned dataset. The BrainWeb online simulator provides the facility of selecting T1, T2, and PD modalities of MRI sequences, varieties of color map, slice thickness, noise levels, and INU. T1 and T2 MRI images are selected as template and reference images. In this research, multi-modal image registration is performed using an evolutionary algorithm. Here, we proposed an improved version of the genetic algorithm with better performance in terms of accuracy even with high levels of noise and INU. Mutual information is used as a similarity measure between multi-modal images, with a higher value representing a greater degree of alignment and accuracy. The methodology of our research is described in this section.

3.1. Multi-Modal Image Registration

The registration of multi-modal images is a primary step in combining data from two or more images that are collected using different modalities. In addition to structural differences and intensity variations among images, partial or full information overlap among them adds an extra hurdle to the success of the registration process. Multi-modal image registration algorithms focus on finding the correspondence between images generated using various modalities and providing intensive visual information from the fusion of different medical imaging modalities. Registration while aligning two images into one geometry is always treated as an optimization problem. It is an iterative process, which is stopped by the optimizer. There are many optimizers, metrics, and different transformations and interpolators for implementation in the registration process.

3.2. Mutual Information

Mutual information (MI) is an information theory-based solution to image registration problems. Particularly, the MI similarity metric is used for the registration of multi-modal medical images [32]. It compares two images and measures the statistical dependence between pixel intensities in these images. MI as a registration measure was first presented by A. Collignon et al. [33], and it became a leading method in medical image registration. MI could be defined as a measure of information that one image contains about another. When the amount of information that images contain about each other is maximum, the images are believed to be correctly registered. If the images are multi-modal, then the predictability of one model from another is necessary. Predictability is closely associated with the notion of entropy. There is low entropy for a predictable random variable, while an unpredictable random variable has high entropy.

MI can be presented in many ways. The most common formulae are based on Shannon entropy and Kullback–Leibler distance measure between two probability distributions. If $H\left(A\right)$ and $H\left(B\right)$ are the entropies of random variables $A$ and $B$, respectively, while $H\left(A,B\right)$ is their joint entropy, then mutual information could be presented as in (1) [34]:

$$MI\left(A,B\right)=H\left(A\right)+H\left(B\right)-H\left(A,B\right)$$

(1)

where $H\left(A\right)$ and $H\left(B\right)$ are Shannon’s entropy of the image $A$ and $B$ calculated from the probability distribution of its pixels’ intensities.

The Kullback–Leibler measure MI is denoted by $MI\left(A,B\right)$, which measures the degree of dependence of $A$ and $B$ by measuring the distance between the joint distribution $P_{AB}\left(a,b\right)$ and the distribution associated to the case of complete independence $P_A\left(a\right)·P_B\left(b\right)$ and is expressed in (2) [35];

$$MI\left(A,B\right)={\displaystyle \sum }_{a,b}{P}_{A,B}\left(a,b\right)\log \left(\frac{P_{A,B}\left(a,b\right)}{P_A\left(a\right)·P_B\left(b\right)}\right)$$

(2)

For the two random variables $A$ and $B$ to be statistically independent, $P_{A,B}\left(a,b\right)=P_A\left(a\right)·P_B\left(b\right)$ and $MI\left(A,B\right)=0$. For maximal statistical dependence of two random variables, one-to-one correspondence between them is $T:P_A\left(a\right)=P_B\left(T\left(a\right)\right)=P_{AB}\left(a,T\left(a\right)\right)$. The random variables $A$ and $B$ are considered to be the two images to be registered. One is the reference image, and the other is the template image. The pair of corresponding voxels in the images to be registered are referred as image intensity values $a$ and $b$. The geometric transformation $T$ relates the intensity values $a$ and $b$ in images $A$ and $B$, respectively. The two images are best aligned with the geometric transformation $T$ when $MI\left(A,B\right)$ is maximal.

3.3. Optimization Techniques

The following optimization techniques are used in this research.

3.3.1. Nesterov’s Accelerated Gradient

Nesterov’s Accelerated Gradient (NAG) [36] is an extension to gradient descent optimization. In the classical momentum, the update of parameters is performed by finding the gradient in the direction of the updated accumulated gradient. On the other hand, NAG first takes a big step by using the previous accumulated gradient and then calculating the gradient for making the correction [37]. It also avoids the problem of overshooting the minima. The update rule (3)–(5) is as follows as described in [38,39].

$$f\left(\theta \right)={\theta }_t-\gamma \ast updat{e}_{t-1}$$

(3)

$$updat{e}_t=\gamma \ast updat{e}_{t-1}+\eta {\nabla }_{\theta }f\left(\theta \right)$$

(4)

$${\theta }_{t+1}={\theta }_t-updat{e}_t$$

(5)

where $f\left(\theta \right)$ is the objective function with $\theta$ as the parameters of the model to update. $\eta$ is the step size, $\gamma$ is the momentum term, and $updat{e}_t$ is the current time step update vector.

3.3.2. Simulated Annealing

Simulated annealing (SA) is an optimization technique that is based on the theory of thermodynamics for annealing ideal crystals [40]. It models the physical process of heating and then slowly cooling the temperature of the material [41]. The controlled lowering of temperature increases the crystal size and hence reduces the defects. SA is used to find the global optimum solution where there are many local optimum solutions. The brief mechanism of SA is as follows.

Let the cost of the current state $\theta$ be represented as $f\left(\theta \right)$ and the cost of the neighboring state ${\theta }^{\prime }$ be represented as $f\left({\theta }^{\prime }\right)$. The difference $D$ between $f\left({\theta }^{\prime }\right)$ and $f\left(\theta \right)$ is as described in [42].

$$D=f\left({\theta }^{\prime }\right)-f\left(\theta \right)$$

(6)

If $D<=0,$ the neighboring state cost is less than or equal to the current state cost. ${\theta }^{\prime }$ is selected as the current state due to the acceptance of downhill criteria. If $D>0,$ such that $e^{-\frac{D}{T}}>V_{RND}$ the acceptance chances of ${\theta }^{\prime }$ as the current state is more as compared to the random value $V_{RND}$, where $0<V_{RND}<1$ and $T$ is the controlling parameter temperature. Moreover, the current state $\theta$ continues to be the candidate solution if $D>0,$ such that $e^{-\frac{D}{T}}<=V_{RND}$.

3.3.3. LBFGS (Limited-Memory BFGS)

Limited-memory BFGS is an optimization algorithm in the family of quasi-Newton methods that approximates the Broyden–Fletcher–Goldfarb–Shanno algorithm (BFGS) using a limited amount of computer memory. It is a popular algorithm for parameter estimation in machine learning. Limited-memory BFGS works well with large datasets because it uses less memory than the standard BFGS. It is also referred to as software for large-scale unconstrained optimization.

The algorithm’s target problem is to minimize $f\left(x\right)$ over unconstrained values of the real vector $x$, where $f$ is a differentiable scalar function. The algorithm starts with an initial estimate of the optimal value, $X_0$, and proceeds to refine the estimate iteratively with a sequence of better estimates, $X_1,X_2,\dots$. As a key driver of the algorithm, the derivatives of the function $g_k:=\Delta f\left(X_k\right)$ are used to define the steepest descent path and also to estimate the Hessian matrix (second derivative) of $f\left(x\right)$.

3.3.4. Single-Stage Genetic Algorithm

Single-Stage Genetic Algorithm (SSGA) is similar to a standard genetic algorithm. Here, individuals are rounded to increasing decimal places in successive stages and then fitness scores are calculated. Search space constraints for SSGA are also user-defined and vary if it is part of MFRGA. The description of SSGA for multi-modal rigid image registration is mentioned in Algorithm 1. The population size of 50 is selected with stall generations of 50 and tolerance function of 10⁻⁶. The results are achievable if it reaches a maximum number of generations 300 or stall generations of 50. Here, stall generation 50 shows the average change in fitness function value is less than 10⁻⁶ for 50 generations.

3.4. Proposed Optimization Technique for Multi-Modal Image Registration

3.4.1. Multi-Stage Forward Path Regenerative Genetic Algorithm

Multi-stage Forward Path Regenerative Genetic Algorithm (MFRGA) is our proposed strategy, which contains iterative stages of (SSGA) with increasing precision levels of individuals and decreasing search space constraints to obtain accurate results. MFRGA is different from the standard genetic algorithm in terms of the use of multiple stages, restricted search space, and precision levels of individuals. In each proceeding stage, the search space decreases, and the precision levels of individuals increase. The image registration result of the previous stage is used by the next stage. The use of multiple stages of SSGA ensures to produce more accurate results by improving image registration results in each preceding stage. The description of MFRGA for multi-modal rigid image registration is mentioned in Algorithm 2.

3.4.2. Image Registration Error and Segmentation Accuracy

The performance of our proposed algorithm is computed using image registration error and segmentation accuracy. Less image registration error means more segmentation accuracy. The following metrics are considered for registration and segmentation accuracy.

Absolute Percentage Registration Error in Each Dimension

Percentage registration error is computed in each dimension, i.e., $x$, $y$ and $\theta$. Percentage registration error for each direction is as follows:

$$\Delta i=\left|\frac{C_i-G_i}{G_i}\right|\times 100, for (i=x,y,\theta ) \wedge (G_i\ne 0)$$

(7)

where $\Delta x$, $\Delta y$ and $\Delta \theta$ are absolute percentage error in $x,y$ translation and $\theta$ rotation, respectively. $C_x$, $C_y$ and $C_{\theta }$ are computed values from applied image registration and $G_x$, $G_y$ and $G_{\theta }$ are ground truth transformation for $x,y$ translations and $\theta$ rotation, respectively.

Algorithm 1: SSGA (Single-Stage Genetic Algorithm) for Multi-Modal Rigid Image Registration.

Result: Rigid image transformations $\left(x,y and \theta \right)$

$I_R\leftarrow$ Reference Image of modality $M_1$

$I_T\leftarrow$ Template Image of modality $M_2$

$C_{min}\leftarrow$ Minimum constraint of search space

$C_{max}\leftarrow$ Maximum constraint of search space

$S_{pop}\leftarrow$ Population Size

$D\leftarrow$ Number of decimal places

Function SSGA($I_R$, $I_T$, $C_{min}$, $C_{max}$, $D$):

Initial random population of individuals containing three parameters of $x,y$ translation and θ rotation of $S_{pop}$ is generated. Individuals are rounded to $D$ decimal places.

while No. of generations < Maximum no. of generations do

If$\left(x,y and \theta \right)\in \left[C_{min},C_{max}\right]$ then 1.  Each individual takes part in evaluating fitness value in fitness function using MI as a similarity measure. The best fitness value is the smallest value selected from the population. 2.  Fitness scores are used for the selection of parents within population. 3.  Cross over is used to generate offspring. 4.  Mutation is applied to each offspring. 5.  Fitness of intermediate population is evaluated. 6.  Fittest individuals are promoted to next generation.

else

return fittest_individuals

end

end

End Function

Overall Registration Error

Overall registration error includes all dimensional transformation errors in one metric. It is defined as follows:

$$RE={\displaystyle \sum }_i\left|C_i-G_i\right|, for (i=x, y, \theta )$$

(8)

where $RE$ is the overall registration error, which is the sum of absolute differences of computed transformations from the image registration algorithm, i.e., $C_i$ and the ground truth transformations, i.e., $G_i$ with $i$ corresponds to dimension index in the image registration algorithm.

Dice Similarity

The template image with both grayscale and labeled image can be used not only for registration but also segmentation. Here, segmentation accuracy is computed after registering grayscale images and applying the same transformations to the labeled image to further segment the reference image. Segmentation accuracy can be found using the Dice Similarity Index (DSI). We found DSI of individual normal brain structures as well as overall brain containing all structures together considered as brain pixels. DSI is defined as follows:

$$DSC=\frac{2\ast \left|S_I{\displaystyle \cap }{G}_T\right|}{\left|S_I|+|G_T\right|}$$

(9)

where $DSC$ is the Dice Similarity Coefficient, segmented image is denoted by $S_I$, and ground truth of the image is denoted by $G_T$.

Algorithm 2: Multi-Stage Forward Path Regenerative Genetic Algorithm (MFRGA) for Multi-Modal Rigid Image Registration

Result: Registered image $I_{Tk}$ with rigid image transformations $\left(x,y and \theta \right)$

$I_R\leftarrow$ Reference Image of modality $M_1$

$I_T\leftarrow$ Template Image of modality $M_2$

$C_{min}\leftarrow$ Minimum constraint of search space

$C_{max}\leftarrow$ Maximum constraint of search space

$S_{pop}\leftarrow$ Population Size

$D\leftarrow$ Number of decimal places

$x_1,x_2,x_3\leftarrow$ First stage parameters

$x_1^i,x_2^i,x_3^i\leftarrow$ Other stages parameters,

where $i\in {\{}^{\prime }{,}^{″}{,}^{‴}\}$

$Stage\_No\leftarrow$ Stage number

$Stage\_No=1$

while$Stage\_No<\mathit{4}$do

If$(Stage\_No==\mathit{1})$ then

1.   Call SSGA $(I_R,I_T,-10,10,0)$ 2.   Store the output of SSGA function 3.   return $x_1,x_2,x_3$ 4.   Apply image transformations $x_1,x_2,x_3$ to get registered template image $I_{Tk1}$

end

end if$(Stage\_No==\mathit{2})$ then

Call SSGA $(I_R,I_{Tk1},-0.5,0.5,1)$

1.   Store the output of SSGA function 2.   return $x_1^{\prime },x_2^{\prime },x_3^{\prime }$ 3.   Apply image transformations $x_1^{\prime },x_2^{\prime },x_3^{\prime }$ to get registered template image $I_{Tk2}$

else if$(Stage\_No==\mathit{3})$ then

1.   Call SSGA $(I_R,I_{Tk2},-0.25,0.25,2)$ 2.   Store the output of SSGA function 3.   return $x_1^{″},x_2^{″},x_3^{″}$ 4.   Apply image transformations $x_1^{″},x_2^{″},x_3^{″}$ to get registered template image $I_{Tk3}$

else if$(Stage\_No==\mathit{4})$ then

1.   Call SSGA $(I_R,I_{Tk3},-0.125,0.125,3)$ 2.   Store the output of SingleStageGA function 3.   return $x_1^{‴},x_2^{‴},x_3^{‴}$ 4.   Apply image transformations $x_1^{‴},x_2^{‴},x_3^{‴}$ to get registered template image $I_{Tk4}$

else

return fittest_individuals

$Stage\_No =Stage\_No+1$

end

end

4. Results

In order to evaluate the performance of our algorithm in rigid image registration, we need a reference image translated in the x and y directions and also rotated. We translated reference image T2 five pixels in the x-direction and seven pixels in the y-direction and 3⁰ rotations. These are considered ground truth transformations, and we are here to find these unknown transformations to which template image will best align with the reference image. The total number of brain MRI sequence combinations of template and reference images is 31. We divide our MRI sequence combination of template and reference into two sets. The first set contains MRI sequences where we have the same levels of noise and INU for template and reference images. The second set contains MRI sequences with template images free of noise and INU, while the reference images have different levels of noise and INU. We have evaluated our results in three different aspects, which are as follows.

4.1. Template and Reference Images with Same Levels of Noise and INU

In the first case shown in Figure 1a, both template T1 and reference T2 have the same levels of noise and INU. We can add noise levels of (0, 1, 3, 5, and 7)% and (0, 20, and 40)% of INU. We generated 16 cases of both template and reference images with varying noise levels and INU. The noise field added in the real and imaginary components of BrainWeb MRI sequences is the Gaussian noise. Experiments are performed using both Single-Stage Genetic Algorithm and MFRGA and accuracy is computed where the template and reference images under experimentation are at same levels of noise and INU.

Table 1. Performance of MFRGA vs. GA with same level of noise and INU of template and reference images.

INU	Noise	Stage 1			Stage 2			Stage 3			Stage 4			Final
		X1	X2	X3	X1′	X2′	X3′	X1″	X2″	X3″	X1‴	X2‴	X3‴	X	Y	θ
0%	0%	5.382	7.376	3.151	−0.363	−0.390	0.242	−0.003	0.003	−0.240	−0.001	0.001	−0.051	5.015	6.989	3.102
0%	1%	4.964	6.799	2.502	0.147	0.191	−0.001	−0.003	0.034	0.225	−0.117	−0.035	−0.001	4.991	6.989	2.724
20%	1%	4.696	7.171	2.251	0.269	−0.117	0.490	−0.002	0.061	0.003	0.000	−0.048	0.098	4.963	7.067	2.842
40%	1%	4.854	6.787	3.414	0.226	0.212	0.198	0.000	0.014	0.087	0.003	−0.008	−0.045	5.083	7.005	3.655
0%	3%	5.331	6.954	2.937	−0.322	0.066	0.179	−0.001	−0.026	−0.169	0.000	−0.010	−0.004	5.009	6.985	2.942
20%	3%	5.284	6.541	3.351	−0.279	0.477	0.018	0.001	0.002	0.211	0.000	−0.001	0.016	5.006	7.018	3.596
40%	3%	4.831	6.688	3.234	0.214	0.272	−0.094	−0.001	0.023	0.041	−0.012	0.004	−0.058	5.031	6.987	3.123
0%	5%	4.137	7.813	5.170	0.462	−0.474	−0.457	0.249	−0.249	0.212	0.125	−0.125	0.032	4.973	6.965	4.958
20%	5%	4.580	7.262	3.101	0.387	−0.156	−0.200	0.246	−0.251	−0.033	0.120	−0.124	0.084	5.333	6.731	2.953
40%	5%	4.753	7.409	3.090	0.217	−0.365	−0.034	0.000	−0.001	−0.255	0.002	−0.002	0.033	4.972	7.041	2.835
0%	7%	5.242	6.875	1.650	−0.258	0.299	0.491	−0.072	−0.127	−0.010	0.034	0.063	0.018	4.946	7.110	2.149
20%	7%	5.442	7.047	3.030	−0.358	−0.081	−0.008	−0.033	−0.012	0.229	−0.005	−0.003	0.111	5.046	6.951	3.362
40%	7%	5.244	6.822	3.113	−0.240	0.244	0.155	−0.004	−0.033	−0.080	−0.001	−0.004	0.067	4.998	7.028	3.255
0%	9%	5.116	7.006	2.519	−0.028	−0.098	0.089	−0.103	0.129	0.219	0.040	−0.069	0.059	5.024	6.968	2.887
20%	9%	5.315	7.288	3.025	−0.252	−0.283	0.016	−0.014	0.000	−0.226	0.000	−0.001	0.089	5.049	7.004	2.903
40%	9%	5.436	6.740	2.834	−0.370	0.193	−0.207	−0.013	0.024	−0.203	−0.005	0.003	0.062	5.048	6.961	2.486

shows the performance of MFRGA vs. Single-Stage Genetic Algorithm which is clear in terms of less registration error in the case of MFRGA. All the available combinations of noise levels and INU have been selected. In the first stage, search space constraints are [−10,10], the result of single stage is near the final or true transformation but further improvement in the results can be made with our proposed MFRGA, having successive stages of GA with reducing search space constraints and iterative GA.

The result from the first stage is within error limits of [−0.5, +0.5]. Keeping in mind this fact, the second, third, and fourth stages of GA contains reducing search space of [−0.5, 0.5], [−0.25, 0.25], and [−0.125, 0.125], improving the accuracy even in the case of increasing noise levels and INU. For most of the cases in

Table 2. Image Registration Error of MFRGA vs. GA with same level of noise and INU of the template and reference images.

Template T1 and Reference T2 Images		Registration Error Using Single-Stage GA			Registration Error Using MFRGA			Overall Registration Error Using Single-Stage GA $\nabla Y(\%)$	Overall Registration Error Using MFRGA $\nabla \mathsf{\theta }(\%)$
INU	Noise	$\nabla X(\%)$	$\nabla Y(\%)$	$\nabla \mathsf{\theta }(\%)$	INU	Noise	$\nabla X(\%)$
0%	0%	7.640	5.371	5.033	0%	0%	7.640	5.371	5.033
0%	1%	0.713	2.875	16.617	0%	1%	0.713	2.875	16.617
20%	1%	6.090	2.436	24.963	20%	1%	6.090	2.436	24.963
40%	1%	2.922	3.042	13.786	40%	1%	2.922	3.042	13.786
0%	3%	6.620	0.651	2.104	0%	3%	6.620	0.651	2.104
20%	3%	5.689	6.553	11.712	20%	3%	5.689	6.553	11.712
40%	3%	3.386	4.460	7.793	40%	3%	3.386	4.460	7.793
0%	5%	17.268	11.616	72.341	0%	5%	17.268	11.616	72.341
20%	5%	8.390	3.739	3.380	20%	5%	8.390	3.739	3.380
40%	5%	4.945	5.840	2.997	40%	5%	4.945	5.840	2.997
0%	7%	4.844	1.781	44.988	0%	7%	4.844	1.781	44.988
20%	7%	8.838	0.664	0.997	20%	7%	8.838	0.664	0.997
40%	7%	4.883	2.548	3.765	40%	7%	4.883	2.548	3.765
0%	9%	2.317	0.093	16.035	0%	9%	2.317	0.093	16.035
20%	9%	6.295	4.107	0.824	20%	9%	6.295	4.107	0.824
40%	9%	8.715	3.712	5.531	40%	9%	8.715	3.712	5.531

, $\Delta x$, $\Delta y$ and $\Delta \theta$ for MFRGA is less than single-stage genetic algorithm. The results of

Table 3. Image Registration Error of MFRGA vs. GA with template T1 (INU and Noise 0%) and reference T2 with different level of noise and INU.

Reference T2 Image		Registration Error Using Single-Stage GA			Registration Error Using MFRGA			Overall Registration Error Using Single-Stage GA $\nabla Y(\%)$	Overall Registration Error Using MFRGA $\nabla \mathsf{\theta }(\%)$
INU	Noise	$\nabla X(\%)$	$\nabla Y(\%)$	$\nabla \mathsf{\theta }(\%)$	INU	Noise	$\nabla X(\%)$
0%	1%	9.839	3.441	12.793	0%	1%	9.839	3.441	12.793
20%	1%	4.128	2.261	8.854	20%	1%	4.128	2.261	8.854
40%	1%	27.382	2.373	7.534	40%	1%	27.382	2.373	7.534
0%	3%	6.738	0.394	8.362	0%	3%	6.738	0.394	8.362
20%	3%	3.109	1.943	1.200	20%	3%	3.109	1.943	1.200
40%	3%	6.090	2.603	13.394	40%	3%	6.090	2.603	13.394
0%	5%	1.974	19.216	1.103	0%	5%	1.974	19.216	1.103
20%	5%	6.325	6.807	7.099	20%	5%	6.325	6.807	7.099
40%	5%	0.177	6.422	15.544	40%	5%	0.177	6.422	15.544
0%	7%	6.563	1.592	3.771	0%	7%	6.563	1.592	3.771
20%	7%	6.020	6.207	10.970	20%	7%	6.020	6.207	10.970
40%	7%	5.289	6.224	9.783	40%	7%	5.289	6.224	9.783
0%	9%	5.618	0.883	11.319	0%	9%	5.618	0.883	11.319
20%	9%	1.193	3.101	11.380	20%	9%	1.193	3.101	11.380
40%	9%	4.195	6.234	5.428	40%	9%	4.195	6.234	5.428

are obtained for the case of reference image with INU and noise 0% and template image with all the cases of increasing INU and noise levels. It is clear that the results of MFRGA are more close to the ground truth transformations as compared to SSGA.

Table 4. Performance of MFRGA vs. GA with template T1 (INU and Noise 0%) and reference T2 with different levels of noise and INU.

Reference T2 Image		Stage 1			Stage 2			Stage 3			Stage 4			Final
INU	Noise	X1	X2	X3	X1′	X2′	X3′	X1″	X2″	X3″	X1‴	X2‴	X3‴	X	Y	θ
0%	1%	5.492	6.759	3.384	−0.456	0.330	−0.082	−0.004	−0.056	0.124	−0.003	−0.005	−0.001	5.029	7.027	3.424
20%	1%	5.206	7.158	3.266	−0.155	−0.220	0.032	−0.023	0.077	−0.124	−0.004	0.000	−0.011	5.025	7.015	3.162
40%	1%	6.369	7.166	2.774	−0.491	−0.188	−0.021	−0.247	0.023	0.196	−0.124	0.001	0.089	5.507	7.002	3.039
0%	3%	5.337	6.972	3.251	−0.268	0.061	0.246	−0.045	−0.080	−0.249	0.001	0.105	−0.018	5.025	7.058	3.230
20%	3%	5.155	6.864	2.964	−0.079	0.111	0.021	−0.055	0.000	0.234	−0.001	−0.001	0.015	5.020	6.974	3.234
40%	3%	4.695	6.818	3.402	0.267	0.233	−0.056	0.040	−0.047	0.130	0.003	0.001	−0.065	5.005	7.004	3.411
0%	5%	5.099	5.655	2.967	−0.194	0.497	−0.254	0.081	0.247	0.243	−0.033	0.123	−0.034	4.952	6.522	2.922
20%	5%	4.684	7.477	2.787	0.299	−0.537	0.333	0.002	0.009	−0.023	0.000	0.004	−0.096	4.985	6.953	3.001
40%	5%	4.991	6.550	2.534	0.090	0.408	0.254	−0.100	0.033	−0.138	0.045	0.000	−0.111	5.027	6.991	2.539
0%	7%	4.672	7.111	2.887	0.266	−0.129	0.128	0.047	0.000	0.002	0.002	−0.002	−0.038	4.987	6.981	2.980
20%	7%	5.301	7.434	3.329	−0.345	−0.389	−0.002	0.045	0.004	0.050	0.013	−0.003	0.065	5.014	7.046	3.443
40%	7%	4.736	7.436	3.293	0.263	−0.405	0.040	0.004	0.007	−0.207	0.000	−0.001	−0.079	5.003	7.037	3.047
0%	9%	4.719	7.062	2.660	0.299	−0.143	0.350	0.001	0.091	0.038	0.002	−0.019	−0.052	5.022	6.991	2.996
20%	9%	5.060	6.783	3.341	0.050	0.227	0.034	0.004	−0.012	0.019	0.001	0.008	−0.110	5.114	7.005	3.284
40%	9%	5.210	6.564	2.837	−0.236	0.359	−0.194	−0.002	0.021	0.083	0.008	0.000	0.083	4.980	6.942	2.809

shows that the overall registration error for all cases of MFRGA is less than single-stage genetic algorithm and is also shown in Figure 2a–c with increasing INU of 0, 20, and 40%, respectively.

4.2. Segmentation Accuracy via Image Registration

In the third case shown in Figure 1c, we evaluated the robustness of our proposed algorithm of genetic algorithm in comparison to Nesterov, SA, LBFGS, and GA. Here, we have taken the case of the template that was free of noise and INU and reference image with all possible combinations of noise and INU. The template image has both a grayscale MRI image and a labeled image. The more accurate the image registration is, the more accurate the segmentation will be. Nesterov, SA, LBFGA, GA, and MFRGA all are applied for image registration, and the same transformations are applied to the labeled image. In this way, images are not only registered but also get segmented. Segmentation results are computed using Dice Similarity (DS) index. A higher value of DS shows more segmentation accuracy, which is the result of accurate image registration.

Table 5. Normal brain structure segmentation via registration techniques with template (INU and Noise 0%) and reference with noise (1%, 3%, 5%, 7%, and 9%) and INU (0%, 20%, and 40%).

DSC	WM	DSC	GM	DSC	CSF	DSC	WM	DSC	GM	DSC	CSF	DSC	WM	DSC	GM	DSC	CSF	DSC	WM	DSC	GM	DSC	CSF	DSC	WM	DSC	GM	DSC	CSF	Noise	INU
0.91		0.77		0.69		0.91		0.78		0.69		0.88		0.8		0.71		0.917		0.804		0.709		0.68		0.491		0.397		1%	0%
0.92		0.8		0.7		0.91		0.79		0.7		0.88		0.8		0.71		0.913		0.792		0.698		0.693		0.509		0.395		1%	20%
0.92		0.81		0.71		0.87		0.67		0.57		0.88		0.8		0.71		0.884		0.714		0.633		0.694		0.51		0.392		1%	40%
0.91		0.79		0.7		0.91		0.79		0.7		0.64		0.41		0.25		0.919		0.807		0.713		0.692		0.509		0.394		3%	0%
0.91		0.79		0.7		0.92		0.81		0.71		0.88		0.8		0.71		0.907		0.773		0.682		0.693		0.509		0.394		3%	20%
0.91		0.78		0.69		0.91		0.77		0.68		0.88		0.8		0.71		0.917		0.802		0.709		0.694		0.511		0.393		3%	40%
0.92		0.8		0.71		0.87		0.69		0.61		0.64		0.41		0.25		0.919		0.807		0.713		0.687		0.504		0.391		5%	0%
0.92		0.81		0.71		0.91		0.79		0.7		0.88		0.81		0.71		0.909		0.785		0.692		0.687		0.504		0.39		5%	20%
0.91		0.77		0.67		0.9		0.77		0.69		0.64		0.41		0.25		0.919		0.806		0.71		0.688		0.508		0.397		5%	40%
0.92		0.81		0.71		0.92		0.8		0.7		0.64		0.41		0.25		0.919		0.807		0.712		0.686		0.501		0.39		7%	0%
0.91		0.77		0.69		0.91		0.78		0.69		0.64		0.41		0.25		0.914		0.792		0.697		0.687		0.503		0.393		7%	20%
0.92		0.8		0.71		0.91		0.78		0.69		0.64		0.41		0.25		0.914		0.797		0.707		0.687		0.504		0.392		7%	40%
0.92		0.81		0.71		0.91		0.78		0.69		0.64		0.41		0.25		0.915		0.798		0.706		0.681		0.491		0.393		9%	0%
0.91		0.79		0.7		0.91		0.78		0.7		0.64		0.41		0.25		0.895		0.757		0.684		0.68		0.491		0.394		9%	20%
0.91		0.8		0.7		0.92		0.8		0.7		0.88		0.8		0.71		0.919		0.807		0.711		0.681		0.491		0.391		9%	40%

shows the segmentation results of Nesterov, SA, LBFGS, GA, and MFRGA. Here, the labeled image of the template contains labels of three normal brain structures, i.e., GM (Gray Matter), WM (White Matter), and CSF (Cerebellum Spinal Fluid). Clearly, the MFRGA has better results than that of Nesterov and LBFGS and comparable for the case of GA and SA. Figure 3 shows that MFRGA performs well even in the case of increasing noise levels and INU. We have already proved in Table 3 that MFRGA performs well in comparison to GA.

5. Discussion

Multi-modal image registration is an essential task in medical image analysis. Information from different modalities is combined to form a more accurate analysis. Sensors of imaging modalities may undergo, translations and rotation. In this regard, it is necessary to align both images to obtain useful information. If one of the images has its corresponding segmentation, then the labeled image can also be transformed using the same transformations as being applied on the grayscale image to align with the reference image. Our proposed algorithm consists of iterative and successive stages of the Genetic Algorithm with reduced search space and increased precision of GA chromosomes. The geometrical transformations obtained from MFRGA as compared to the SSGA are closed to the true transformations and are shown in Table 1 and Table 3. It is clear that the $x$ and $y$ translational and rotational transformations from MFRGA as compared to SSGA are closer to 5, 7, and 3 pixels, respectively. As a result of these geometrical transformations, the overall registration error of MFRGA is reduced as compared to SSGA and is clear from Table 2 and Table 4. The reduced overall registration error refers to the fact that results are better and closer to the true transformations.

The statistical analysis of results from

Table 6. Statistical Analysis of Results.

Result	Model	Mean ± Standard Deviation	Min	Max
Overall Registration Error	GA *	1.008 ± 0.820	0.440	3.847
	Proposed (MFRGA) *	0.492 ± 0.487	0.083	2.020
	GA **	0.898 ± 0.369	0.327	1.761
	Proposed (MFRGA) **	0.317 ± 0.195	0.035	0.604
Segmentation Accuracy (Dice CSF)	Nesterov **	0.393 ± 0.002	0.390	0.397
	SA **	0.698 ± 0.021	0.633	0.713
	LBFGS **	0.461 ± 0.239	0.245	0.710
	GA **	0.681 ± 0.038	0.572	0.712
	MFRGA **	0.701 ± 0.012	0.674	0.713
Segmentation Accuracy (Dice GM)	Nesterov **	0.502 ± 0.008	0.491	0.511
	SA **	0.790 ± 0.025	0.714	0.807
	LBFGS **	0.592 ± 0.205	0.406	0.805
	GA **	0.772 ± 0.039	0.672	0.805
	MFRGA **	0.792 ± 0.013	0.773	0.807
Segmentation Accuracy (Dice WM)	Nesterov **	0.687 ± 0.005	0.680	0.694
	SA **	0.912 ± 0.010	0.884	0.919
	LBFGS **	0.753 ± 0.126	0.639	0.884
	GA **	0.905 ± 0.016	0.868	0.918
	MFRGA **	0.913 ± 0.005	0.905	0.919

* Image registration with template and reference image at the same level of INU (0–40%) and noise (0–9%). ** Image registration with template (0% noise and 0% INU) and reference (1–9% noise and 0–40% INU).

proved MFRGA to be better as compared to GA with a mean registration error of 0.492 for the same level of noise and INU in both reference and template images and 0.317 in case of the template with no noise and INU and varying levels of noise and INU for the reference image. Here, the minimum registration error of the proposed methodology is 0.035 and the maximum is 2.02. The minimum registration error found in MFRGA is 10 times less than that in GA. The maximum registration error in GA is 3.847, which is higher than that found in MFRGA. Better registration accuracy provides better segmentation results. Dice similarity is used to evaluate the segmentation accuracy of brain structures, i.e., CSF, GM, and WM. The comparison is performed with Nesterov, SA, GA, and LBFGS optimization techniques. The higher value of dice coefficient shows better segmentation results. The mean dice value using MFRGA for CSF, GM, and WM is 0.701, 0.792, and 0.913. The minimum and maximum dice values of MFRGA are higher than Nesterov, GA, and LBFGS and comparable with SA. The higher value of the minimum and maximum dice values shows that MFRGA is robust and yields reliable results even for increasing noise and INU values. In almost all of the experiments, MFRGA provides good segmentation accuracy.

6. Conclusions

Evolutionary algorithms are considered in the class of the most popular optimization techniques. The robustness of the evolutionary algorithm is computed in this research along with the effectiveness of our proposed MFRGA. Multiple stages of MFRGA guarantee to provide more accurate results in comparison to single-stage genetic algorithm. Decreasing search space and increasing precision levels force the converged results to reduce the registration errors and hence provide more accurate results. Its performance is prominent in our experiments with increasing noise up to 9% and INU up to 40% levels. The overall registration error, including x, y translations and rotation of MFRGA, is much less than single-stage genetic algorithm in all cases of increasing noise and INU levels. The mean registration error of MFRGA is less than 0.5. Even the maximum registration error using MFRGA found out of all experiment cases is less than that of GA. This shows good performance of MFRGA even for the worst case. Moreover, good segmentation accuracy of normal brain structures was recorded via registration using MFRGA in comparison to GA and LBFGS. The mean dice similarity coefficient for CSF, GM, and WM obtained is 0.701, 0.792, and 0.913, respectively, with a standard deviation 10 times less than 0.1. The difference between minimum and maximum dice coefficient values is almost less than 0.05, with minimum and maximum dice similarity values higher than GA and LBFGS.

The statistical analysis shows the effectiveness of MFRGA in all cases of changing noise and INU levels. MFRGA can further be extended to the most challenging deformations at pixel level called non-rigid image registration where an increased accuracy can be achieved with better performance even for higher noise and INU levels. Moreover, the rigid image registration is crucial for extracting useful information from multiple modalities. It is used to remove global transformation differences among the multi-modal medical images. The aligned images aid the physicians and surgeons in analyzing medical diagnostics. In future, the atlas-based image registration combined with improved clustering methods [16,17] may result in more accuracy for simultaneous segmentation of both normal and abnormal.