1. Introduction
Image registration is one of the well-known techniques belonging to the computer vision field [
1,
2,
3]. In the last few years, nature-inspired algorithms and metaheuristics have been used to address the image registration problem, becoming a solid alternative to direct optimization methods. Even though image registration using the nature-inspired algorithms represents a niche in the research, a variety of evolutionary algorithm techniques has been proposed, starting from the most used, Genetic Algorithm, and moving to the newest, Coral Reefs Optimization [
4,
5,
6,
7,
8,
9,
10,
11].
The aim of the research work presented in this paper is to accurately register binary images using evolutionary search techniques. The considered perturbation is of rigid type. The proposed metaheuristics use symmetric uncertainty, designed to maximize the similarity between the computed variants of the sensed image and the target one. The aim is to develop a population-based evolutionary optimization model, in which the individuals evolve toward the chromosome corresponding to the given target image. Note that our approaches can register both gray-scale and colored images after a pre-processing step designed to compute the boundaries.
The rest of the paper is organized as follows. A brief review of the literature regarding the state of the art in the field of metaheuristics for image registration is provided in
Section 2. In
Section 3, the 2M-ES local search implemented for the rigid perturbation model is briefly outlined. The use of the Firefly algorithm—developed based on the updating of two rules introduced in [
12]—to register binary images is presented in
Section 4. Note that the versions of the Firefly algorithm proposed in [
12] proved extremely accurate, but very time consuming. In order to reduce the computational effort without decreasing the accuracy, we developed two new methods, provided in
Section 5 and
Section 6.
A two-stage hybrid technique, that involves a population-based Firefly search and a variant of 2M-ES as the local search algorithm, is introduced in
Section 5. The hybrid algorithms consist of two stages. In the first stage, a variant of Firefly technique [
12] computes a “good” candidate solution, i.e., a chromosome whose fitness is larger than a certain threshold value. The obtained individual is considered as the initial solution of the 2M-ES method. Briefly, the Firefly search technique directs the exploration in an appropriate direction, then the 2M-ES algorithm is used to compute an optimal solution. In
Section 6 of the paper, we introduce a memetic algorithm that embeds a version of the Firefly algorithm and 2M-ES.
The accuracy and efficiency of the proposed algorithms are experimentally confirmed by our tests outlined in
Section 7. The conclusions of our study are provided in
Section 8 of the paper.
2. Literature Review
Taking into account the main application of the methodology and the type of the optimization problem, various straightforward and hybrid methods have been presented in the literature.
In [
11], the bio-inspired meta-heuristic Coral Reef Optimization Algorithm with Substrate Layers (CRO-SL) has been introduced. The algorithm is used to solve the real-coding image registration problem focusing on both mono-modal and inter-modal scenarios, the perturbation model being of 3D affine type. It has been experimentally established that CRO-LS is both robust and accurate.
The rigid transformation is studied in [
13]. The authors proposed an evolutionary rigid body docking algorithm to register medical images, where the ligands are the sensed images and proteins are references. The aim is to minimize an energy function using genetic algorithms. The experiments have been conducted on images belonging to the Retrospective Image Registration Evaluation project, and the results are encouraging.
To enhance the feature selection in a static image-based facial expressions system, an evolutionary algorithm combined with standard linear discriminant analysis was introduced in [
14]. The proposed method proved to be more efficient than convolutional neural network-based approaches, from the point of view of both training time and features size.
In order to solve the problem of deformable image registration, the Gene-pool Optimal Mixing Evolutionary Algorithm (GOMEA) has been applied by introducing a multi-objective real-valued adaptation [
15]. The authors introduced a real-valued adaptation of GOMEA by using a prescribed dependency model to incrementally improve parts of solutions. The proposed method proved well-suited to register medical images as CT scans and MRI scans, being also vey efficient from the computational complexity point of view.
The problem of non-rigid multi-modal image registration is addressed in [
16]. The authors proposed a novel optimization method, combining the limited memory Broyden–Fletcher–Goldfarb–Shanno with boundaries with cat swarm optimization. The algorithm uses an NMI fitness function to address the problem of the free-form deformations model. Extensive experiments proved that the method is quite accurate and tractable.
In [
17], an improved artificial bee colony algorithm hybrid with a differential evolution for image registration has been proposed. The algorithm uses the simple MI measure to evaluate the candidate solutions. The reported results indicate accurate registration.
To solve the quality inspection for free-form surfaces, a design model-inspection method with range image registration has been proposed in [
18]. A series of 3D discrete points were used to align images. The first stage of the method computes simplified cloud points, using the Hausdorff distance technique with a new point descriptor. Then, a differential algorithm based-optimizer is applied to evaluate the similarity between the designed model and the measurement model in a recursive manner. The experimental results showed that the model is efficient and effective.
Furthermore, numerous approaches for image reconstruction and modeling using metaheuristics, such as the Bacterial Foraging Optimization Algorithm (BFOA) [
19], the Adaptive Invasion–based Mode and distributed Differential Algorithm (AIM-dDE) [
20], the Histogram-based Fruit Fly Optimization Algorithm (HFFO) [
21], Asynchronous Migration and a Mechanism Multi-Population Recombination (AsAMP) [
22], have been reported in recent years. Some of the methodologies are based on single or multiple evolutionary algorithms as a hybrid method, use different codding schemes of solutions (real codding, binary codding, integer codding, etc.), use a parameter or matching approach, address the single or multiple objective optimization, and use different types of images (2D or 3D).
3. 2M-ES Algorithm for Image Alignment and Recognition
The Evolutionary Strategies (ES) class is one of the most popular self-adaptive metaheuristics designed to solve continuous parameter optimization problems [
23]. The basic ES algorithm is 2M-ES, a simple local search procedure developed based on the Gaussian mutation operator.
Given a current candidate solution x, the algorithm computes a new vector by adding to each component of x Gaussian noise with zero mean and standard deviation σ. For each dimension, the step size σ is updated using the Rechenberg rule [
24].
The 2M-ES algorithm can be implemented to align a certain sensed image I to the target T, as follows. Given the rigid model
where
, the aim is to compute a vector
and a function
such that
. The rigid transformation is characterized by the translation vector
, the scale
s and the rotation matrix R.
The genetic representation of a candidate solution is a four-size vector .
For each chromosome c, its fitness value reflects the similarity between and
in terms of the symmetric uncertainty NMI [
25],
where
is the Shannon entropy, and
is the joint entropy.
Obviously, the maximum value of the fitness function is 1 and it is reached when .
Note that, since the search landscape of NMI proved more appropriate for evolutionary computation algorithms than simple MI, symmetric uncertainty is very well-suited to measure the similarity between two images.
The 2M-ES algorithm is described below (Algorithm 1). The inputs are: Max, the upper bound of the number of iterations;
, the initial value of
parameter;
and
, the Rechenberg rule parameters; the threshold
; the pair (I = sensed image, T = target image).
Algorithm 1. 2M-ES Algorithm for Image Registration |
1. Inputs: |
2. |
3. Randomly generate an initial point |
4. ; ; ; |
5. while do |
6. Compute , randomly generated |
7. if |
8. |
9.
|
10. end if |
11. |
12. if |
13. Apply the Rechenberg rule |
14. ; |
15. end if |
16. |
17. end while |
18. Output:
|
4. Image Registration Using the Firefly Algorithm
The Firefly algorithm (FA) is a swarm intelligence algorithm inspired by the flashing patterns of fireflies and the phenomenon of bioluminescent communication, successfully used to solve various parameter-optimization problems [
26,
27,
28]. It was developed based on three rules, briefly described in the following. First, it is assumed that all individuals are unisex, each firefly being attracted to all other fireflies. Secondly, the attractiveness associated to a firefly refers to its brightness. Finally, it is assumed that the attractiveness between fireflies is directly influenced by the brightness and the distance. In terms of metaheuristics, the light intensity is proportional to the value of the objective function, and it represents the fitness value of the considered individual.
The attractiveness of a certain firefly denoted by j seen by another firefly i is expressed by
where
is the current position of j,
is the current position of i,
is the Euclidian distance between
and
,
is the brightness of
at
, and
represents the light absorption coefficient. Note that, in the standard Firefly algorithm,
does not depend on the individual j. We denote by
the attractiveness at the distance 0 [
26].
Each firefly
is attracted by a brighter firefly
, updating its corresponding position as follows,
where
is the attraction to the light intensity,
controls the randomness and
is randomly drawn from
.
In the standard FA, the firefly i changes its position according to (7) if the attractiveness produced by is higher than the attractiveness of the old position . Usually, the termination criterion of the FA is given by the iterations number. In a case where the optimal value of the brightness function is known, the FA search is also over when the best individual is good enough, i.e., its fitness is close to MAXF.
The population diversity in the Firefly algorithm is essentially influenced by the randomization term in (7). In most cases, the parameter is static or decreases linearly in time, each firefly from a certain generation having the same randomness degree. The value of the parameter should rapidly decrease through the first generations, to explore new search space. During the final iterations, in order to maintain the direction of search, the parameter should slightly vary.
In the following, we briefly describe the variant of the Firefly algorithm used for solving the binary image registration task introduced in [
12].
Each firefly is encoded as a D-dimensional real-valued sequence, representing the transformation parameters . In our work , an individual i being defined in terms of its position by corresponding to the degradation model (2). The brightness, , of the firefly, identified by , is computed in terms of (4).
We considered the fixed-size model, where each population has
fireflies,
,
Consequently, the search space is given by
The initial population,
, is randomly generated according to
where
is a draw from uniform distribution
.
We denote by
a constant scale factor, and let
be a number drawn from
. The updating rule introduced in [
12] is defined based on the fitness of the attractor and the variable ranges, as follows,
To deal with unfeasible candidate solutions, the following border reflection mechanism is installed
and
represents a draw from uniform distribution. Taking into account the results reported in [
12], the following proposed approaches that use FA are developed based on the updating rule (12).
5. Two-Stage Hybrid Algorithm for Image Recognition
In order to obtain improved image recognition algorithms, we present a two-stage hybrid algorithm that combines the Firefly-based algorithm described in
Section 3 with the standard 2M-ES local search method. The algorithm first uses a variant of the Firefly algorithm to obtain a promising candidate solution,
best, then applies the local search algorithm, 2M-ES, to reach an optimal solution,
bestc.
Since the fine tune of solutions obtained by the heuristic population-based methods usually implies high execution times, the main purpose is to obtain a tractable recognition method. More efficient methods, incorporating systematic searches of the neighborhood of good solutions, can be derived by adding a local search mechanism [
24].
Our hybrid approach uses the variant of FA described in
Section 3 to compute a promising candidate solution that is going to be improved via 2M-ES. Note that a firefly
i is defined in terms of its position by
The position
corresponds to the rigid degradation model (2). The brightness of the firefly identified by
,
, is computed in terms of (4). The FA component of the hybrid technique is defined by the following parameters:
represents the population size,
represents the maximum number of generations,
represents the light absorption,
is the attractiveness at the distance 0,
represents the desired quality (
), I is the sensed image and T is the target image. The proposed two-stage hybrid technique is provided below (Algorithm 2).
Algorithm 2. Two-Stage Hybrid Firefly Algorithm |
1. Inputs: |
2. APPLY FA |
3. |
4. Randomly generate an initial population |
5. Compute , the light intensity of each |
6. Compute |
7. while |
8. for |
9. for |
10. if |
11. Compute by moving firefly toward firefly (12) |
12. For use the border reflection mechanism (14) |
13. Compute the brightness of , |
14. Replace with in |
15. end if |
16. end for |
17. end for |
18. Compute : |
19. if |
20. |
21. end if |
22.
|
23. end while |
24. APPLY 2M-ES PROCEDURE |
25. Compute |
26. Output: |
6. The Memetic Approaches of Image Registration
The memetic algorithms (MA) are optimization methods in which the evolutionary process is enhanced with deterministic, heuristics or other local search techniques, which reduce the probability of premature convergence. In the memetic approaches, during the evolution process, the information transmission is improved by incorporating local, exact or heuristic methods [
29].
In order to reduce the worst-case run times, one has to use a local search method with an updating rule that differs from population-based search techniques variation operator. From the intuitive point of view, in MA variation operators, mutation in particular, generate points lying in various basins of attractions with respect to the local point operator. The diversification is done either using large mutation rates or, even better, by applying mutation operators with a different neighborhood structure.
The standard MA scheme is described below (Algorithm 3) [
24].
Algorithm 3. Generic Memetic Algorithm |
1. Initialize population |
2. Evaluate each candidate |
3. while Not Termination Condition do |
4. Select parents |
5. Recombine parents to obtain offspring |
6. Mutate offspring |
7. Evaluate offspring |
8. Improve offspring via local search |
9. Select the next generation |
10. end while |
Usually, the initial population can be generated using a mixed variant of randomly drawn individuals and a local search procedure. This way, the initial population contains not only randomly generated individuals, but also some sub-optimal solutions. Further, procedures underlying mass mutation and selective initialization could be applied to initialize a population. [
24,
29,
30].
The proposed MA combines the variants of the Firefly algorithm described in
Section 3 with the 2M-ES local search. Consequently, the neighborhood characteristics of the updating rules depend on the particular local topology, and also on the considered algorithm that operates at a certain moment of the evolution time. The resulting algorithm uses different variation operators, with various neighborhood structures.
We developed the memetic approach, taking into consideration that the number of parents is significantly smaller that the offspring population size. If a variant of the Firefly algorithm with
individuals is used, the number of updated positions of fireflies could be around
. Therefore, due to excessive computational effort, the local search procedure cannot be applied to each child/updated individual. Moreover, in order to avoid the premature convergence and to preserve the diversity of the population, the local search is applied on a small subset of children [
24]. We also used the local search optimization to improve the current best firefly solution, if and only if it is less bright than the global best solution. We applied the 2M-ES search procedure for
offspring randomly selected, where
. In our work we considered
.
The memetic algorithm, based on the proposed variant of FA and the 2M-ES local search, is described as follows (Algorithm 4). The parameters of the resulting algorithm are similar to those used to present Algorithm 2.
Algorithm 4. Memetic Firefly Algorithm |
1. Inputs: |
2. |
3. Randomly generate an initial population |
4. for |
5. Randomly select |
6. Compute : |
7. Replace with in |
8. end for |
9. Evaluate each |
10. Compute |
11. while do |
12. for |
13.
|
14. if |
15. Compute by moving firefly toward firefly using (12) |
16. For use the border reflection mechanism (14) |
17. Evaluate |
18. Replace with in |
19. end if |
20. end for |
21. end for |
22. Compute : |
23. if |
24. for |
25. Randomly select |
26. Compute : |
27. Replace with in |
28. end for |
29. Compute : |
30. if |
31. |
32. end if |
33. end if |
34. |
35. end while |
36. Output: |
7. Experimental Results
In order to experimentally evaluate the accuracy and the efficiency of the proposed algorithms, a series of experiments were performed on different binary images representing signatures. The tests were conducted on simulated data. We used 16 pairs (sensed image, target image), with each observed image representing a certain rigid perturbation of the original one. The proposed registration methods have been applied 500 times for each pair of images, in order to derive significant conclusions. Note that we used the perturbation model (2) with various translations, scale factors and rotation matrices.
Further, we have conducted a comparative analysis to evaluate the performance of the proposed algorithms against the well-known Principal Axes Transformation (PAT) registration method, a standard ES population-based approach [
31] and a variant of the Firefly algorithm reported in [
12].
Obviously, one of the most challenging tasks was to derive tractable algorithms, given the complexity of the fitness function and the fact that the searching space is of continuous type. The chromosome evaluation was performed using only contour pixels, the procedure being outlined in [
31].
We evaluated the accuracy of the evolutionary-based algorithms introduced in this paper through the success rate and the Signal-to-Noise-Ratio (SNR) measure. The success rate of an algorithm A,
, is given by [
31].
where
is the number of algorithm executions and
stands for the number of successful runs. A successful run is defined by a solution whose quality is above a certain threshold value. In our work, the threshold was set to 0.89. For each pair of images (S,T), the SNR value is given by:
where
is the image size.
For each pair of images, we recorded the mean value, the maximum value and the minimum value of the best fitness, the SNR measure and the run time value, respectively. In this study, we express the complexity of a certain algorithm using the mean value of the execution time recorded for each test, no matter if it was a successful run or not. The tests were conducted on the following configuration: Processor—Intel Core I7-7700k 3.6 GHZ, Memory—8 GB DDR4 2400 MHZ, Storage—1TB HDD 7200 RPM SATA 3.
Note that, in order to come to meaningful conclusions concerning the accuracy of the proposed methods, we split the test data into two classes of images, TI1 and TI2. TI1 consists of images correctly recognize by a standard ES algorithm [
31], while the rest of the images belong to TI2.
The results of the standard ES algorithm presented in [
31] are provided in
Table 1 and
Table 2.
In
Table 3 and
Table 4 are displayed the results of the Firefly algorithm version reported in [
12]:
The run times obtained when the Two-Stage Hybrid Firefly algorithm was applied to each pair (sensed image, target image) belonging to TI1 and TI2, respectively, are reported in
Table 5 and
Table 6. Note that
and
. The values of the accuracy measures are presented in
Table 7 and
Table 8.
We used the following parameter settings. The population size in the first stage is . The attractiveness at the distance 0 is , while the light absorption coefficient is . The termination criteria parameters are and . In the second stage, the parameters are as follows: , the initial dispersions vector is , , and .
For each class of images, we summarized the performances of the Two-Stage Firefly 2M-ES Hybrid Algorithm in
Table 9 and
Table 10. Note that the accuracy of this algorithm is experimentally proved to be similar to that corresponding to the Firefly algorithm [
12], and better than the accuracy recorded for the standard ES-based method [
31]. Moreover, the efficiency measured in run times is significantly improved by this method. In the case of the first class of images, TI1, the run times are significantly decreased, from 14.59 s and 15.29 s, respectively, to 5.27 s, while the computation effort carried out by the proposed hybridization algorithm in the case of images belonging to TI2 decreased from 16.71 s and 26.21 s, respectively, to 6.63 s.
We implemented the proposed Memetic Firefly algorithm with the following parameter values. The population size is . The attractiveness at the distance 0 is , while the light absorption coefficient is . The termination criteria parameters are and . The initial population is randomly generated, and of its individuals are improved via 2M-ES. During the evolution process, the local search is used only when the best new individual, represented by the position of an updated firefly, is below the quality of the current best candidate solution. In such a case, we randomly optimize of the new individuals by applying the 2M-ES procedure. The values of parameters in the 2M-ES procedure are set as follows: , , . The mutation size is updated every 40 iterations.
The run times of the Firefly 2M-ES Memetic Algorithm are shown in
Table 11 and
Table 12, while the recorded values of the fitness function and SNR are displayed in
Table 13 and
Table 14. To compare the obtained results against the standard ES-based method [
31] and the Firefly algorithm in [
12], we summarized the runtime and accuracy values in
Table 15 and
Table 16. Note that the accuracy of this algorithm is almost 100% for both classes of images, while the computation effort is significantly decreased as compared to both the standard ES-based method and the considered version of the Firefly algorithm.
The results obtained when the PAT method was applied are less accurate, the mean value of the NMI defined by (4) being below 0.8 in most of the cases. Further, the mean value of SNR is around 23, lower than the values corresponding to the proposed methods.
8. Conclusions
The main aim of the reported work is to solve the registration problem for recognition purposes. We proposed two new methods, a two-stage hybrid algorithm and a memetic technique, respectively consisting of the Firefly population-based search method followed by or combined with a local search scheme.
We investigated the potential of using a version of the Firefly algorithm to conduct the population-based search, while the local exploration was performed using the Two-Membered Evolutionary Strategy. The proposed strategies use the mutual information similarity measure, defined in terms of symmetric uncertainty, to evaluate the quality of the candidate solutions. The fitness function computation scheme is designed to obtain tractable algorithms.
A long series of tests, designed to establish the performances of the proposed methods versus other commonly used classes of techniques for image registration in cases of rigid perturbation model, have been conducted. It was experimentally proved that the proposed methods outperform PAT registration, from both accuracy and efficiency points of view. Our tests also pointed out that the introduced techniques are very accurate, and significantly faster than both the ES-based optimization method reported in [
31] and the version of the Firefly algorithm [
12].
The results indicated that the new methods are excellently fitted to register binary images, their corresponding accuracy being almost 100%. Further, the obtained algorithms are very efficient in terms of the recorded run times.
The experimentally established conclusions are very promising, entailing future research toward more general perturbation models, not necessarily corresponding to registration problems. Moreover, the registration process in cases of more complex images could be designed using similar approaches.