Next Article in Journal
Common-Mode Voltage Reduction Algorithm for Photovoltaic Grid-Connected Inverters with Virtual-Vector Model Predictive Control
Next Article in Special Issue
Service Robots: A Systematic Literature Review
Previous Article in Journal
Rollover Index for Rollover Mitigation Function of Intelligent Commercial Vehicle’s Electronic Stability Control
Previous Article in Special Issue
Enhance the Language Ability of Humanoid Robot NAO through Deep Learning to Interact with Autistic Children
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Cluster-Based Memetic Approach of Image Alignment

Department of Economic Informatics and Cybernetics, Bucharest University of Economic Studies, 10552 București, Romania
*
Author to whom correspondence should be addressed.
Electronics 2021, 10(21), 2606; https://doi.org/10.3390/electronics10212606
Submission received: 23 September 2021 / Revised: 14 October 2021 / Accepted: 23 October 2021 / Published: 25 October 2021

Abstract

:
The paper presents a new memetic, cluster-based methodology for image registration in case of geometric perturbation model involving translation, rotation and scaling. The methodology consists of two stages. First, using the sets of the object pixels belonging to the target image and to the sensed image respectively, the boundaries of the search space are computed. Next, the registration mechanism residing in a hybridization between a version of firefly population-based search procedure and the two membered evolutionary strategy computed on clustered data is applied. In addition, a procedure designed to deal with the premature convergence problem is embedded. The fitness to be maximized by the memetic algorithm is defined by the Dice coefficient, a function implemented to evaluate the similarity between pairs of binary images. The proposed methodology is applied on both binary and monochrome images. In case of monochrome images, a preprocessing step aiming the binarization of the inputs is considered before the registration. The quality of the proposed approach is measured in terms of accuracy and efficiency. The success rate based on Dice coefficient, normalized mutual information measures, and signal-to-noise ratio are used to establish the accuracy of the obtained algorithm, while the efficiency is evaluated by the run time function.

1. Introduction

Often inspiration comes from nature and this extends into the field of computer science. Many algorithms mimic the behavior of biological organisms to solve problems that are difficult or impossible to solve in other way. Such algorithms are increasingly used to approach various problems. They are permanently adapted, modified, combined, developed and seem to have a bright future [1].
With the advancement of technology came an avalanche of images that are used in various sectors. More than often, they are images of the same object but they are not identical, being recorded by different sensors, at different times, angles, luminosity and other variations. These images must be processed in order to be used and the huge number of images makes this a perfect candidate for automation. The process of image registration has raised a lot of attention in the last two decades, reflected in numerous papers published. Due to the wide range of variations, many authors turn to bio-inspired evolutionary algorithms. Advancements are regularly surveyed and reported in scientific publications such as [2,3,4,5,6].
Image registration can be applied to both deformable images and rigid transformations and both types are studied through the use of bio-inspired evolutionary algorithms. In [7], an evolutionary algorithm (EA) with multiple objective optimization is used to find the best way to automate registration of deformable images in the medical field. The results indicate the algorithm is suitable for solving problems with limited deformations and can create better images to be used by experts, also freeing their time by automating image processing.
Registration of images with rigid transformations is advanced in [8] where authors propose a genetic algorithm (GA) that computes best parameters (for translation, rotation and scale) based on matching shapes of molecules. The results have been validated by applying the algorithm for registration of various medical image types: magnetic resonance image (MRI), computed tomography (CT) and positron emission tomography (PET). Images used for testing were obtained from the retrospective image registration evaluation (RIRE) project. The accuracy of the registration is enhanced by using a better fitness function for the genetic algorithm. Authors highlight the fact that the most commonly used similarity function (mutual information of two images) has local optimums which bring the risk of the algorithm becoming stuck in one such point. Improvements are needed in order to ensure the global optimum is found and for this purpose the authors combine the widely-used mutual information function with an interaction energy function.
All kinds of bio-inspired algorithms are used in reported works, from GA [8,9] and evolutionary algorithms (EA) [7] to newer approaches that employ hybridizations and metaheuristics.
Various articles report on the use of swarm intelligence and derivate algorithms for image registration. An in-depth study of particle swarm optimization, with shortcomings and numerous developments and hybridizations is reported in [10]. In [11], authors also review such algorithms and hybridization with evolutionary strategies (ES) for biological and medical image registration, indicating promising results for future developments.
The GA approach is compared to the artificial bee colony (ABC) approach in [9], highlighting the advantages of each algorithm: while GA is faster, ABC gives better quality of image registration. Another comparison between GA and swarm approach, using the correlation function of two images to estimate the quality of registration process, is reported in [12] with the conclusion that the PSO approach provides superior results.
Particle swarm optimization sample consensus (PSOSAC) is used in [13] to optimize registration efficiency. The results are compared against random sample consensus (RANSAC) algorithm and proved to lead to better results.
In [14] authors use an adaptation of coral reef optimization algorithm with substrate layers (CRO-SL) with real numbers for encoding the information. Both feature-based and intensity-based variants for registration are attempted. This approach is compared with others and yields very good results.
Bacterial foraging optimization (BFO) algorithm is applied on image registration in [15,16]. Results are compared with those of other recent algorithms proving to be competitive.
The intensive calculations required for the convergence of bio-inspired algorithms might lead to unfeasible computation time, which is why one of the main concerns is speeding up the algorithms. Use of multiple clusters of data in order to speed up the registration algorithms is an idea presented in several articles. In [17], multiple swarms of ABC are used to this purpose, with very good, reported results regarding the computation time. In [18], authors compare PSO with multi-swarm optimization (MSO) and cuckoo search algorithm (CSA) for image registration. For the dataset, used PSO offers the best precision, while PSO and MSO offer best speed and CSA and MSO offer the least scatter of results. As such, no algorithm prevails on all criteria, but authors mention that these results might be particular to the problem solved and may be different in other cases.
The Firefly paradigm is also used to approach the problem of image registration, with results reported in scientific publications. The many local optimums are a trap for algorithms that use mutual information as fitness indicator for image registration, as mentioned before. In [19], Firefly is used to overcome this problem by combining the use of lower and higher resolution variants of an image and the Powell algorithm. Firefly is used to produce an imprecise result using the lower resolution images, then Powell algorithm is applied on higher resolution images.
A hybrid firefly algorithm (HFA) is used in [20] to solve the problem of slow convergence and for a better coverage of the entire solution space during the search process.
This paper presents a new memetic, cluster-based methodology for image registration. The working assumption is that the sensed images are variants of the targets perturbed by the geometric transformation consisting in rotation, translation and scaling. The proposed approach is applied to align either binary or monochrome images. In both cases, the first step consists in computing the boundaries of the search space based on the object pixels of the processed images. Then, the memetic registration procedure is applied. The alignment of pairs of monochrome images is performed on binarized and resized images. The scaling step is meant to speed up processing of pairs of images and it is used in case of gray scale images only. The quality of the resulted algorithms is measured in terms of accuracy and efficiency. The success rate based on Dice coefficient, the normalized Shannon/Tsallis mutual information measures and signal-to-noise ratio are used to evaluate the accuracy, while the efficiency is established by the run time function. A comparative analysis against two of the most commonly used methods to align images in case of rigid/affine perturbation, namely one plus one evolutionary optimizer [21] and principal axes transform (PAT) [22] experimentally proves the quality of the proposed methodology.
The rest of the paper is organized as follows. The similarity measures used both to define the fitness function and to evaluate the accuracy of the alignment are supplied in Section 2. The proposed methodology is exposed in the core section of the paper. We describe the accuracy and efficiency indices in Section 4. A series of experimental results and the comparative analysis concerning the accuracy and the efficiency of the resulted algorithms are presented next. The final part of the paper includes conclusions and suggestions for further developments regarding bio-inspired methods for image registration.

2. Similarity Measures

Let X and Y be two binary sets. The Dice coefficient measures the similarities between X and Y by:
Dice ( X , Y ) = 2 | X   Y | | X | + | Y |
where | X | stands for the cardinal of X . Obviously, max X , Y   binary   sets Dice ( X ,   Y ) = 1 and Dice ( X ,   Y ) = 1 if and only if X = Y . The Dice coefficient can be directly applied to pairs of binary images.
In general cases of monochrome and colored images, more complex functions should be considered instead, one of the most commonly used being the normalized mutual information computed using entropic measures.
Let X and Y be monochrome images with distributions p ( x ) and p ( y ) , respectively. Note that p ( x ) is the probability of intensity x appearing in image X . We denote by p ( x ,   y ) the joint probability, that is the probability that corresponding pixels in X and Y have intensity x and y , respectively. The joint probability distribution of the images X and Y reflects the relationship between intensities in X and Y . Assuming that L is the number of grey levels of the images, the Shannon entropy of X is defined by:
H S ( X ) = x = 0 L 1 p ( x ) log 2 p ( x )
The joint Shannon entropy is given by:
H S ( X , Y ) = x = 0 L 1 y = 0 L 1 p ( x , y ) log 2 p ( x ,   y )
Shannon normalized mutual information is defined by [23]:
NMI S ( X , Y ) = 2 MI S ( X , Y ) H S ( X ) + H S ( Y )
MI S ( X , Y ) = H S ( X ) + H S ( Y ) - H S ( X , Y )
where MI S ( X ,   Y ) is the Shannon mutual information. The maximum value of Shannon normalized mutual information is one and it is reached when X = Y .
Shannon mutual information is widely used in image registration, but it is sensitive to noise. To reduce the influence of outliers, one can use similarity measures based on Tsallis entropy instead [22,24].
Tsallis entropy of order α is defined by [24]
H α T ( X ) = 1 α 1 ( 1 x = 0 L 1 p ( x ) α )
The joint Tsallis entropy of order α is given by
H α T ( X , Y ) = 1 α 1 ( 1 x = 0 L 1 y = 0 L 1 p ( x , y ) α )
Note that when α approaches to 1, Tsallis entropy approaches Shannon entropy.
For α > 1, Tsallis mutual information is expressed as:
MI α T ( X , Y ) = H α T ( X ) + H α T ( Y ) H α T ( X , Y )
and Tsallis normalized mutual information
NMI α T ( X , Y ) = MI α T ( X , Y ) H α T ( X , Y )
For α > 1, the following properties hold [25]
  • NMI α T ( X ,   Y ) [ 0 , 1 ]
  • NMI α T ( X ,   Y ) = 0 if X and Y are independent
  • NMI α T ( X ,   Y ) = 1 if X = Y .
In our work the fitness function is defined in terms of Dice coefficient, while NMI S and NMI α T , α > 1, are used to evaluate the accuracy of the registration procedure.

3. The Proposed Methodology for Binary Image Alignment

The proposed methodology used to align two binary images is developed to deal with perturbations involving rotation, translation and scaling. Note that in image registration literature, a rigid transformation involves either rotation and translation [4,22], or rotation, translation and scale changes [26]. The first case corresponds to rigid geometric transformations which preserve distances and it is given by three parameters, the translation [a, b] and the rotation angle θ . In the second approach, objects in the images retain their relative shape and position, rigid transformation being defined by four parameters, the translation [a, b], the rotation angle θ and the scale factor s. From the geometrical point of view, the transformation corresponds to a similarity with stretching factor s. In our work, we used the second version of rigid transformations.
Let T be the target image of size M × N . The sensed image S results as a geometric transformation of T, defined in terms of rotation, translation and scale changes. Let θ be the rotation angle defining the rotation matrix R = [ cos θ sin θ sin θ cos θ ] , s the scale factor and [ a , b ] T the translation vector. For each pixel ( x ,   y ) ,   1 x M ,   1 y N , the output S ( x ,   y ) is given by:
S ( x ,   y ) = T ( f ( x ,   y ) )
where
f ( x ,   y ) = [ a b ] + s     R [ x y ]
The computation of the transformation parameters can be carried out by an evolutionary algorithm, where the boundaries of the search space are established using the sets of object pixels belonging to the target image and to the sensed image respectively.
The components of our methodology are described below.

3.1. The Search Space Boundaries

The first stage of our method consists in defining the search space. Based on the assumption that the deformation is reversible, that is each object in T corresponds to a certain object in S, namely its perturbed version, the computation of the search space boundaries is performed taking into account only the object pixels.
We consider that the initial image was rotated to the left, that is θ [ π ,   0 ] and the scale factor s ( 0 , smax ] . Obviously, similar results can be obtained in case of right-side rotations. We denote by { ( x S 1 , y S 1 ) ,   ,   ( x S p , y S p ) } the set of object pixels belonging to S, let { ( x T 1 , y T 1 ) , , ( x T p , y T p ) } be the set of object pixels of T and
minxS = min i = 1 , , p x S i ,   minyS = min i = 1 , , p y S i ,   maxxS = max i = 1 , , p x S i ,   maxyS = max i = 1 , , p y S i
minxT = min i = 1 , , p x T i ,   minyT = min i = 1 , , p y T i ,   maxxT = max i = 1 , , p x T i ,   maxyT = max i = 1 , , p y T i
maxS = max { maxxS ,   maxyS }
Using (11) we obtain
minxT     a + s     cos θ x s sin θ y maxxT
Using straightforward computation, since sin θ 0 , we obtain
minxT     a + s ( | cos θ | x + | sin θ | y ) a + smax 2 maxS
and
maxx   T     a     s     | cos θ | x + s | sin θ | y a smax maxxS
We obtain the following definition domain of parameter a
Da = [ minxT smax 2 maxS , maxxT + smax maxxS ]
In the same way, since
minyT     b + s   sin θ x + s cos θ y = b     s     | sin θ | x + s cos θ y maxyT
we obtain
minyT     b     s     | sin θ | x + s | cos θ | y b + smax     maxyS
maxyT     b     s     | sin θ | x s | cos θ | y = b s ( | cos θ | x + | sin θ | y ) b smax 2 maxS
and consequently, the definition domain of b is given by
Db = [ minyT smax     maxyS , maxyT + smax 2 maxS ]
The proposed image registration method aims to compute the parameter ( a , b , θ , s ) such that the relations (10) and (11) hold, where:
( a ,   b ,   θ ,   s )     D ( S , T ) = Da × Db × [ π ,   0 ] × ( 0 ,   smax ]

3.2. Metaheuristics for Image Registration

The binary image registration procedure can be developed using evolutionary approaches. The proposed methodology uses a special tailored version of Firefly algorithm and standard two membered evolutionary strategy (2MES) to compute a solution of (10). In this section, we briefly describe the versions of Firefly algorithm and 2MES specially tailored to binary image registration [27,28].
From the evolutionary algorithms point of view, solving the problem (10) involves defining a search space and a fitness function, and applying an iterative procedure to compute an individual that maximizes the fitness. In our approach, the search space is defined by (19) and, for each candidate solution c = ( ca , cb , c θ , cs ) , the fitness function measures the similarity between the target image T and the image T ˜ ,
T ˜ ( x ,   y ) = S ( g c ( x ,   y ) )
g c ( x ,   y ) = 1 cs cR T ( [ x y ] [ ca cb ] )
fitness ( c ) = Similarity ( T ˜ , T )
where cR = [ cos c θ sin c θ sin c θ cos c θ ] .
Evolutionary Strategies (ES) are self-adaptive methods for continuous parameter optimization. The simplest algorithm belonging to ES class is 2MES, a local search procedure that computes a sequence of candidate solutions based on Gaussian mutation with adaptive step size. Briefly, the search starts with a randomly generated/input vector c 0 , an initial step size σ 0 and the values ϑ [ 0.817 , 1 ) and τ implementing the self-adaptive Rechenberg rule [29]. At each iteration t, the algorithms computes:
c t = { c t 1 + z   ,   if   fitness ( c t - 1 + z ) > fitness ( c t 1 ) c t 1 ,                                                                                                 otherwise  
where z is randomly generated from the distribution N ( 0 , σ t 1 ) . The dispersion is updated every τ steps according to Rechenberg rule:
σ t = { σ t 1 ϑ ,   p τ > 0.2 σ t 1 ϑ ,   p τ < 0.2 σ t 1 ,   p τ = 0.2  
where p is the number of distinct vectors computed by the last τ updates. The search is over either when the fitness if good enough, i.e., the maximum value exceeds a threshold υ or when a maximum number of iterations MAX has been reached. Let us denote by 2MES(x,   σ 0 ,   ϑ ,   τ ,   υ , MAX, S, T) the 2MES procedure with the initial input vector x = c 0 . The procedure computes the improved version of x, x final , using the termination condition defined by the parameters υ and MAX, respectively.
Note that 2MES algorithm usually computes local optima and it is used to locally improve candidate solutions computed by global search procedures in hybrid or memetic approaches.
Firefly algorithm (FA) is a nature inspired optimization procedure, introduced in [30]. FA belongs to the class of swarm intelligence methods and it mimics the behavior of fireflies and their bioluminescent communication. The ideas underlying FA are that each firefly is attracted by the flashes emitted by all other fireflies, the attractiveness of an individual is linked to the brightness of its flashes, and influenced by the light absorption and the law of light variations with distance.
In terms of image registration problem (10), the position of a firefly i corresponds to a candidate solution c i = ( ca i , cb i , c θ i , cs i ) , its light intensity being given by fitness ( c i ) . For each pair of fireflies i and j, if j is brighter than i, that is fitness ( c j t ) > fitness ( c i t ) , then i is attracted by j, its position being updated based on the following equation:
c i ( t + 1 ) = c i ( t ) + β j ( r )   ( c j ( t ) - c i ( t ) ) + α     ε
where α controls the randomness, ε is randomly drawn from U ( 0 ,   1 ) , β j ( r ) is the attractiveness of j seen by i defined by:
β j ( r ) = β 0     e - γ r 2
where r = | | x j   x i | | is the Euclidian distance between i and j. The constant β 0 is the brightness of any firefly at r = 0 and γ represents the light absorption coefficient.
Usually, the update rule (25) is applied if the attractiveness of i in the new location c i ( t + 1 ) is higher than the attractiveness corresponding to the old position c i ( t ) . The termination criterion of the FA is formulated in terms of number of iterations. Obviously, if the maximum value of the brightness function is known, the FA ends when the current best individual is good enough.
The image registration problem (10) has been solved using the fixed-size model of FA, where the population at time t, t 0 , has n individuals, X t = { c 1 t ,   c 2 t ,   ,   c n t } , c i t   = ( c i t ( 1 ) ,   ,   c i t ( 4 ) ) and c i t ( k )     [ l ( k ) ,   h ( k ) ] ,   k = 1 , , 4 [27]. Note that the search space k [ l ( k ) ,   h ( k ) ] used in the cited work is predefined, it does not change or adapt in any way depending on the image properties.
The initial population, X 0 , is randomly generated according to the uniform probability distribution U ( l ( k ) ,   h ( k ) ) . We denote by cf a constant scale factor and let U ( a ,   b ) be a draw from the uniform distribution on the interval [a, b]. The update rule introduced in [27] is given by:
c i t + 1 ( k ) = c i t ( k ) + β ij ( k )   ( c j t ( k ) - c i t ( k ) ) + h ( k ) - l ( k ) max k ( h ( k ) - l ( k ) )   cf     exp ( 1 -   fitness ( c j t ) )     U ( 0 , 1 )
In addition, a border reflection rule has been proposed to deal with unfeasibility:
if   c i t + 1 ( k )   >   h ( k ) then   c i t + 1 ( k ) = U ( p l ( k ) + ( 1 - p )     h ( k ) ,   b ( k ) ) else if   c i t + 1 ( k )   <   l ( k ) then   c i ( k ) = U ( l ( k ) ,   p l ( k ) + ( 1 - p )   h ( k ) )
where p     ( 0 , 1 ) and U ( a ,   b ) represents a draw from uniform distribution on [a, b].
Note that in Attractiveness Formula (27) the quality of the attractor affects the randomness parameter. In case of high luminous intensity individual j , less randomness value is added. If the flashes emitted by the firefly j are weak then the perturbation grows.

3.3. Cluster-Based Memetic Registration

The proposed methodology is based on a core cluster-based memetic algorithm developed to register pairs of binary images. The global search procedure is directed by the variant of FA described in Section 3.2, where the positions of fireflies belong to the domain defined by (19). The fitness function defined by (22) implements Dice coefficient. Note that the maximum value of the fitness function is one.
The initial population is randomly generated and a small number of individuals are locally improved (a fixed percentage of population size). The local optimization is implemented by 2MES method provided in Section 3.2. We denote by nr the number of individuals to be initially processed by 2MES.
The population-based optimization is an iterative process, at each iteration t the algorithm selecting pairs of distinct individuals ( c i t ,   c j t ) and, if fitness ( c j t ) > fitness ( c i t ) , applies the update rule (27). If the brightness of the best individual in X t + 1 does not exceed the highest fitness value in X t , then the local optimization procedure 2MES is used. The individuals selected for further improvements are computed based on clustered data. The candidate solutions in X t + 1 are grouped in k clusters using the Euclidian distance metric, one individual per cluster being chosen to be locally improved. The selection can be random or deterministic, for instance, one may choose either the centroid or the best candidate solution of each cluster.
In addition, the proposed hybridization between the population-based search and the local optimization procedure is designed to reduce the risk of premature convergence. Basically, two mechanisms are developed to deal with the situation of premature convergence. On one hand, at each iteration t, the number of clusters is set inverse proportional to the fitness value of the best individual in X t + 1 , denoted by fitness   ( best ) . Let k 0 be the initial number of clusters, set as a small percentage of population size. We propose the computation rule:
k = k 0 [ 1 fitness ( best ) ]
Moreover, the initial step size of 2MES procedure increases in case its consecutive iterations do not lead to quality improvement. The proposed update rule is given by:
σ = σ 0 fitness ( best )
Note that fitness ( best ) 1 .
On the other hand, if the fitness is not improved over it2, it2 > it1, consecutive iterations, some new individuals are created to replace a set of randomly selected old ones. The newly created individuals are randomly generated using the uniform probability distribution, each one of them being improved next by 2MES procedure. We denote by NEW( X t + 1 , ind) the procedure that refreshes X t + 1 by adding individuals, as we explained above.
The search is over after NMAX iterations or when the best computed fitness value is above a threshold τ stop .
The detailed description of the proposed algorithm is provided below. We denote by S and T the sensed image and the target image, respectively. The parameters σ 0 ,   ϑ ,   τ ES ,   υ and MAX correspond to the 2MES procedure applied to improve the initial population and we denote by σ 0 , ϑ , τ ES , υ , MAX the parameters of the local optimizer applied on clustered data. The parameters β 0 and γ are specific to FA, according to Section 3.2 and let us denote by X t = { c 1 t ,   c 2 t ,   ,   c n t } the current population at the tth iteration. The variable counter counts the number of consecutive populations having the same best fitness value.
Algorithm 1 Cluster-based memetic algorithm
  • Inputs:  n ,   NMAX ,   τ stop ,   nr ,   k 0 ,   ind ,   β 0 ,   γ ,   cf ,   σ 0 ,   ϑ ,   τ ES ,
                  υ ,   MAX ,   σ 0 ,   ϑ ,   τ ES ,   υ ,   MAX ,   S ,   T
  • Compute D ( S , T ) according to (19)
  • t = 0 ; counter = 0
  • Compute X t , the initial population according to Algorithm 2
  • Evaluate the individuals in X t and compute best :   fitness ( best ) = max x X t   fitness ( x )
  • while  t   <   NMax and fitness ( best )   <   τ stop  do
  •        Apply an FA iteration, according to Algorithm 3
  •        Compute best c : fitness ( best c ) = max x X t + 1   fitness ( x )
  •        if  fitness ( best c )     fitness ( best )
  •               if counter == it1
  •                       Update σ 0 according to (30)
  •               end if
  •               if counter == it2 NEW( X t + 1 , ind)
  •               end if
  •               compute k according to (29)
  •               apply k-means to X t + 1 and obtain the clusters C 1 , , C k
  •               for i = 1 k
  •                       Select x     C i   –randomly/the centroid/the best candidate solution
  •                       x new = 2 MES ( x ,   σ 0 ,   ϑ ,   τ ES ,   υ ,   MAX ,   S ,   T )
  •                       Replace x by x new in X t + 1
  •               end for
  •               Compute best c : fitness ( best c ) =   max x X t + 1   fitness ( x )  
  •               if   fitness ( best c )   >   fitness ( best )
  •                       best = best c ; counter = 0
  •               else if fitness ( best c )   <   fitness ( best )
  •                             Randomly select x     X t + 1
  •                             replace x by best in X t + 1  
  •                       else counter = counter + 1
  •                       end if
  •               end if
  •        end if
  •        t = t + 1  
  • end while
  • Output: b e s t , T ˜ = S ( g best ) according to (21)
Algorithm 2 Computation of the initial population
  • Inputs:  n ,   σ 0 ,   ϑ ,   τ ES ,   υ ,   MAX ,   S ,   T
  • Randomly generate an initial population X 0 = { c 1 0 ,   c 2 0 ,   ,   c n 0 }  
  • for   i = 1 nr
  • Randomly select x     X t
  • x new = 2 MES ( x ,   σ 0 ,   ϑ ,   τ ES ,   υ ,   MAX ,   S ,   T )
  • Replace x by x new in X 0
  • end for
  • Output:   X 0
Algorithm 3 FA iteration
  • Inputs:  n ,   X t ,   β 0 ,   γ ,   cf ,   S ,   T  
  • for   i = 1 n  
  •         for   j = 1 n
  •                 if fitness ( c j t ) > fitness ( c i t )
  •                         Compute c i t + 1 by moving firefly i toward firefly j using (27)
  •                         Use the border reflection mechanism (28) to adjust c i t + 1
  •                         Evaluate c i t + 1
  •                 end if
  •         end for
  • end for
  • Output:   X t + 1

3.4. Monochrome Image Registration

The method described by Algorithm 1 can also be applied, after a preprocessing stage, when monochrome images should be registered. Obviously, the main idea is to binarize the images by representing them using only the boundaries of their objects. However, depending on the complexity and quality of the analyzed images, further specific image processing techniques may be needed, as for instance image enhancement, de-blurring and noise removal. Alternatively, one can use edge detectors insensitive to noise and variations in illumination. Examples of such filters are reported in [31,32,33].
In the following we assume that the input images have already been processed such that a contour detection mechanism can be applied.
Let S and T be the M × N sized sensed and target image respectively and we assume that the rigid transformation is given by the parameters ( a ,   b ,   θ ,   s ) . The proposed registration procedure consists of the following steps. First, a contraction mechanism, for example a scale transformation with supra-unitary factor, is applied. If we denote by s c the contraction factor, the images S and T are transformed according to:
S p ( x ,   y ) = S ( h ( x ,   y ) )
T p ( x ,   y ) = T ( h ( x ,   y ) )
where 1 x M ,   1 y N and
h ( x ,   y ) = s c [ x y ]
The main aim of the transforms (31) and (32) is to reduce the size of objects in the processed images and hence the complexity of search.
The next step is to represent S p and T p using only the contour of the objects belonging to the images. We denote by OS p and OT p the results of applying an edge detector to S p and T p , respectively. Obviously, OS p is a perturbed version of T p . We denote by ( ba , bb , b θ , bs ) the parameters of the corresponding perturbation. Using straightforward computation, we obtain:
a = s c ba ,   b = s c bb ,   θ = b θ , s = bs
Consequently, to compute the parameters ( a ,   b ,   θ ,   s ) we first apply the Algorithm 1 to obtain an approximation of ( ba , bb , b θ , bs ) , and then use Equation (34).

3.5. Monochrome Image Registration in Case of Scaling on Multiple Dimensions

The proposed methodology can be extended to the case of more general perturbation models, in which each dimension is scaled with a specific stretching factor. The transformation is given by:
S ( x ,   y ) = T ( f ( x ,   y ) )
where
f ( x ,   y ) = [ a b ] + [ s x 0 0 s y ]   R [ x y ]
for each ( x ,   y ) ,   1 x M ,   1 y N . The scale matrix s = [ s x 0 0 s y ] is such that s x ,   s y > 0 .
The search space boundaries can be computed similarly to (10) and (11). If s x ( 0 , smax x ] and s y ( 0 , smax y ] , by straightforward computation we obtain,
Da = [ minxT smax x 2 maxS , maxxT + smax x maxxS ]
Db = [ minyT smax y   maxyS , maxyT + smax y 2 maxS ]
Consequently, the proposed image alignment method aims to compute the parameters ( a , b , θ , s x , s y ) such that the relations (37) and (38) hold, where
( a , b , θ , s x , s y )     D ( S , T ) = Da × Db × [ π ,   0 ] × ( 0 , smax x ] × ( 0 , smax y ]
For each candidate solution c = ( ca , cb , c θ , cs x , cs y ) , the fitness function measures the similarity between the target image T and the image T ˜ ,
T ˜ ( x ,   y ) = S ( g c ( x ,   y ) )
g c ( x ,   y ) = cR T [ 1 cs x 0 0 1 cs y ] ( [ x y ] [ ca cb ] )
fitness ( c ) = Similarity ( T ˜ , T )
where cR = [ cos c θ sin c θ sin c θ cos c θ ] .
The methodology described in Section 3.4 can be applied to align images perturbed by (38) using the fitness function defined by (44).

4. Efficiency Measures

Let S be the sensed image and we denote by T the target. The images have the same size, M × N . The accuracy of the registration method is measured through the success rate and using the similarity between T and the result of applying the alignment process on S, denoted by T ˜ . We evaluate the similarity between T and T ˜ using two metrics commonly used in image processing, signal-to-noise-ratio (SNR) and peak-signal-to-noise ratio (PSNR), and two entropic measures, Shannon normalized mutual information defined by (4) and Tsallis normalized mutual information given by (9). The values SNR ( T , T ˜ ) and PSNR ( T , T ˜ ) are given by
SNR ( T , T ˜ ) = 10     log 10 [ x = 1 M y = 1 N ( T ( x , y ) ) 2 x = 1 M y = 1 N ( T ( x , y ) T ˜ ( x , y ) ) 2 ]
PSNR ( T , T ˜ ) = 10     log 10 [ max ( T ( x , y ) ) 2 1 M N x = 1 M y = 1 N ( T ( x , y ) T ˜ ( x , y ) ) 2 ]
Note that the fitness function used to align pairs of images is also a similarity measure computed between binary images, Dice coefficient (1).
Due to the fact that the proposed methodology is of stochastic type, one way to evaluate its effectiveness is to compute the success rate, that is the percentage of runs that led to the correct registration. From technical point of view, the result of applying Algorithm 1 to the pair (S, T) is correct if the obtained image T ˜ corresponds to an individual best such that fitness ( best ) = Dice ( T , T ˜ )     τ stop . Consequently, the success rate of the algorithm is given by:
SR ( T , S ) = NS NR 100 %
where NS is the number of successful runs and NR is the total number of algorithm executions.
To evaluate the accuracy of the proposed method, we compute the mean values of the above-mentioned similarities. If we denote by T ˜ 1 ,   , T ˜ NR the registered versions computed by the algorithm and SIM { DICE , NMI S , NMI α T , SNR , PSNR } , we obtain
MeanSIM ( T , S ) = i = 1 NR SIM ( T , T ˜ i ) NR
The efficiency of the proposed method is experimentally evaluated by the runtime function. If we denote by t 1 ,   ,   t NR the execution times consumed by Algorithm 1 to register the pair (S, T), the mean runtime value is computed by
MeanRT ( T , S ) = i = 1 NR t i NR
From theoretical point of view, computational complexity may be evaluated in several ways. In our analysis we use the size of population (n) and maximum number of iterations corresponding to FA (NMAX) and 2MES (MAX, MAX, where MAX and MAX have the same magnitude) to estimate the worst-case scenario. Since 2MES performs at most MAX iterations, the initial population generation algorithm (Algorithm 2) has a complexity of O(n*MAX). The complexity of Algorithm 1 is influenced by the k-means clustering algorithm. Since in our method the number of clusters linearly depends on the population size and the complexity of k-means is O(k*n*chromosome size), the resulting complexity of the clustering algorithm is O(n2). The 2MES performs at most, MAX iterations and is called inside a loop depending on k, which means the complexity of this loop is O(n*MAX). An iteration of FA algorithm (Algorithm 3) has a complexity of O(n2). Consequently, the complexity of the proposed method is O(NMAX*n*MAX). Note that in most cases n << MAX and NMAX and MAX have a similar magnitude, which leads to a quadratic complexity depending on the number of iterations.

5. Experimental Results and Discussion

To derive conclusions regarding the quality of the proposed approach a long series of test have been conducted on both binary and monochrome images. The results were obtained using the following configuration: processor Intel Core i7-10870H up to 5.0 GHz, 16 GB RAM DDR4, SSD 512 GB, NVIDIA GeForce GTX 1650Ti 4 GB GDDR6.

5.1. Binary Image Registration

Our tests have been conducted on a set of 16 binary images representing signatures, all having the same size 192 × 192 pixels. The images, denoted by S 1 ,   ,   S 16 , are perturbed by the rigid transformation (10) and (11) with various perturbation parameters. The rotation angle is between π and 0, while the scale factor was set in [ 0.5 ,   1.5 ] . The translation parameters are a [ 40 ,   10 ] and b [ 40 ,   60 ] . The rigid transformation parameters correspond to the working assumption that the perturbation process is totally reversible, that is the object pixels are completely encoded in the sensed images.
The search space is computed using (19). Note that the intervals D a and D b are significantly larger than [ 35 ,   10 ] and [ 40 ,   60 ] . For instance, in case of S 1 , D a = [ 402 ,   411 ] and D b = [ 258 ,   579 ] , while a = 36 and b = 46 .
Since the perturbation process is totally reversible, the fitness threshold τ stop is set close to the maximum value, one. In our test τ stop = 0.9 . The rest of the input parameters are set as follows: n = 20 , NMAX = 200 , nr = 6 , k 0 = 4 , ind = 4 , β 0 = γ = 1 , cf = 2, σ 0 = [ 7 ,   7 ,   0.3 ,   0.3 ] , ϑ = ϑ = 0.85 , τ ES = 20 , υ = 0.5 , MAX = 800 , σ 0 = [ 3 ,   3 ,   0.02 ,   0.02 ] , τ ES = 10 and MAX = 200 .
The experimentally established results regarding the accuracy and the efficiency of Algorithm 1 are provided in Table 1. Note that the success rate is 100% for all pairs of images, NR = 700 and the SNR values are computed for images having the gray levels in { 0 ,   1 } . The computation is over when the maximum fitness value is at least 0.9.

5.2. Monochrome Image Registration

In case of more complex, monochrome images, the assumption that the perturbation process is completely reversible is rather unrealistic. From the technical point of view, it means that the search procedure cannot manage to compute an individual with fitness 1, that is even when the rigid transformation parameters are correctly determined. Obviously, in such cases the threshold τ stop should be set on lower values and the evaluation of accuracy should take into account the similarity between the aligned version of S and the initial image with the missing parts instead of the target T.
Our tests were conducted on images belonging to the well-known Yale Face Database [34,35], which contains 165 greyscale images of 15 individuals, 11 images per subject/class. The results reported in Table 2, Table 3 and Table 4 refer to 30 images, two for each person, while the images displayed in Figure 1, Figure 2, Figure 3, Figure 4, Figure 5, Figure 6, Figure 7, Figure 8, Figure 9 and Figure 10 correspond to two classes.
First, we provide a report regarding the new method proposed in Section 3.4. Examples of target images, sensed ones and images obtained when the correct inverse rigid transformation is applied are presented in Figure 1 (Subject 5, the first image, corresponding to Row 5 in Table 2, Table 3 and Table 4) and Figure 2 respectively (Subject 14, the second image, corresponding to Row 29 in Table 2, Table 3 and Table 4).
The perturbation parameters are set as follows: the rotation angle is between π and 0, the scale factor was set in [ 0.5 ,   1.5 ] , and the translation parameters are a [ 130 ,   250 ] and b [ 35 ,   420 ] . For instance, the target image displayed in Figure 1 suffered a perturbation with the parameter vector [ 40 ,   70 , π 8 ,   0.9 ] , while the one provided in Figure 2 was perturbed by the rigid transformation with [ 230 ,   370 , π 1.1 ,   1.45 ] .
Scaled and binarized versions of the pairs (S, T) are computed by the preprocessing step. The corresponding versions of the images displayed in Figure 1 and Figure 2 are depicted in Figure 3 and Figure 4. The scaling factor used for this step is 2.
Next, the Algorithm 1 is applied, where the inputs are the images computed by the preprocessing step. The search space is computed using (19). In our test τ stop [ 0.5 ,   0.6 ] . The rest of the input parameters are set as follows: n = 20 , NMAX = 250 , nr = 6 , k 0 = 4 , ind = 4 , β 0 = γ = 1 ,   cf = 2 , σ 0 = [ 12 ,   12 ,   0.3 ,   0.5 ] , ϑ = ϑ = 0.85 , τ ES = 20 , υ = 0.3 , MAX = 800 , σ 0 = [ 7 ,   7 ,   0.03 ,   0.05 ] , τ ES = 15 and MAX = 240 .
The results obtained by the proposed registration methodology are depicted in Figure 5 and Figure 6.
The accuracy and efficiency analysis are provided in Table 2. Note that the results concerning the similarity measures SIM { DICE , NMI S , NMI α T , SNR , PSNR } are computed by
RSIM ( T ˜ ) = MeanSIM ( T , T ) MeanSIM ( T ˜ , T )
where T is the target, T ˜ is the restored version computed by the proposed methodology, and T’ is the version obtained when the correct inverse rigid transformation is applied. In our tests, α = 1.3 Obviously, the ideal value of the ratios defined by (50) is 1. However, possible larger values may be obtained, due to calculation and rounding errors.
The success rate of the proposed method is 100% for all the tested images,   NR = 200 and the SNR values are computed for images having the gray levels in { 0 ,   ,   255 } .
In order to analyze the registration capabilities of the proposed method, we experimentally compared it against two of the most commonly used align procedures in case of rigid transformation, namely one plus one evolutionary optimizer (EO) [21] and principal axes transform (PAT) [22]. Note that the function EO was tested with 100 different parameter settings per pair of images to establish the best alignment from the similarity ratio point of view (48), where SIM = NMI S . The registered images using PAT method are displayed in Figure 7 and Figure 8, while the results produced by EO are depicted in Figure 9 and Figure 10.
The numerical results are reported in Table 3 and Table 4.
Note that PAT image alignment method has a widely known problem that in some cases produces results rotated 180 degrees along principal axes. In practice, this leads to some results being rotated upside-down. PAT stops at computing the aligned image and does not go further into analyzing if it is rotated or not, from a visual point of view. Some research [36] aims to correct such results by automatically assessing which of the two possible rotations represents the correct image. In case of images rotated to the left with large angles, PAT and EO may fail to provide the correct alignment. In such cases, the ratios values are significantly smaller than one. In case of PAT registration, the run time values vary between 4 and 6 s, while EO method consumes significantly more time due to the need to establish the appropriate input parameters.
The results experimentally derived from a long series of tests lead to the conclusion that the proposed method outperforms PAT and EO from both accuracy points of view, informational and quantitative.

5.3. Monochrome Image Registration in Case of Scaling on Multiple Dimensions

The methodology introduced in Section 3.5 has been applied on images belonging to Yale Face Database perturbed according to (36). The results are as follows.
Examples of target images, sensed ones and images obtained when the correct inverse rigid transformation is applied are presented in Figure 11 (Subject 4, corresponding to Row 4 in Table 5, Table 6 and Table 7) and Figure 12 respectively (Subject 10, corresponding to Row 10 in Table 5, Table 6 and Table 7).
The perturbation parameters are set as follows: the rotation angle is between π and 0, the scale factors were set in [ 0.5 ,   1.5 ] , and the translation parameters are a [ 190 ,   245 ] and b [ 35 ,   400 ] . The target image displayed in Figure 11 suffered a perturbation with the parameter vector [ 50 ,   130 , π 5 ,   1 , 1.4 ] , while the one provided in Figure 12 was perturbed by the rigid transformation with [ 40 ,   220 , π 3 ,   1.37 , 1.1 ] .
The corresponding versions of scaled and binarized images displayed in Figure 11 and Figure 12 are depicted in Figure 13 and Figure 14. The scaling factor used for this step is two.
Next, the Algorithm 1 is applied, the input parameters being the same as those reported in Section 5.2. The results obtained by the proposed registration methodology are depicted in Figure 15 and Figure 16.
The accuracy and efficiency analysis are provided in Table 5. The results concerning the similarity measures SIM { DICE , NMI S , NMI α T , SNR , PSNR } are computed by (40). The success rate of the proposed method is 100% for all the tested images,   NR = 50 and the SNR values are computed for images having the gray levels in { 0 ,   ,   255 } .
Note that PAT and EO algorithms lead to similar results as those reported in Section 5.2. (Figure 17, Figure 18, Figure 19 and Figure 20). They fail to correctly perform the alignment from the rotation point of view, when the rotation angle is close to −180°. PAT may also misalign images in case of large magnitude distortions, mainly because the computation of the PAT solution involves arbitrary unitary matrices. Moreover, there are images with the same principal directions set and PAT cannot distinguish between them [37].
The numerical results are reported in Table 6 and Table 7.

6. Conclusions

The work reported in this paper focuses on developing a novel and comprehensive methodology for image registration. The primary algorithm introduced in Section 3.3 addresses the problem of binary images, while its extended version deals with more complex, monochrome images. Note that the proposed approach could be further extended to colored images, the problem of representing the objects by their contour being solved either directly, using contour following algorithms or edge detectors, or after an additional procedure that computes the grey level versions of the inputs.
The main advantages of the proposed methodology are the comprehensiveness, due to the way we compute the search space, and the effectiveness, mainly due to the properties of the memetic firefly algorithm—ES approach, the embedded clustering procedure and the two mechanisms implemented to alleviate the risk of premature convergence.
The experimental results were derived based on very large number of tests and using various accuracy and efficiency measures. Both information-based similarity functions and quantitative measures, as for instance SNR and PSNR, were used to evaluate the effectiveness of our method and to compare it against two of the most commonly used align procedures in case of rigid transformation, EO algorithm and PAT method.
In case of binary images, the success rate is 100%, i.e., the target images are always identified by applying the inverse rigid transform on their corresponding sensed images, where the parameter vectors are computed by Algorithm 1. The recorded runtime values proved that the method is also efficient, especially being given its stochastic properties. The general methodology dealing with monochrome images also proved effective and efficient. Moreover, unlike PAT registration or EO, the proposed approach manages to correctly reverse the perturbation for all tested pair of images.
We conclude that the results are encouraging and entail future work toward extending this approach to more complex perturbation models as well as more advanced bio-inspired optimizations and evolutionary algorithms.

Author Contributions

Conceptualization, C.-L.C. and C.R.U.; formal analysis, C.-L.C. and C.R.U.; methodology, C.-L.C.; software, C.-L.C. and C.R.U.; supervision, C.-L.C.; validation, C.-L.C. and C.R.U.; writing—original draft, C.-L.C.; writing—review and editing, C.-L.C. and C.R.U. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Lim, Y.O.M.H.; Chen, X. Memetic Computation—Past, Present & Future [Research Frontier]. IEEE Comput. Intell. Mag. 2010, 5, 24–31. [Google Scholar] [CrossRef]
  2. Rundo, L.; Militello, C.; Vitabile, S.; Russo, G.; Sala, E.; Gilardi, M.C. A Survey on Nature-Inspired Medical Image Analysis: A Step Further in Biomedical Data Integration. Fundam. Inform. 2020, 171, 345–365. [Google Scholar] [CrossRef]
  3. Sarvamangala, D.R.; Kulkarni, R.V. A Comparative Study of Bio-inspired Algorithms for Medical Image Registration. In Advances in Intelligent Computing. Studies in Computational Intelligence; Mandal, J., Dutta, P., Mukhopadhyay, S., Eds.; Springer: Singapore, 2019; Volume 687. [Google Scholar] [CrossRef]
  4. Santamaría, J.; Rivero-Cejudo, M.L.; Martos-Fernández, M.A.; Roca, F. An Overview on the Latest Nature-Inspired and Metaheuristics-Based Image Registration Algorithms. Appl. Sci. 2020, 10, 1928. [Google Scholar] [CrossRef] [Green Version]
  5. Shaw, K.; Pandey, P.; Das, S.; Ghosh, D.; Malakar, P.; Dhabal, S. Image Registration using Bio-inspired Algorithms. In Proceedings of the 2020 IEEE 1st International Conference for Convergence in Engineering (ICCE), Kolkata, India, 5–6 September 2020; pp. 330–334. [Google Scholar] [CrossRef]
  6. Valsecchi, A.; Bermejo, E.; Damas, S.; Cordón, O. Metaheuristics for Medical Image Registration. In Handbook of Heuristics; Martí, R., Panos, P., Resende, M., Eds.; Springer: Cham, Switzerland, 2016. [Google Scholar] [CrossRef]
  7. Pirpinia, K.; Bosman, P.A.; Sonke, J.J.; van Herk, M.; Alderliesten, T. Evolutionary Machine Learning for Multi-Objective Class Solutions in Medical Deformable Image Registration. Algorithms 2019, 12, 99. [Google Scholar] [CrossRef] [Green Version]
  8. Panda, R.; Agrawal, S.; Sahoo, M.; Nayak, R. A novel evolutionary rigid body docking algorithm for medical image registration. Swarm Evol. Comput. 2017, 33, 108–118. [Google Scholar] [CrossRef]
  9. Sarvamangala, D.R.; Kulkarni, R.V. Swarm Intelligence Algorithms for Medical Image Registration: A Comparative Study. In Computational Intelligence, Communications, and Business Analytics, Proceedings of the CICBA 2017, Communications in Computer and Information Science, Kolkata, India, 24–25 March 2017; Mandal, J., Dutta, P., Mukhopadhyay, S., Eds.; Springer: Singapore, 2017; Volume 776. [Google Scholar] [CrossRef]
  10. Freitas, D.; Lopes, L.G.; Morgado-Dias, F. Particle Swarm Optimisation: A Historical Review Up to the Current Developments. Entropy 2020, 22, 362. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  11. Rundo, L.; Tangherloni, A.; Militello, C.; Gilardi, M.C.; Mauri, G. Multimodal medical image registration using Particle Swarm Optimization: A review. In Proceedings of the 2016 IEEE Symposium Series on Computational Intelligence (SSCI), Athens, Greece, 6–9 December 2016; pp. 1–8. [Google Scholar] [CrossRef]
  12. Das, A.; Bhattacharya, M. Affine-based registration of CT and MR modality images of human brain using multiresolution approaches: Comparative study on genetic algorithm and particle swarm optimization. Neural Comput. Applic. 2011, 20, 223–237. [Google Scholar] [CrossRef]
  13. Wu, Y.; Miao, Q.; Ma, W.; Gong, M.; Wang, S. PSOSAC: Particle Swarm Optimization Sample Consensus Algorithm for Remote Sensing Image Registration. IEEE Geosci. Remote Sens. Lett. 2018, 15, 242–246. [Google Scholar] [CrossRef]
  14. Bermejo, E.; Chica, M.; Damas, S.; Salcedo-Sanz, S.; Cordón, O. Coral Reef Optimization with substrate layers for medical Image Registration. Swarm Evol. Comput. 2018, 42, 138–159. [Google Scholar] [CrossRef]
  15. Bermejo, E.; Valsecchi, A.; Damas, S.; Cordón, O. Bacterial Foraging Optimization for intensity-based medical image registration. In Proceedings of the 2015 IEEE Congress on Evolutionary Computation (CEC), Sendai, Japan, 25–28 May 2015; pp. 2436–2443. [Google Scholar] [CrossRef]
  16. Costin, H.; Bejinariu, S.; Costin, D. Biomedical Image Registration by means of Bacterial Foraging Paradigm. Int. J. Comput. Commun. Control 2016, 11, 331–347. [Google Scholar] [CrossRef] [Green Version]
  17. Wen, T.; Liu, H.; Lin, L.; Wang, B.; Hou, J.; Huang, C.; Pan, T.; Du, Y. Multiswarm Artificial Bee Colony algorithm based on spark cloud computing platform for medical image registration. In Computer Methods and Programs in Biomedicine; Elsevier: Amsterdam, The Netherlands, 2020; Volume 192, p. 105432. ISSN 0169-2607. [Google Scholar] [CrossRef]
  18. Bejinariu, S.I.; Costin, H. A Comparison of Some Nature-Inspired Optimization Metaheuristics Applied in Biomedical Image Registration. Methods Inf. Med. 2018, 57, 280–286. [Google Scholar] [CrossRef]
  19. Xiaogang, D.; Jianwu, D.; Yangping, W.; Xinguo, L.; Sha, L. An Algorithm Multi-Resolution Medical Image Registration Based on Firefly Algorithm and Powell. In Proceedings of the 2013 Third International Conference on Intelligent System Design and Engineering Applications, Hong Kong, China, 16–18 January 2013; pp. 274–277. [Google Scholar] [CrossRef]
  20. Wang, J.; Zhang, M.; Song, H.; Cheng, Z.; Chang, T.; Bi, Y.; Sun, K. Improvement and Application of Hybrid Firefly Algorithm. IEEE Access 2019, 7, 165458–165477. [Google Scholar] [CrossRef]
  21. Styner, M.; Brechbuehler, C.; Székely, G.; Gerig, G. Parametric estimate of intensity inhomogeneities, applied to MRI. IEEE Trans. Med Imaging 2000, 19, 153–165. [Google Scholar] [CrossRef]
  22. Goshtasby, A.A. Image Registration: Principles, Tools and Methods; Springer Science & Business Media: London, UK, 2012. [Google Scholar] [CrossRef]
  23. Kvålseth, T. On Normalized Mutual Information: Measure Derivations and Properties. Entropy 2017, 9, 631. [Google Scholar] [CrossRef] [Green Version]
  24. Vila, M.; Bardera, A.; Feixas, M.; Sbert, M. Tsallis Mutual Information for Document Classification. Entropy 2011, 13, 1694–1707. [Google Scholar] [CrossRef]
  25. Amigó, J.M.; Balogh, S.G.; Hernández, S. A Brief Review of Generalized Entropies. Entropy 2018, 20, 813. [Google Scholar] [CrossRef] [Green Version]
  26. Brown, L.G. A Survey of Image Registration Techniques. ACM Comput. Surv. 1992, 24, 325–376. [Google Scholar] [CrossRef]
  27. Cocianu, C.L.; Stan, A. A New Evolutionary-Based Techniques for Image Registration. Appl. Sci. 2019, 9, 176. [Google Scholar] [CrossRef] [Green Version]
  28. Cocianu, C.L.; Stan, A.D.; Avramescu, M. Firefly-Based Approaches of Image Recognition. Symmetry 2020, 12, 881. [Google Scholar] [CrossRef]
  29. Eiben, A.; Smith, J. Introduction to Evolutionary Computing; Springer: Berlin, Germany, 2015. [Google Scholar] [CrossRef]
  30. Yang, X.S. Nature-Inspired Metauristic Algorithms; Luniver Press: Frome, UK, 2008. [Google Scholar]
  31. Arandiga, F.; Cohen, A.; Donat, R.; Matei, B. Edge detection insensitive to changes of illumination in the image. Image Vis. Comput. 2010, 28, 553–562. [Google Scholar] [CrossRef]
  32. Song, F.; Jutamulia, S. New wavelet transforms for noise insensitive edge detection. Opt. Eng. 2002, 41, 50–54. [Google Scholar] [CrossRef]
  33. Wu, K.C. Heuristic edge detector for noisy range images. In Intelligent Robots and Computer Vision XIII: 3D Vision, Product Inspection, and Active Vision; International Society for Optics and Photonics: Bellingham, WA, USA, 1994; Volume 2354, pp. 292–299. [Google Scholar] [CrossRef]
  34. Yale. Yale Face Database. Available online: http://cvc.yale.edu/projects/yalefaces/yalefaces.html (accessed on 14 December 2020).
  35. Belhumeur, P.; Hespanha, J.; Kriegman, D. Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection. IEEE Trans. Pattern Anal. Mach. Intell. 1997, 19, 711–720. [Google Scholar]
  36. Rehman, H.Z.U.; Lee, S. Automatic Image Alignment Using Principal Component Analysis. IEEE Access 2018, 6, 72063–72072. [Google Scholar] [CrossRef]
  37. Modersitzki, J. Numerical Methods for Image Registration; Oxford University Press: Oxford, UK, 2004. [Google Scholar]
Figure 1. An image of Subject 5. (a) Target image, (b) sensed image, (c) correctly restored image.
Figure 1. An image of Subject 5. (a) Target image, (b) sensed image, (c) correctly restored image.
Electronics 10 02606 g001
Figure 2. An image of Subject 14. (a) Target image, (b) sensed image, (c) correctly restored image.
Figure 2. An image of Subject 14. (a) Target image, (b) sensed image, (c) correctly restored image.
Electronics 10 02606 g002
Figure 3. Scaled and binarized images for Subject 5: (a) target; (b) sensed.
Figure 3. Scaled and binarized images for Subject 5: (a) target; (b) sensed.
Electronics 10 02606 g003
Figure 4. Scaled and binarized images for Subject 14: (a) target; (b) sensed.
Figure 4. Scaled and binarized images for Subject 14: (a) target; (b) sensed.
Electronics 10 02606 g004
Figure 5. The restored image—Subject 5.
Figure 5. The restored image—Subject 5.
Electronics 10 02606 g005
Figure 6. The restored image—Subject 14.
Figure 6. The restored image—Subject 14.
Electronics 10 02606 g006
Figure 7. The restored image, PAT—Subject 5.
Figure 7. The restored image, PAT—Subject 5.
Electronics 10 02606 g007
Figure 8. The restored image, PAT—Subject 14.
Figure 8. The restored image, PAT—Subject 14.
Electronics 10 02606 g008
Figure 9. The restored image, EO—Subject 5.
Figure 9. The restored image, EO—Subject 5.
Electronics 10 02606 g009
Figure 10. The restored image, EO—Subject 14.
Figure 10. The restored image, EO—Subject 14.
Electronics 10 02606 g010
Figure 11. An image of Subject 4: (a) target image; (b) sensed image and (c) correctly restored image.
Figure 11. An image of Subject 4: (a) target image; (b) sensed image and (c) correctly restored image.
Electronics 10 02606 g011
Figure 12. An image of Subject 10: (a) target image; (b) sensed image and (c) correctly restored image.
Figure 12. An image of Subject 10: (a) target image; (b) sensed image and (c) correctly restored image.
Electronics 10 02606 g012
Figure 13. Scaled and binarized images for Subject 4: (a) target; (b) sensed.
Figure 13. Scaled and binarized images for Subject 4: (a) target; (b) sensed.
Electronics 10 02606 g013
Figure 14. Scaled and binarized images for Subject 10: (a) target; (b) sensed.
Figure 14. Scaled and binarized images for Subject 10: (a) target; (b) sensed.
Electronics 10 02606 g014
Figure 15. The restored image—Subject 4.
Figure 15. The restored image—Subject 4.
Electronics 10 02606 g015
Figure 16. The restored image—Subject 10.
Figure 16. The restored image—Subject 10.
Electronics 10 02606 g016
Figure 17. The restored image using PAT—Subject 4.
Figure 17. The restored image using PAT—Subject 4.
Electronics 10 02606 g017
Figure 18. The restored image using PAT—Subject 10.
Figure 18. The restored image using PAT—Subject 10.
Electronics 10 02606 g018
Figure 19. The restored image using EO—Subject 4.
Figure 19. The restored image using EO—Subject 4.
Electronics 10 02606 g019
Figure 20. The restored image using EO—Subject 10.
Figure 20. The restored image using EO—Subject 10.
Electronics 10 02606 g020
Table 1. The results of applying Algorithm 1 in case of pairs of binary images.
Table 1. The results of applying Algorithm 1 in case of pairs of binary images.
InputMeanRTMeanDice MeanNI S MeanSNRMeanPSNR
S 1 11.450.920.8020.9670.19
S 2 10.820.920.8218.1972.99
S 3 10.050.920.8120.4470.73
S 4 7.560.920.8213.7677.47
S 5 12.430.910.8111.9379.39
S 6 9.380.920.8112.3378.99
S 7 8.280.920.8111.5779.73
S 8 7.950.920.8111.6179.69
S 9 10.300.920.8113.1378.09
S 10 9.060.920.8112.3178.88
S 11 9.150.920.8212.978.31
S 12 8.110.920.8111.7079.46
S 13 11.970.920.8112.2478.94
S 14 7.680.920.8213.4877.74
S 15 9.930.920.789.1381.81
S 16 9.250.920.8212.8678.34
Table 2. The numerical results obtained by applying the proposed method.
Table 2. The numerical results obtained by applying the proposed method.
Image SampleMeanRTRSNR RNMI S RNMI α T
115.189960.844510.8949460.976625
225.759120.9302050.9376610.987729
36.9628340.7586790.8863080.968475
49.1991120.8181030.9255620.980803
522.920940.9842060.9171820.982447
642.076030.7551140.8428850.960883
78.6064540.9422320.9766990.993051
811.657040.9731770.9375040.983867
99.1630150.8605620.873840.974975
106.5175120.870360.8916110.980161
1122.926360.8200370.8778440.973008
12119.430.8682110.9174240.98466
1318.723450.8549310.959130.985814
1416.166620.8626840.9628720.989826
1539.5140.9617560.9896510.995074
1615.484230.9476860.8623590.973975
1715.813410.9655140.9272760.983535
1810.270140.9550420.952050.98956
197.5527460.8647620.9299750.971995
2015.219460.9028820.9159160.976381
21146.53440.8459770.9168050.985143
227.9545410.7284340.8698760.964251
239.1465190.8616260.9511030.986323
2443.526190.9044230.9586070.984734
2511.136690.8163020.9095040.966766
2622.709440.874230.9012170.97484
27183.89190.9249760.8688970.97005
2839.189390.7914530.8684680.961354
2966.534520.8752320.9334250.975518
309.4542510.9680320.9739150.992451
Table 3. The numerical results obtained by applying the PAT method.
Table 3. The numerical results obtained by applying the PAT method.
Image SampleRSNR RNMI S RNMI α T
10.8032930.8134670.933241
20.9345740.8868840.945906
30.2957910.3532570.696968
40.7666590.8433430.878612
50.9058750.6951980.943975
60.2748740.3726850.772999
70.8403010.9100590.950973
80.7133650.5271110.828832
90.6267970.6984610.841559
100.2852190.3510830.700893
110.2517740.3596190.739244
120.3901970.4189680.747504
130.808740.8750540.925707
140.8276270.8926070.947912
150.3662020.3840680.732574
160.706480.5507860.854011
170.688480.594780.841351
180.3232530.3931440.735975
190.346270.4020990.734227
200.4539270.4231020.731206
210.1919220.322470.810919
220.8250420.9106280.951726
230.8513940.9094860.949252
240.3543950.4244610.712203
250.271670.3622910.697021
260.6254610.7880120.889674
270.4061820.3874720.767243
280.3989960.4433410.734644
290.3373420.4270120.748608
300.3587850.4236410.721053
Table 4. The numerical results obtained by applying the EO method.
Table 4. The numerical results obtained by applying the EO method.
Image SampleRSNR RNMI S RNMI α T
10.9037361.0281680.974588
20.598260.5875830.829827
30.8738051.0110280.96525
40.8814051.0147170.967766
50.8970921.0397720.998585
60.3272350.3781490.699523
70.902521.0301840.971119
80.9619391.0080780.976823
90.8644341.0137260.969337
100.5184220.6096770.861264
110.9432441.0274090.986622
120.337080.3913520.7305
130.9066371.0112590.947328
140.9973611.021430.959039
150.3934840.4358510.763053
160.9075020.9849020.965526
170.9334751.0028380.968684
180.2476420.4679260.840874
190.2647230.4365410.776755
200.7811810.7647760.911667
210.0789780.306980.533648
220.9175981.015580.961737
230.9797041.0305730.961777
240.3182060.4068620.601202
250.2586630.3675880.584295
260.2586630.3675880.584295
270.3902470.4078120.8087
280.3915230.4629680.745298
290.3440730.4960780.822182
300.8869350.9867150.955883
Table 5. The numerical results obtained by applying the proposed method.
Table 5. The numerical results obtained by applying the proposed method.
Image SampleMeanRTRSNR RNMI S RNMI α T
135.690.83917690.87910360.9670912
227.241.00498940.94474870.9942313
327.250.7106060.86974910.9563255
445.820.89969280.90999420.9773764
521.80.84817220.8685520.9696185
623.990.85405150.89629320.9770259
753.961.05750770.94569530.9982119
827.220.97615960.90071550.9778646
920.560.84860360.86587710.9687854
1021.530.84316120.92512210.9780587
1175.750.8350150.89704380.9761927
1289.930.87771210.88550520.9764733
1343.540.84817750.94004610.9807903
1455.530.84747740.93395620.9808082
15188.340.84473510.9069710.9555983
1630.750.99051180.92027320.9832426
1733.460.96552970.97674840.9851373
1825.110.98377810.89929680.9727791
1974.30.85619470.86597470.9607904
2022.90.92649670.91244750.9794836
2142.610.96770680.91249450.98168
2220.470.76392910.86793230.9587384
2331.920.76563390.89634420.9636461
24128.130.75548260.8696410.9579413
2522.670.95210660.95141850.9883335
2633.690.81878330.87847680.9676486
2745.900.90235680.89951640.971906
2841.430.81717420.90566560.9725819
2940.830.77008650.92489330.972424
3027.30.81749080.89470070.9685286
Table 6. The numerical results obtained by applying the PAT method.
Table 6. The numerical results obtained by applying the PAT method.
Image SampleRSNR RNMI S RNMI α T
10.6803570.6719150.898062
20.7301960.7815020.860837
30.2855520.349460.698664
40.7029410.7358720.912137
50.7112880.6692230.899995
60.286270.3851590.772776
70.8380950.9480660.938333
80.7862460.5660110.860282
90.7589410.7556680.881690
100.2827570.3135530.656165
110.2964010.3729480.780554
120.4109220.3424840.622750
130.6885580.7394240.887327
140.3880740.3943770.731979
150.3604520.3786280.719361
160.8589770.8055540.941650
171.2595011.1874130.928381
180.376370.3985320.784592
190.3766740.3719830.715186
200.4721150.4017410.744438
210.1796250.3157110.787056
220.8509590.8957470.940529
230.7186420.8345650.934975
240.3355310.4032260.699645
250.3002160.3484450.690366
260.3540870.3853840.759621
270.416680.3731880.773804
280.3910130.4362760.727903
290.3468010.4310990.770062
300.5102310.5174760.729654
Table 7. The numerical results obtained by applying the EO method.
Table 7. The numerical results obtained by applying the EO method.
Image SampleRSNR RNMI S RNMI α T
10.48404950.48512640.8291119
20.45609610.49420830.7928845
30.31807290.38669960.6517158
40.49610640.54165560.8193649
50.59404450.62147040.8908949
60.3276120.39907960.7584589
70.59733590.68464580.8605331
80.86035180.74464690.9115472
90.53865130.55277230.8304493
100.13347810.15329150.2963478
110.40443890.47261330.8341134
120.22026320.14082880.167821
130.50981960.55992190.8161328
140.43581680.49875850.7843876
150.28787390.39218590.6948516
160.99840820.83565350.9701007
171.24886980.93572470.9713906
180.25577190.43269060.7551308
190.12005480.16496430.2182064
200.47607540.48760990.8062592
210.01164940.3155980.5602149
220.52505370.57731890.8285102
230.48176680.58609950.8326726
240.29454080.41879630.6699495
250.31475390.3855770.6504573
260.42079690.48089880.8230934
270.18894740.20754590.4060359
280.41671640.48741890.7887366
290.35237380.48889010.8267513
300.24775790.38081040.4784807
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Cocianu, C.-L.; Uscatu, C.R. Cluster-Based Memetic Approach of Image Alignment. Electronics 2021, 10, 2606. https://doi.org/10.3390/electronics10212606

AMA Style

Cocianu C-L, Uscatu CR. Cluster-Based Memetic Approach of Image Alignment. Electronics. 2021; 10(21):2606. https://doi.org/10.3390/electronics10212606

Chicago/Turabian Style

Cocianu, Catalina-Lucia, and Cristian Răzvan Uscatu. 2021. "Cluster-Based Memetic Approach of Image Alignment" Electronics 10, no. 21: 2606. https://doi.org/10.3390/electronics10212606

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop