Skip to Content
SymmetrySymmetry
  • Article
  • Open Access

24 February 2022

29 Pages

Hybrid Reptile Search Algorithm and Remora Optimization Algorithm for Optimization Tasks and Data Clustering

and
1
Computer Engineering Department, Umm Al-Qura University, Makkah 24382, Saudi Arabia
2
Faculty of Computer Sciences and Informatics, Amman Arab University, Amman 11953, Jordan
3
School of Computer Sciences, Universiti Sains Malaysia, Pulau Pinang 11800, Malaysia
*
Authors to whom correspondence should be addressed.

Abstract

Data clustering is a complex data mining problem that clusters a massive amount of data objects into a predefined number of clusters; in other words, it finds symmetric and asymmetric objects. Various optimization methods have been used to solve different machine learning problems. They usually suffer from local optimal problems and unbalance between the search mechanisms. This paper proposes a novel hybrid optimization method for solving various optimization problems. The proposed method is called HRSA, which combines the original Reptile Search Algorithm (RSA) and Remora Optimization Algorithm (ROA) and handles these mechanisms’ search processes by a novel transition method. The proposed HRSA method aims to avoid the main weaknesses raised by the original methods and find better solutions. The proposed HRSA is tested on solving various complicated optimization problems—twenty-three benchmark test functions and eight data clustering problems. The obtained results illustrate that the proposed HRSA method performs significantly better than the original and comparative state-of-the-art methods. The proposed method overwhelmed all the comparative methods according to the mathematical problems. It obtained promising results in solving the clustering problems. Thus, HRSA has a remarkable efficacy when employed for various clustering problems.

1. Introduction

Unsupervised learning methods are instrumental in machine learning because they may explore data without any prior knowledge of them, i.e., there are no labels linked with the data [1]. These algorithms try to represent the data’s underlying mechanism or pattern, which may be helpful for things such as decision making and forecasting future inputs. Clustering and feature methods are classic examples of unsupervised algorithms [2,3].
Clustering is an important unsupervised method that uses a number of data objects or items to discover homogenous groupings [4]. Clustering is used to split data items into groups so that objects in the same cluster are similar to one another and different from objects in other clusters. Algorithms for clustering have been utilized in a wide range of applications. They are used in biology to extract interesting patterns from gene expression [5,6]. Additionally, they are used to divide sensor networks into groups in wireless sensor networks and in information retrieval for grouping text content and generating thematic hierarchies. For a product search task, clustering can be utilized to group people or objects [7].
The data and available information are the root of the fast development model of today [8], and are extensively employed in various information technology applications such as manufacturing, marketing, and commerce. Because there are so many data, data mining is one of the most critical methods for extracting meaningful information. Data mining is a revolutionary idea utilized to tailor knowledge across numerous industries, such as medical records examination, client transaction analysis, and market surveying programs [9]. The clustering and classification algorithms are the two most essential requirements for extracting information from data mining. As a result, data clustering has become a valuable and demanding process for generating clusters where data are clustered together. Data clustering is achieved by using two clustering mechanisms: hierarchical and partitional clustering [10,11]. The data items are grouped hierarchically, either in the shape of a tree or a cluster analysis, in a clustering algorithm. The partitional clustering technique, on the other hand, produces non-overlapping groups. This method has a propensity to cope with noise and outliers, manageability, integration, usability, and other issues in real-world applications. K-means clustering, Fuzzy c-means method, density accurate and robust, and so on are some of the most frequently used classification techniques [12,13].
Several related works have been conducted based on using optimization techniques to solve the clustering problems, which is worth mentioning [14,15]. They motivated us to propose a new efficient method to deal with various data clustering issues.
A novel Class Topper Optimization (CTO) method is presented in [16]. The cognitive intelligence of pupils in a class inspired the optimization method. A population-based search technique is used. The answer narrows towards the optimal solution in this technique. This may lead to a worldwide optimal solution. A clustering challenge is explored to validate the algorithm’s performance. For real-time testing, five common data sets are examined. A comparison of the performance of the proposed approach to many well-known existing optimization techniques reveals that it outperforms them all.
A novel clustering approach is presented based on the evolution particle swarm optimization (EPSO) algorithm [17]. The suggested approach is based on the development of swarm generations. At first, the particles are equally dispersed in the input data set, and a new swarm population emerges after a defined number of repetitions. This paper covers the novel technique and its initial implementation and testing using real clustering benchmark data. The findings demonstrate that the technique creates compact clusters and is effective.
A modernized firefly technique is combined with the well-known PSO approach to handle automated data clustering difficulties [18]. The proposed fusion technique is designed against four standard metaheuristic algorithms from the literature using twelve common datasets and the two moons dataset to see how well it performs. The thorough computational tests and results analysis reveals that the suggested approach outperforms the standard published methods.
The ABCWOA method, which uses Random Memory (RM) and Elite Memory (EM) to overcome investigation problems and late convergence in ABC, is presented in [19]. The search phase for the bait in the whale optimization algorithm (WOA) is employed by RM in the ABCWOA method, and EM is also used to boost convergence. The ABCWOA algorithm outperforms other metaheuristic algorithms, according to the results.
This study suggests a novel heuristic strategy based on the big bang–big crunch algorithm for clustering issues [20]. In comparison to previous heuristic strategies, the suggested method not only utilizes the use of the heuristic nature to reduce standard clustering algorithms such as k-means, but it also benefits from the memory-based scheme. Furthermore, the proposed algorithm’s system is assessed using various benchmark functions, including well datasets. The experimental findings reveal that the suggested technique outperforms similar algorithms substantially.
This study proposes K–NM–PSO, a hybrid approach that combines the K-means algorithm, Nelder–Mead simplex search, and PSO [21]. The K–NM–PSO method, like the K-means algorithm, looks for cluster centers in any data set. However, it can also discover global optima effectively and rapidly. The results suggest that K–NM–PSO is reliable and appropriate for data clustering. Other optimization methods can be used to solve the data clustering problems. Additionally, many optimization methods have been used to solve various problems [22]. It is also worth mentioning that the optimization methods proved their ability in solving many other problems such as solve complex engineering optimization problems [23], reactive power planning problems [24], parameter estimation problems [25], voltage-constrained power problems [26], and others.
The Reptile Search Algorithm (RSA) is a recent and novel optimizer that was proposed by Abualigah et al. in [27]. This method mimics the behavior of crocodiles on two principal activities—encircling and hunting—each of which is performed in two main strategies: (1) encircling: high walking or belly walking; (2) hunting: hunting coordination or hunting cooperation. The Remora Optimization Algorithm (ROA) is a recent and novel optimizer that has been proposed by Jia et al. in [28], a new bionics-based optimization method. The main inspiration of the ROA is Remora’s behavior.
Various optimization methods have been employed in the literature to solve different machine learning problems, especially clustering problems. The proposed method’s primary motivation is that the recent studies’ that used optimization methods for solving similar problems yielded performances that were not good enough, and a new, improved method can find new optimal solutions. Optimization methods usually suffer from optimal local problems and unbalance between the search mechanisms (i.e., exploration and exploitation). Moreover, we selected the most reputed methods to incorporate them and created a new hybrid approach to yield new and better results. This paper proposed a novel hybrid optimization search method for solving challenging optimization problems. The proposed method is called HRSA, which combines the original Reptile Search Algorithm (RSA) and Remora Optimization Algorithm (ROA) and handles these mechanisms’ search processes by a novel transition method. The proposed HRSA method solves the main weaknesses raised by the original methods and finds better solutions. The proposed HRSA is tested on two classes of optimization problems: twenty-three benchmark test functions and eight real-world data clustering problems. The results illustrate that the proposed HRSA method performs significantly better than the original and comparative state-of-the-art methods. The proposed method overwhelmed all the comparative methods according to the mathematical problems. It obtained promising results in solving the clustering problems. Thus, HRSA has a remarkable ability to be employed for various clustering problems.
The major contributions in this paper are covered in the following points.
  • A novel hybrid optimization method is proposed using the original Reptile Search Algorithm (RSA) and Remora Optimization Algorithm (ROA).
  • A new transition method is proposed to handle the mechanisms’ search processes and help the proposed method in enabling the suitable search operator during the optimization process.
  • The performance of the proposed HRSA method is tested on several benchmark functions to show the mai.
  • A real-world problem, data clustering, is used to prove further the proposed HRSA’s ability to deal with complicated problems, finding symmetric and asymmetric objects.
  • The results showed the superiority of the proposed method in solving the given problems compared to other various state-of-the-art methods.
In the rest of the paper, Section 2 presents the background of the used methods and the procedure of the proposed method. Section 3 shows the experiments, results, and discussion. Section 4 gives the research conclusions and potential future work.

2. Background

This section presents the original methods used in the proposed method. Additionally, the main procedure of the proposed hybrid Reptile Search Algorithm (RSA) and Remora Optimization Algorithm (ROA) with a novel transition mechanism (HRSA) is presented.

2.1. Reptile Search Algorithm (RSA)

This section presents the original Reptile Search Algorithm (RSA) and its procedure. The basic Reptile Search Algorithm (RSA) is described in its exploration (global search) and exploitation (local search) stages, which were inspired by the encircling mechanics, hunting processes, and social behavior of crocodiles in real life [27].

2.1.1. Encircling Phase (Exploration)

The exploratory behavior (encircling) of the RSA is introduced in this section. Crocodiles engage in two movements when encircling: high walking and belly walking, according to their encircling behavior [29].
The RSA switches between exploration and exploitation search phases based on four scenarios: splitting the number of iterations into four parts; dividing the total number of iterations into four parts. The RSA exploration mechanisms investigate the search regions and approach to discover a better answer based on two major search techniques.
One condition must be met throughout this phase of the search. The high walking search method is performed according to t ≤ T 4 , and the belly walking search method is performed according to t ≤ 2 T 4 and t > T 4 . The position-updating process is presented in Equation (1).
x ( i , j ) ( t + 1 ) = B e s t j ( t ) × − η ( i , j ) ( t ) × β − R ( i , j ) ( t ) × r a n d , t ≤ T 4 B e s t j ( t ) × x ( r 1 , j ) × E S ( t ) × r a n d , t ≤ 2 T 4 a n d t > T 4
where B e s t j ( t ) is the best-obtained solution, r a n d is a random number, t is the current iteration, and T is the maximum iterations. η ( i , j ) is the hunting parameter determined by Equation (2). β is a parameter fixed to 0.1. The reduce function ( R ( i , j ) ) is determined by Equation (3). r 1 − r 4 are random numbers, x ( r 1 , j ) is a random position, and N is the used solutions. Evolutionary Sense ( E S ( t ) ) is a probability parameter determined by Equation (4).
η ( i , j ) = B e s t j ( t ) × P ( i , j ) ,
R ( i , j ) = B e s t j ( t ) − x ( r 2 , j ) B e s t j ( t ) + ϵ ,
E S ( t ) = 2 × r 3 × 1 − 1 T ,
where ϵ a small value. P ( i , j ) is a difference parameter determined by Equation (5).
P ( i , j ) = α + x ( i , j ) − M ( x i ) B e s t j ( t ) × ( U B ( j ) − L B ( j ) ) + ϵ ,
where M( x i ) represents the average positions determined by Equation (6). U B ( j ) and L B ( j ) are the upper and lower boundaries. α is a parameter fixed to 0.1.
M ( x i ) = 1 n ∑ j = 1 n x ( i , j ) ,

2.1.2. Hunting Phase (Exploitation)

The exploitative behavior of RSA is discussed in this section. Crocodiles use two hunting techniques: hunting coordination and hunting collaboration, according to their hunting behavior.
The searching in this phase (hunting coordination) is executed and determined according to t ≤ 3 T 4 and t > 2 T 4 ; otherwise, the hunting cooperation is executed according to t ≤ T and t > 3 T 4 . The position-updating processes are presented in Equation (7):
x ( i , j ) ( t + 1 ) = B e s t j ( t ) × P ( i , j ) ( t ) × r a n d , t ≤ 3 T 4 a n d t > 2 T 4 B e s t j ( t ) − η ( i , j ) ( t ) × ϵ − R ( i , j ) ( t ) × r a n d , t ≤ T a n d t > 3 T 4
where B e s t j ( t ) is the best obtained solution, and η ( i , j ) is the hunting parameter determined by Equation (2). P ( i , j ) is a difference parameter determined by Equation (5). η ( i , j ) is the hunting parameter determined by Equation (2). R ( i , j ) is determined by Equation (3).

2.2. Remora Optimization Algorithm (ROA)

This section presents the original Remora Optimization Algorithm (ROA) and its procedure, which is as follows [28].

2.2.1. Free Travel (Exploration)

SFO Strategy
The formulation for this algorithm’s location update was modeled based on the algorithm’s elite notion, yielded by Equation (8).
R i t + 1 = R b e s t t − r a n d × ( R b e s t t − R r a n d t 2 ) − R r a n d t
where R r a n d t is a random location.
Experience Attack
To evaluate whether or not it is required to replace the host, the tuyu must regularly take modest steps around the host, analogous to the development of knowledge. The formula for modeling the principles as mentioned above is as follows:
R a t t = R i t − ( R i t − R p r e ) × r a n d n
where R p r e is the position of the previous iteration, and R a t t is a tentative step.
The evaluation of the fitness function of the present solution f( R i t ) and the attempted solution f( R a t t ) is described as the decision of this step. As an example, when addressing the minimal problem, if the fitness function value produced by the proposed solution is less than the present solution,
f ( R i t ) > f ( R a t t )
Remora chooses a different mechanism for local optima, as shown in the following section. It returns to host choosing if the fitness function value of the attempted solution is greater than that of the existing solution.
f ( R i t ) < f ( R a t t )

2.2.2. Eat Thoughtfully (Exploitation)

WOA Strategy
The location update formula of Remora connected to the whale was recovered using the original WOA technique, as seen in the Equations below:
R i + 1 = D × e α × c o s ( 2 π α ) + R i
α = r a n d × ( a − 1 ) + 1
a = − ( 1 + t T )
D = | R b e s t − R i |
Their locations might be considered the same when a Remora is on a whale in the broader solution space. D is the space between both the hunter and the prey, α is a random value in the range [−1,1], and a is a number that decreases exponentially between [−2, −1].
Host Feeding
The exploitation procedure is further subdivided into host feeding. At this point, the optimal solution can be condensed to the host’s location area. Incremental steps can be thought of as moving on or around the host, which can be mathematically characterized as:
R i t = R i t + A
A = B × ( R i t − C × R b e s t )
B = 2 × V × r a n d − V
V = 2 × ( 1 − t T )
A was employed here to represent a tiny movement connected to the size space of the host and remora. A Remora factor (C) was utilized to limit the position of Remora to distinguish between the host and Remora. If the size of the host is 1, then the volume of the Remora is about a percent of the host’s volume.

2.3. The Proposed HRSA Method

This section presents the main procedure of the proposed hybrid Reptile Search Algorithm (RSA) and Remora Optimization Algorithm (ROA) with a novel transition mechanism (HRSA).
The suggested HRSA is used to solve various issues using two major search strategies and a new mean transition mechanism. The traditional RSA is the principal search technique, which has strong global search capabilities but occasionally suffers from local search problems, premature convergence, and disequilibrium between global and local search methods. As a result, the second search strategy, the ROA, avoids the local search issue and premature convergence. The strategy can improve the RSA’s searchability by generating new local solutions based on the best solutions currently available. Furthermore, a new mean transition mechanism is provided to govern the execution of the search methods (i.e., RSA and ROA) in the proposed HRSA to tackle the disequilibrium problem between the global and local search techniques. As a result, the search space may be efficiently expanded by incorporating new approaches from other places. These solutions inspire the proposed HRSA to use more robust solutions to obtain better results.

2.3.1. Initialization Phase

The optimization process in RSA begins with a set of candidate solutions (X) created stochastically, as indicated in Equation (20). The best obtained solution is deemed nearly optimal in each iteration.
X = x 1 , 1 ⋯ x 1 , j x 1 , n − 1 x 1 , n x 2 , 1 ⋯ x 2 , j ⋯ x 2 , n ⋯ ⋯ x i , j ⋯ ⋯ ⋮ ⋮ ⋮ ⋮ ⋮ x N − 1 , 1 ⋯ x N − 1 , j ⋯ x N − 1 , n x N , 1 ⋯ x N , j x N , n − 1 x N , n
where X is a collection of the solutions that are created by using Equation (21), x i , j is the j t h position of the i t h solution, N is the number of solutions, and n is the dimension size.
x i j = r a n d × ( U B − L B ) + L B , j = 1 , 2 , . . . , n
where r a n d is a random value and L B and U B denote the lower and upper bound, respectively.

2.3.2. The Proposed Mean Transition Mechanism (MTM)

The mean transition mechanism (MTM) is devised and given in Algorithm 1 in this section. This technique is used to control the search process and the transition between the RSA and the MT. The transition from one search process to the next is quite delicate. It necessitates an effective way to change the update procedures across the various approaches. The suggested MTM’s fundamental concept is to regulate the search techniques when the fitness function does not increase after five iterations (I). When no gains can be made by experimentation, the number of iterations changes.
Algorithm 1 The proposed Mean Transition Mechanism (MTM).
1:
Initialize the TM parameter value (TM = 0).
2:
sumFF = 0;
3:
for (t = 1 to T) do
4:
   sumFF = sumFF + currentFF
5:
   C = C + 1;
6:
   if (currentFF ≠ sumFF) then
7:
     if (C > I) then
8:
        TM = flip(TM);
9:
        sumFF = 0;
10:
        C = 0;
11:
     end if
12:
   end if
13:
end for
T M is a binary variable utilized to adjust the search process between the RSA and the MT, s u m F F is a variable used to calculate the mean fitness function values, c u r r e n t F F is the current fitness function value, C is a counter, I is the number of iterations to change when no improvements and it can be adjusted by experiment, and f l i p is a function to flip the TM value from 0 to 1 or vice versa, as described in Algorithm 1.

2.3.3. The Detailed Process of the HRSA

This section goes through the proposed technique in detail. The main purpose of the proposed strategy is to obtain better results than those yielded by the existing approaches. We also want to avoid the defects presented by the original processes, such as local search issues, convergence constraints, and search balance concerns.
To conclude, the proposed HRSA starts with the creation of a random set of solutions. During the renewal process, the HRSA’s search criteria explore possible placements of the current best solution. Each answer progresses to the next stage of the procedure. According to Figure 1, the suggested HRSA uses the Reptile Search Algorithm (RSA) and the Remora Optimization Algorithm (ROA) approach. Each iteration will update and improve the potential solutions using one of these search methodologies.
Figure 1. Flowchart of the proposed HRSA.
The suggested HRSA’s search techniques are divided into two categories, as shown Figure 1: RSA and ROA. Following this, the RSA’s search techniques are divided into global and local search methods. There are two search techniques for each method: (1) at the global level, with high and walking tactics; and (2) hunting coordination and cooperation at the local level. Candidate solutions try to figure out what is going on in the search space t ≤ T 2 and seek to discover the near-optimal solution if t > T 2 . For the first section, if T M == 0, the search process of the RSA will be performed; otherwise, the search process of the ROA will be conducted. In the exploration step of the RSA, the first search strategy from the global search methods is performed when t ≤ T 4 , and the second strategy from the global search processes are accomplished when t ≤ 2 T 4 and t > T 4 . In the exploitation of the conventional RSA, the first search approach from the local methods is carried out when t ≤ 3 T 4 and t > 2 T 4 ; otherwise, the second search approach from the local methods is executed, when t ≤ T and t > 3 T 4 . Finally, the HRSA is finished when it reaches the end criterion. The complete rule of the recommended HRSA is shown in Figure 1.

3. Results and Discussion

In this section, the proposed HRSA method is evaluated using twenty-three common benchmark functions.

3.1. Experimental Settings

The results of the proposed HRSA method are compared with other state-of-the-art methods including Aquila Optimizer (AO), Dwarf Mongoose Optimization Algorithm (DMOA), Whale Optimization Algorithm (WOA), Sine Cosine Algorithm (SCA), Dragonfly Algorithm (DA), Grey Wolf Optimizer (GWO), Particle Swarm Optimizer (PSO), Ant Lion Optimizer (ALO), Reptile Search Algorithm (RSA), Remora Optimization Algorithm (ROA), Arithmetic Optimization Algorithm (AOA), and the proposed HRSA method. All the tested methods are tuned according to the original paper and its parameters, as shown in Table 1. Each algorithm is executed 30 times using 50 solutions and 1000 iterations.
Table 1. Parameter values of the tested algorithms.
Twenty-three benchmark problems are used with different characterizations, as shown in Table 2. These test functions are categorized into three main categories, unimodal, multimodal, and multimodal, with fixed dimension functions. They are always used in the domain of machine learning and optimization algorithms [30,31].
Table 2. Details of the tested benchmark functions.

3.2. Benchmark Function: Experiments

In this section, the results of the comparative methods are presented in several formats to validate their performances. Figure 2 shows the qualitative analysis of the first thirteen benchmark functions (F1–F13) in terms of function topology, trajectory curves of the first dimension values, the average fitness function values, and the convergence curves of the original methods (i.e., RSA and ROA) and the proposed HRSA method.
Figure 2. Qualitative analysis of F1–F13.
It is clear from Figure 2 that the proposed method obtained promising results, and it can be an excellent alternative in this domain. The proposed method reaches near-optimal solutions for all the tested functions according to the functions typologies. Additionally, the trajectory curves of the first dimension values show that the proposed method acceptably changes the values. This refers to the diversity of the position-updating mechanisms. The proposed method obtained the minimal values regarding the average fitness function values. The convergence is always near the optimal area. Finally, the convergence curves show that the proposed method overwhelmed the original methods in all the tested problems.
Table 3 shows the proposed HRSA method in solving thirteen benchmark functions with different population sizes. This experiment was conducted to find the optimal number of solutions to proceed with the following experiments. It is clear from Table 3 that the proposed HRSA obtained the best results when the population size was equal to 50, followed by a population of 45. According to the given results from the Friedman ranking test, when the population size is equal to 50, the proposed HRSA is ranked as the first best method followed by the number of solutions 45, 35, 40, 30, 25, 20, 15, 10, and 5. Thus, the proposed method obtains better results when the number of solutions is larger.
Table 3. The results of several population sizes on F1–F13.
Table 4 shows the results of the comparative methods and the proposed HRSA for solving thirteen benchmark functions problems (F1–F13) when the dimension size is equal to 10. It is clear from this table that the proposed method obtained more promising results compared to the other comparative methods. This outcome proved the ability of the modified method to avoid the main problems of the original methods. According to the Wilcoxon signed-rank test, the proposed HRSA overwhelmed almost all the comparative methods in all the given problems. For example, the proposed HRSA significantly overwhelmed DOMA and DA in F1. Additionally, it significantly improved compared to ROA, PSO, GWO, DA, DMOA, and WOA in F4. According to the Friedman ranking test, the proposed HRSA is ranked as the first-best method, followed by the AO method as the second-best method, the RSA as the third-best method, the GWO as the fourth-best method, the AOA as the fifth-best method, the WOA as the sixth-best method, the PSO as the seventh-best method, the ROA as the eighth-best method, the DMOA as the ninth-best method, the SCA as the tenth-best method, the ALO as the eleventh-best method, and finally, the DA as the twentieth-best method. We concluded that the proposed method can find better solutions than the other methods’ solutions.
Table 4. The results of the comparative methods and the proposed HRSA methods for F1–F13, Dim = 10.
Table 5 shows the results of the comparative methods and the proposed HRSA for solving thirteen benchmark functions problems (F1–F13) when the dimension size is equal to 100. It is clear from this table that the proposed method achieved better results compared to the other comparative methods for these cases. This result demonstrated the improved method’s capacity to overcome the original method’s major flaws. According to the Wilcoxon signed-rank test, the proposed HRSA overwhelmed almost all the comparative methods in all the given cases. For example, the proposed HRSA significantly overwhelmed ROA, PSO, GWO, SCA, and DMOA in F1. Additionally, it led to significant improvements compared to AOA, ROA, RSA, ALO, PSO, GWO, SCA, DMOA, and WOA in F10. According to the Friedman ranking test, the proposed HRSA is ranked as the first-best method, followed by the AO method as the second-best method, the RSA as the third-best method, the WOA as the fourth-best method, the GWO as the fifth-best method, the AOA as as the sixth-best method, the PSO as the seventh-best method, the DMOA as the eighth-best method, the ROA as the eighth-best method, the ALO as the tenth-best method, the DA as the eleventh-best method, and finally, the SCA as the twentieth-best method. We came to the conclusion that the proposed HRSA can find better solutions than previous methods when the dimensions are greater (i.e., Dim = 100).
Table 5. The results of the comparative methods and the proposed HRSA methods for F1–F13, Dim = 100.
Table 6 shows the results of the comparative methods and the proposed HRSA methods for fixed dimensional functions (F14–F23). As shown in the table, the suggested technique outperformed the existing comparison methods in these circumstances. This result revealed the enhanced approach’s ability to overcome the critical weaknesses of the old method. According to the Wilcoxon signed-rank test, the proposed HRSA overwhelmed almost all the comparative methods in all the given cases. For example, the proposed HRSA significantly overwhelmed DMOA, SCA, DA, ALO, RSA, and ROA in F14. Additionally, it gave a significant improvement compared to AO, DMOA, and SCA in F23. According to the Friedman ranking test, the proposed HRSA ranked as the first-best method, followed by the RSA method as the second-best method, the ROA as the third-best method, the GWO as the third-best method, the ALO as the fifth-best method, the DMOA as the sixth-best method, the GWO as the seventh-best method, the AO as the eighth-best method, the DA as the ninth-best method, the WOA as the tenth-best method, the SCA as the eleventh-best method, and finally, the AOA as the twentieth-best method. We came to the conclusion that the proposed HRSA can find better solutions than previous methods when the dimensions are fixed.
Table 6. The results of the comparative methods and the proposed HRSA methods for fixed dimensional functions (F14–F23).
The convergence behaviors of the comparative optimization methods employing the evaluated twenty-three benchmark functions are shown in Figure 3. The optimization history and trajectory during the evolution processes are depicted in these diagrams. In comparison to the previous comparative approaches, it is evident that the proposed HRSA obtains optimum solutions in all of the studied cases and converges to the optimal solution smoothly. Furthermore, the observed convergence tendency is appealing and reinforces the original method’s primary flaws. As a result, the primary issues with the original approaches are addressed, as seen in the figures. Figure 4 shows the distribution values of the given position during the optimization process. It is clear that the proposed method generates higher distribution values during the improvement (optimization) process. This evidences that modifying the original method achieved new results that satisfied the proposed idea.
Figure 3. Convergence curves of HRSA and other methods for F1–F23.
Figure 4. The distribution vales of the given position during the optimization process.

3.3. Data Clustering: Experiments

In this section, the proposed method is tested further for solving the data clustering problems.
Eight problems are used with different characterizations shown in Table 7: the number of features, objects, and clusters. These datasets were taken from the UCI repository. They are always used in the domain of machine learning and clustering algorithms. The results of the proposed method are compared with other methods such as Aquila Optimizer (AO) [32], Particle Swarm Optimizer (PSO) [33], Grey Wolf Optimizer (GWO) [34], African Vultures Optimization Algorithm (AVOA) [35], Whale Optimization Algorithm (WOA) [36], Reptile Search Algorithm (RSA) [27], Remora Optimization Algorithm (ROA) [28], Arithmetic Optimization Algorithm (AOA) [37], and the proposed HRSA method. All the tested methods are tuned according to the original paper and its parameters. Each method was executed 30 times using 50 solutions and 1000 iterations.
Table 7. UCI benchmark datasets.
Table 8 shows the obtained results of the proposed HRSA method and other comparative clustering methods using eight datasets. This table presents and measures the results of nine comparative algorithms in terms of the worst, best, average fitness function values. Additionally, the standard deviation values are presented to show the variate of the presented results. The Wilcoxon signed-rank test was also used to show the degree of improvements achieved by the proposed method compared to other comparative methods. Then, the Friedman ranking test was used to find the rank of the tested methods for all the used datasets.
Table 8. The comparative results of HRSA and other clustering methods using eight datasets.
In Table 8, the proposed HRSA achieved better results in almost all the tested problems compared to the other comparative methods. The proposed method proved its performance in solving the given problems. It is the best method overall, compared to the other methods. The proposed method reaches optimal solutions in almost all the tested datasets, and its solutions are better than the previous solutions. According to the Wilcoxon signed-rank test, the proposed HRSA overwhelmed almost all the comparative methods. For example, the proposed HRSA significantly overwhelmed the cancer dataset’s ROA, GWO, PSO, and AO. Additionally, it achieved significant improvement compared to ROA, WOA, GWO, PSO, and AO in the class dataset. According to the Friedman ranking test, the proposed HRSA is ranked the first-best method, followed by the PSO method as the second best method, the GWO as the third-best method, the AO as the fourth-best method, the ROA as the fifth-best method, the AOA and RSA as the sixth-best methods, the WOA as the eight best method, and finally, the AVOA as the ninth-best method. The final results are explained in Figure 5.
Figure 5. Execution time ranking for the benchmark functions.
Figure 6 shows the convergence behavior of the comparative optimization algorithms using the eight tested data clustering problems. These figures are used to show the optimization history and the trajectory during the evolution processes. It is clear that the proposed HRSA reaches the optimal solutions in all the tested problems and converges to the optimal solution smoothly compared to the other comparative methods. Moreover, the convergence behavior observed is attractive and strengthens the main weaknesses of the original method. Thus, the main problems of the original methods are solved as given in the given figures.
Figure 6. Convergence behaviour of the comparative algorithms using the tested data clustering problems.
Figure 7 shows the clustering plot images using the eight tested datasets. This figure is used to display the clustered objects and the distribution of the objective over the specified area. The given results clearly show that the given objects are distributed in a similar area, and each area presents a similar topic. The clustering accuracy is recognized, which proves the performance of the proposed HRSA method. The execution time ranking is given in Figure 8, which clearly shows that the proposed method has a good execution time compared to other comparative methods.
Figure 7. Clustering plot images using 8 datasets.
Figure 8. Execution time ranking for the data clustering problems.
We concluded that the proposed method obtained promising results in almost all the tested problems. In comparison with other methods, the proposed method proved superior in solving the tested problems. As mentioned above, the proposed HRSA has been tested on solving the most common and standard benchmark functions; these functions are typically used to evaluate the performance of the optimization methods. According to the reported results, the proposed method obviously proved its ability to deal with different benchmark functions. Moreover, the data clustering problems are also used to evaluate the performance of the proposed method as a real-world application. The results given above showed that the proposed method obtained better results in almost all the tested problems and overwhelmed the comparative methods.

4. Conclusions

Various optimization methods have been employed in the literature to solve different machine learning problems, especially clustering problems. Data clustering is one of the complex data mining problems that clusters the given objectives, which usually amounts to a massive number of data objects, into a predefined number of clusters. Optimization methods usually suffer from optimal local problems and unbalance between the search mechanisms (i.e., exploration and exploitation). This paper proposed a novel hybrid optimization search method for solving challenging optimization problems. The proposed method is called HRSA, which combined the original Reptile Search Algorithm (RSA) and Remora Optimization Algorithm (ROA) and handled these mechanisms’ search process by a novel transition method. The proposed HRSA method solves the main weaknesses raised by the original methods and finds better solutions. The proposed HRSA is tested on various complicated optimization problems: twenty-three benchmark test functions and eight data clustering problems. The results illustrated that the proposed HRSA method performs significantly better than the original and comparative state-of-the-art methods. The proposed method was overwhelmingly better than all the other methods according to the mathematical problems. It obtained promising results in solving the clustering problems. Thus, HRSA has a remarkable efficacy when employed for various clustering problems.
In future work, the proposed method can be further studied to discover the weaknesses that can be improved. Additionally, it can be hybridized with other metaheuristic components. Moreover, the proposed HRSA can be investigated to solve other problems such as text clustering, text classification, image enhancement, image segmentation, task scheduling in computing, parameter estimation, forecasting problems, advanced mathematical problems, prediction problems, industrial problems, engineering problems, constrained mechanical design problems, home energy management, wastewater quality parameters, and other real-world problems [38,39,40]. The limitation of this paper is that the used data sets for the data clustering can be real data in the future. Additionally, in some cases, the proposed method needs more execution time.

Author Contributions

K.H.A.: Conceptualization, supervision, methodology, formal analysis, resources, data curation, and writing—original draft preparation. L.A.: Conceptualization, supervision, writing—review and editing, and project administration. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Data Availability Statement

Data is available from the authors upon reasonable request.

Acknowledgments

The authors would like to thank the Deanship of Scientific Research at Umm Al-Qura University for supporting this work by Grant Code: (22UQU4320277DSR02).

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Fiorini, L.; Cavallo, F.; Dario, P.; Eavis, A.; Caleb-Solly, P. Unsupervised machine learning for developing personalised behaviour models using activity data. Sensors 2017, 17, 1034. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  2. Alswaitti, M.; Albughdadi, M.; Isa, N.A.M. Density-based particle swarm optimization algorithm for data clustering. Expert Syst. Appl. 2018, 91, 170–186. [Google Scholar] [CrossRef] [Scilit]
  3. Gao, K.; Khan, H.A.; Qu, W. Clustering with Missing Features: A Density-Based Approach. Symmetry 2022, 14, 60. [Google Scholar] [CrossRef] [Scilit]
  4. Jain, A.K.; Murty, M.N.; Flynn, P.J. Data clustering: A review. ACM Comput. Surv. (CSUR) 1999, 31, 264–323. [Google Scholar] [CrossRef] [Scilit]
  5. Kushwaha, N.; Pant, M.; Kant, S.; Jain, V.K. Magnetic optimization algorithm for data clustering. Pattern Recognit. Lett. 2018, 115, 59–65. [Google Scholar] [CrossRef] [Scilit]
  6. Kant, S.; Ansari, I.A. An improved K means clustering with Atkinson index to classify liver patient dataset. Int. J. Syst. Assur. Eng. Manag. 2016, 7, 222–228. [Google Scholar] [CrossRef] [Scilit]
  7. Rout, R.; Parida, P.; Alotaibi, Y.; Alghamdi, S.; Khalaf, O.I. Skin lesion extraction using multiscale morphological local variance reconstruction based watershed transform and fast fuzzy C-means clustering. Symmetry 2021, 13, 2085. [Google Scholar] [CrossRef] [Scilit]
  8. Chander, S.; Vijaya, P.; Dhyani, P. Multi kernel and dynamic fractional lion optimization algorithm for data clustering. Alex. Eng. J. 2018, 57, 267–276. [Google Scholar] [CrossRef] [Scilit]
  9. George, G.; Parthiban, L. Multi objective hybridized firefly algorithm with group search optimization for data clustering. In Proceedings of the 2015 IEEE International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN), Kolkata, India, 20–22 November 2015; pp. 125–130. [Google Scholar]
  10. Kumar, Y.; Kaur, A. Variants of bat algorithm for solving partitional clustering problems. Eng. Comput. 2021, 1–27. Available online: https://www.springerprofessional.de/en/variants-of-bat-algorithm-for-solving-partitional-clustering-pro/18956468 (accessed on 20 January 2022).
  11. Mittal, H.; Pandey, A.C.; Saraswat, M.; Kumar, S.; Pal, R.; Modwel, G. A comprehensive survey of image segmentation: Clustering methods, performance parameters, and benchmark datasets. Multimed. Tools Appl. 2021, 1–26. Available online: https://link.springer.com/article/10.1007/s11042-021-10594-9 (accessed on 20 January 2022).
  12. Yin, X.; Chen, S.; Hu, E.; Zhang, D. Semi-supervised clustering with metric learning: An adaptive kernel method. Pattern Recognit. 2010, 43, 1320–1333. [Google Scholar] [CrossRef] [Scilit]
  13. Koryshev, N.; Hodashinsky, I.; Shelupanov, A. Building a fuzzy classifier based on whale optimization algorithm to detect network intrusions. Symmetry 2021, 13, 1211. [Google Scholar] [CrossRef] [Scilit]
  14. Hussein, A.M.; Abdullah, R.; AbdulRashid, N. Flower Pollination Algorithm With Profile Technique For Multiple Sequence Alignment. In Proceedings of the 2019 IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology (JEEIT), Amman, Jordan, 9–11 April 2019; pp. 571–576. [Google Scholar]
  15. Hussein, A.M.; Abdullah, R.; AbdulRashid, N.; Ali, A.N.B. Protein multiple sequence alignment by basic flower pollination algorithm. In Proceedings of the 2017 8th International Conference on Information Technology (ICIT), Amman, Jordan, 17–18 May 2017; pp. 833–838. [Google Scholar]
  16. Das, P.; Das, D.K.; Dey, S. A new class topper optimization algorithm with an application to data clustering. IEEE Trans. Emerg. Top. Comput. 2018, 8, 948–959. [Google Scholar] [CrossRef] [Scilit]
  17. Alam, S.; Dobbie, G.; Riddle, P. An evolutionary particle swarm optimization algorithm for data clustering. In Proceedings of the 2008 IEEE Swarm Intelligence Symposium, St. Louis, MO, USA, 21–23 September 2008; pp. 1–6. [Google Scholar]
  18. Agbaje, M.B.; Ezugwu, A.E.; Els, R. Automatic data clustering using hybrid firefly particle swarm optimization algorithm. IEEE Access 2019, 7, 184963–184984. [Google Scholar] [CrossRef] [Scilit]
  19. Rahnema, N.; Gharehchopogh, F.S. An improved artificial bee colony algorithm based on whale optimization algorithm for data clustering. Multimed. Tools Appl. 2020, 79, 32169–32194. [Google Scholar] [CrossRef] [Scilit]
  20. Bijari, K.; Zare, H.; Veisi, H.; Bobarshad, H. Memory-enriched big bang–big crunch optimization algorithm for data clustering. Neural Comput. Appl. 2018, 29, 111–121. [Google Scholar] [CrossRef] [Scilit]
  21. Kao, Y.T.; Zahara, E.; Kao, I.W. A hybridized approach to data clustering. Expert Syst. Appl. 2008, 34, 1754–1762. [Google Scholar] [CrossRef] [Scilit]
  22. Mahmoudi, M.R.; Akbarzadeh, H.; Parvin, H.; Nejatian, S.; Rezaie, V.; Alinejad-Rokny, H. Consensus function based on cluster-wise two level clustering. Artif. Intell. Rev. 2021, 54, 639–665. [Google Scholar] [CrossRef] [Scilit]
  23. Mahapatra, S.; Dey, B.; Raj, S. A novel ameliorated Harris hawk optimizer for solving complex engineering optimization problems. Int. J. Intell. Syst. 2021, 36, 7641–7681. [Google Scholar] [CrossRef] [Scilit]
  24. Raj, S.; Bhattacharyya, B. Optimal placement of TCSC and SVC for reactive power planning using Whale optimization algorithm. Swarm Evol. Comput. 2018, 40, 131–143. [Google Scholar] [CrossRef] [Scilit]
  25. Shaikh, M.S.; Hua, C.; Raj, S.; Kumar, S.; Hassan, M.; Ansari, M.M.; Jatoi, M.A. Optimal parameter estimation of 1-phase and 3-phase transmission line for various bundle conductor’s using modified whale optimization algorithm. Int. J. Electr. Power Energy Syst. 2022, 138, 107893. [Google Scholar] [CrossRef] [Scilit]
  26. Shekarappa, G.S.; Mahapatra, S.; Raj, S. Voltage constrained reactive power planning problem for reactive loading variation using hybrid harris hawk particle swarm optimizer. Electr. Power Components Syst. 2021, 49, 421–435. [Google Scholar] [CrossRef] [Scilit]
  27. Abualigah, L.; Abd Elaziz, M.; Sumari, P.; Geem, Z.W.; Gandomi, A.H. Reptile Search Algorithm (RSA): A nature-inspired meta-heuristic optimizer. Expert Syst. Appl. 2022, 191, 116158. [Google Scholar] [CrossRef] [Scilit]
  28. Jia, H.; Peng, X.; Lang, C. Remora optimization algorithm. Expert Syst. Appl. 2021, 185, 115665. [Google Scholar] [CrossRef] [Scilit]
  29. Shinawi, A.E.; Ibrahim, R.A.; Abualigah, L.; Zelenakova, M.; Elaziz, M.A. Enhanced Adaptive Neuro-Fuzzy Inference System Using Reptile Search Algorithm for Relating Swelling Potentiality Using Index Geotechnical Properties: A Case Study at El Sherouk City, Egypt. Mathematics 2021, 9, 3295. [Google Scholar] [CrossRef] [Scilit]
  30. Agushaka, J.O.; Ezugwu, A.E.; Abualigah, L. Dwarf Mongoose Optimization Algorithm. Comput. Methods Appl. Mech. Eng. 2022, 391, 114570. [Google Scholar] [CrossRef] [Scilit]
  31. Oyelade, O.N.; Ezugwu, A.E.; Mohamed, T.I.; Abualigah, L. Ebola Optimization Search Algorithm: A new nature-inspired metaheuristic algorithm. IEEE Access 2022. [Google Scholar] [CrossRef] [Scilit]
  32. Abualigah, L.; Yousri, D.; Abd Elaziz, M.; Ewees, A.A.; Al-qaness, M.A.; Gandomi, A.H. Aquila Optimizer: A novel meta-heuristic optimization Algorithm. Comput. Ind. Eng. 2021, 157, 107250. [Google Scholar] [CrossRef] [Scilit]
  33. Kennedy, J.; Eberhart, R. Particle swarm optimization. In Proceedings of the ICNN’95-International Conference on Neural Networks, Perth, WA, Australia, 27 November–1 December 1995; Volume 4, pp. 1942–1948. [Google Scholar]
  34. Mirjalili, S.; Mirjalili, S.M.; Lewis, A. Grey wolf optimizer. Adv. Eng. Softw. 2014, 69, 46–61. [Google Scholar] [CrossRef] [Scilit]
  35. Abdollahzadeh, B.; Gharehchopogh, F.S.; Mirjalili, S. African vultures optimization algorithm: A new nature-inspired metaheuristic algorithm for global optimization problems. Comput. Ind. Eng. 2021, 158, 107408. [Google Scholar] [CrossRef] [Scilit]
  36. Mirjalili, S.; Lewis, A. The whale optimization algorithm. Adv. Eng. Softw. 2016, 95, 51–67. [Google Scholar] [CrossRef] [Scilit]
  37. Abualigah, L.; Diabat, A.; Mirjalili, S.; Abd Elaziz, M.; Gandomi, A.H. The arithmetic optimization algorithm. Comput. Methods Appl. Mech. Eng. 2021, 376, 113609. [Google Scholar] [CrossRef] [Scilit]
  38. Gupta, S.; Abderazek, H.; Yıldız, B.S.; Yildiz, A.R.; Mirjalili, S.; Sait, S.M. Comparison of metaheuristic optimization algorithms for solving constrained mechanical design optimization problems. Expert Syst. Appl. 2021, 183, 115351. [Google Scholar] [CrossRef] [Scilit]
  39. Wang, X.; Mao, X.; Khodaei, H. A multi-objective home energy management system based on internet of things and optimization algorithms. J. Build. Eng. 2021, 33, 101603. [Google Scholar] [CrossRef] [Scilit]
  40. Abunama, T.; Ansari, M.; Awolusi, O.O.; Gani, K.M.; Kumari, S.; Bux, F. Fuzzy inference optimization algorithms for enhancing the modelling accuracy of wastewater quality parameters. J. Environ. Manag. 2021, 293, 112862. [Google Scholar] [CrossRef] [Scilit] [PubMed]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.