Data Clustering Using Moth-Flame Optimization Algorithm

A k-means algorithm is a method for clustering that has already gained a wide range of acceptability. However, its performance extremely depends on the opening cluster centers. Besides, due to weak exploration capability, it is easily stuck at local optima. Recently, a new metaheuristic called Moth Flame Optimizer (MFO) is proposed to handle complex problems. MFO simulates the moths intelligence, known as transverse orientation, used to navigate in nature. In various research work, the performance of MFO is found quite satisfactory. This paper suggests a novel heuristic approach based on the MFO to solve data clustering problems. To validate the competitiveness of the proposed approach, various experiments have been conducted using Shape and UCI benchmark datasets. The proposed approach is compared with five state-of-art algorithms over twelve datasets. The mean performance of the proposed algorithm is superior on 10 datasets and comparable in remaining two datasets. The analysis of experimental results confirms the efficacy of the suggested approach.


Introduction
Data clustering methods are being widely implemented in various real-world applications such as data mining [1], machine learning [2], information retrieval [3,4], pattern recognition [5][6][7], face clustering and recognition [8][9][10], wireless sensor networks [11], etc. The objective of this method is to partition data objects in such a way to minimize accumulated distances between data objects and their respective centroids. After clustering, the data objects in a cluster should be as similar as possible with each other and should be vastly different from the items of other clusters. A number of algorithms have been proposed to solve the data clustering problem. K-means is one among the popular algorithms to handle clustering problem. K-means algorithm is simple and efficient but the accuracy of its result highly dependent on initially selected cluster centers, hence prone to trap in local optima solution. Data clustering is one of the NP-hard problems and hence difficult to solve using deterministic algorithms. Being an NP-hard problem, deterministic approaches cause local entrapment which in turn, affects the overall performance of the algorithm.
Although the approach adopted by the k-means is being widely accepted for dealing with the clustering problems, it is deterministic in nature except for initialization. Therefore, to deal with data clustering problems a better approach could be expected compared to the k-means algorithm. Alternatively, the heuristic clustering algorithms are being widely used these days as a substitute for the conventional clustering technique. These heuristic clustering algorithms target their goals with the random search in the specified search domain. These algorithms are unable to capture the exact values, but, they try to find the values that are as close as possible to the exact solution. In this view, it is considered as an optimization problem in which the sum of the intra-cluster distances between the cluster center and associated data objects need to be minimized. To solve this problem, nature-inspired algorithms are being widely used these days [12][13][14][15][16] along with [17,18]. The problem of data clustering has attracted the attention of many researchers from different domains of science and engineering and it is still an important area of research in data mining.
Due to the involvement of non-linear objective function and large search domain in data clustering problems, the algorithms find difficulty during the optimization process. In such a case, the selection of an appropriate optimization algorithm for solving this problem becomes more important. An optimization algorithm for solving data clustering problems needs to have a balanced and proper mechanism of exploitation and exploration. Hatamlou proposes a black hole algorithm guided approach for data clustering [19]. A cuckoo search optimization algorithm based approach for data clustering is presented in [20]. A detailed review of data clustering using particle swarm optimization and its variants have been presented in [21]. Han et al. [22] proposed a modified gravitational search algorithm (GSA) for data clustering. They have included a new mechanism in GSA for improving the global search ability of solutions during the optimization. Apart from this, the use of chaotic sequences instead of random numbers improves the overall performance of optimization algorithms [23,24]. Inspired by the chaos theory, Chuang et al. [25] proposed a method for data clustering using chaotic map and PSO. Additionally, another approach based on the chaotic sequences is presented in [26] for data clustering. Another approach for data clustering based on chaotic sequence harris hawks optimizer is proposed in [27]. A chaotic number and opposition learning based method for data clustering is presented in [28]. A levy flight-based cuckoo search algorithm for data clustering is proposed in [29]. Abdulwahab et al. [30] suggested a levy flight guided approach for data clustering. The opposition learning-based approach is being successfully implemented in optimization algorithms to improve diversity during the search [31]. Kumar et al. [32] proposed an opposition learning and Cauchy operator-based scheme for data clustering. Sun et al. [33] introduced an opposition learning and random local perturbation guided monarch butterfly optimization algorithm for data clustering. Nasiri et al. [34] suggested a whale optimization algorithm based approach to handle data clustering problems. The authors have validated the effectiveness of their approach based on various comparative performance analysis. A magnetic optimization algorithm-guided approach is presented for data clustering in [14]. A hybrid approach based on exponential Grey Wolf Optimizer (GWO) and Whale Optimization Algorithm (WOA) is presented in [35] for dealing with data clustering problem. A variance-based differential evolution approach for data clustering is suggested in [36]. Zhou et al. [37] proposed a symbiotic organism search algorithm-guided approach to handle data clustering problems. A cuttlefish optimization algorithm based-approach for data clustering is presented in [38]. A density-based clusterability measure is proposed to measure the prominence of the clustering structure in the time-series data [39]. A whale optimization algorithm based approach for data clustering is presented in [40].
Although various optimization algorithms have already been suggested, none of them can outperform all other algorithms in all benchmark datasets of data clustering. A general idea of no free lunch theorem [41] states that not a single algorithm can defeat all other algorithms in all test problems. This could be the reason behind the development of various optimization algorithms for data clustering. The methods of exploitation and exploration are two main aspects that need to be balanced to achieve the desired goal in reasonable time. However, these two aspects are conflicting in nature. Excessive exploitation causes premature convergence; while excessive exploration causes random search. A proper exploitation method intentionally tries to search in the neighborhood of good solutions found so far. However, exploration is supposed to capture good solutions from the different regions of a search domain. Therefore, an optimization algorithm designed to solve the data clustering problem needs to have the capability of searching the solutions around the neighborhood of good solutions and also capturing those solutions that are far away from the poor solutions. In order to design an optimization algorithm that achieves the aforementioned capability, the programmer has to concentrate on how to update the position vectors of the solutions in an efficient manner.
Recently, Moth Flame Optimization (MFO) [42] for solving global optimization problems and real-world applications is proposed. The effectiveness of MFO in terms of population diversity and convergence rate has already been justified while solving multifaceted problems. A new method based on the MFO for data clustering is proposed in this paper. The competitiveness of the MFO such as fast convergence towards the global optimal solution and ability to avoid local optima is utilized in this study. Our main objective is to divide the data objects into clusters using MFO with better accuracy and complete coverage of search space to the existing methods. In short, the novelty and contribution of this paper are highlighted below.
• MFO based approach for data clustering is presented. • The proposed approach is evaluated using 12 machine learning benchmark datasets. • The quality of the solutions produced by the proposed approach is compared against five well-known algorithms. • Three statistical tests have been performed to measure the quality of the proposed approach statistically. • Based on experimental values, statistical values, and convergence curves, the efficacy of the proposed approach is justified.
The rest of the paper is structured as follows. In Section 2, the fundamental concepts of data clustering and the MFO algorithm is illustrated. Section 3 explains the proposed approach in detail. In Section 4, the benchmark datasets and experimental setup are described. The result analysis and discussion are given in Sections 5 and 6. Section 7 highlights the conclusions and future research directions.

Clustering
Data clustering is a method to group the given set of N data objects into K clusters. The primary objective of clustering is that the data objects belong to the same cluster must be highly similar, whereas the data objects must show high dissimilarity to those belonging to different clusters [43,44]. The similarity and dissimilarity are measured using Euclidean distance. Precisely, the sum of the square of Euclidean distances between each data object and its associated cluster center is calculated. Hence, data clustering intents to minimize the sum of intra-cluster distances. Formally it can be defined as follows: At the end of clustering process a set Z of K cluster formed as Z = Z 1 , Z 2 , . . . Z K . Each object belongs to exactly one of the formed K clusters such that objects sharing the same cluster are most similar, while objects belonging to separate clusters should be sufficiently distinct. The produced clusters and dataset should also adhere to the following criterion: In order to identify clusters and their most suitable data objects, Equation (2) is used as a fitness function in this paper to achieve the above objectives.
where the fitness value F(O, Z) is to be minimized. The Euclidean distance between a i th data object, O i and the j th cluster center, Z j is ||(O i − Z j )||. w ij is the weight associated with data object O i and cluster j. The value of w ij is assigned to 1 if i th data object is with j th cluster, else w ij = 0.

Moth Flame Optimization (MFO)
The MFO [42] is a swarm-based optimization algorithm to solve global optimization problems. This algorithm depicts moths intelligence used to navigate in nature which is known as transverse orientation. The algorithm assumes moths and their positions to be candidate solutions and problem's variables, respectively. Moths can fly in multidimensional search domains with changing their position vectors. Like other optimization algorithms, MFO initializes candidate solutions randomly within the boundary range as follows.
where S p (1 ≤ p ≤ P) is the p th solution and P is the population size. β is a random number in (0, 1).l andû represent the lower and upper bounds for the problem taken under consideration and . is a point-by-point multiplication. The population i.e., set of all solutions are represented in P × D size matrix form as follows.
where D the number of variables, represent the dimension of the problem taken under consideration. After random initialization, each solution is evaluated using fitness function and stored in a matrix of size P × 1 as follows.
where FS p is the fitness value of p th solution. Another component in MFO algorithm is flames. Similar to the moths, flames are also matrix of size P × D and represented as follows.
The fitness of each flame is evaluated and stored in a matrix of size P × 1 as follows.
Here, it is important to note that both moths and flames are solutions. The difference is, moths represent the candidate solutions and work as search agents to capture the optimal solution. On the other hand, flame keeps the top solutions searched so far and ensures that the best solution is never lost.
Moths progress in search space is controlled through logarithmic spiral function, which simulates moth's transverse orientation navigation. The positions of the p th moth S p is updated using following function: where D p represents the distance between the p th moth and q th flame, t is a random number that varies from −1 to 1, and b is a constant used to define the shape of the logarithmic spiral. Here, D p is calculated as follows.
where S p and F q are the p th moth and q th flame, respectively. The process continues until the termination condition is met. In the next Section, MFO guided approach for data clustering is presented.

Moth Flame Optimization for Data Clustering
The Moth-flame optimization algorithm is one of the nature-inspired algorithms to solve optimization problems. Since data clustering is one of the NP-hard problems, MFO is a viable candidate to solve it. MFO is a stochastic algorithm and can produce an optimal or near-optimal solution efficiently. It is also designed in such a way that it avoids local optima and converges towards a globally optimal solution. This motivates us to employ MFO for identifying clusters of data objects in the given datasets.
As mentioned in definition of clustering problem, the number of clusters K is predefined, finally represented as Z = Z 1 , Z 2 , . . . Z K . Each cluster Z k , k = 1 to K, preserves its centre. In order to find optimal solution of clustering problem using MFO algorithm, each moth corresponds to K cluster centres. Let M = {S 1 , S 2 , . . . S P )} T is the set of P moths, the p th moth S p is represented as where, S pk is the centre of k th cluster in the p th moth. Precisely, , a unique data object of the dimension D from the dataset. In general, the set of moths is represented in the form of matrix of the order (PK × D) as: The proposed approach models the clustering as a minimization problem where intracluster distance work as objective function as mentioned in (2). The fitness of each moth is computed and stored in PK × 1 order column vector FM.
Flames F = {F 1 , . . . F q . . . F P } T and corresponding fitness FF = {FF 1 , . . . FF q . . . FF P } T matrices are populated with sorted moths positions and their respective fitness in ascending order with respect to fitness. Here ∀q, q = 1 to P, F q = S t such that S t ∈ M and < FF 1 , FF 2 . . . FF P > are sorted in ascending order.
Moths iterative update their positions according to logarithmic spiral function defined in Equation (8) in turn update flames to retain top P solutions in ascending order until termination condition met.

The Procedure
The following steps are derived from the above discussion for the proposed MFO based clustering algorithm. It is assumed that dataset X of dimension D with N data objects is in place and available for processing. The number of clusters K is also initialized with a suitable integer.

Analysis of Time Complexity
In this subsection, time complexity and space complexity are presented in terms of big-oh notation. In this work, the common parameters for all approaches are the number of independent runs R, population size P, and the maximum number of iterations Maxitr. For the proposed approach, a careful observation indicates that the time complexity will be O(Maxitr × P × N). Here, N is the number of data objects in a benchmark dataset taken under consideration. Hence this number varies with respect to the dataset. Therefore, based on common parameters, the time complexity of the proposed approach is O(Maxitr × P). On the other hand, space complexity is the total amount of memory space used by the benchmark dataset and other variables during the program execution. Simple variables such as P, Maxitr etc. need constant space as the size of these data types is constant. Therefore, simple variables add a space complexity of O(1). However, to store the randomly initialized and intermediate populations in the memory, it needs O(P × K × D). Here, K is the number of clusters present in a benchmark dataset, whereas D is the number of features in a benchmark dataset. P is the population size.

Experimental Setup
The quality of the solutions of the proposed algorithm was compared against Black Hole Algorithm (BHA) [19], Multi-Verse Optimizer (MVO) [45], Harris Hawks Optimizer (HHO) [46], Grey Wolf Optimizer (GWO) [47], and K-means algorithm [44]. The parameters of BHA, MVO, HHO, GWO, and k-means algorithms were set according to their corresponding references [19,[44][45][46][47], respectively. In this study, three parameters were common for all the approaches. The values of these common parameters were considered as follows: • Population size = 50 • Maximum iterations = 1000 • Independent runs = 20 The details of benchmark datasets used in this study are given in Tables 1 and 2. All the employed datasets used multivariate real valued features for characterizing individual objects.

Results Analysis
The experimental results of algorithms for benchmark datasets are given in Tables 3 and 4. These tables represent the quantitative values described with Equation (2). The statistical metrics best, worst, mean and standard deviation of achieved objective values by participating algorithms are compared. Since the performance of metaheuristic randomized algorithms are not deterministic, best and worst values in different runs may reasonably represent upper and lower performance bounds respectively. The mean may represent the center of all objective function values obtained in multiple runs. In addition to that standard deviation helps to understand the performance consistency of algorithm. Lower the standard deviation, more consistent the performance of algorithm around mean. Table 5 represents the relative average rank of the algorithms considered in this study. This table indicates that the proposed approach outperformed all other approaches considered in this study. Whereas, the k-means algorithm observed the worst performance in all benchmark datasets.
Iman-Davenport [48], Friedman [49], and Holm [50] tests were carried out to demonstrate the effectiveness of the algorithms in solving the data clustering problem. Tables 6 and 7 show the experimental values of Iman-Davenport and Friedman tests for the Shape and UCI datasets, respectively. The rejection of the null hypothesis for both cases confirmed that there was a significant difference in the performances of the algorithms. Therefore, a post hoc test (Holm test) was performed to show the efficacy of the best performing algorithm against the rest of the algorithms. Tables 8 and 9 represent the experimental results of the Holm test for Shape and UCI datasets, respectively. The rejection of the null hypothesis for k-means and MVO algorithms indicated that the proposed approach was statistically better than these algorithms. The null hypothesis was not rejected for GWO, HHO, and BHA which indicated that the performance of these algorithms was comparable with the proposed approach. However, Table 5 shows that the proposed approach was better than GWO, HHO, and BHA.      6 show the convergence curves of algorithms for benchmark datasets taken under consideration. These curves show that the convergence rate of the suggested approach was much better than other approaches. Comparing algorithms using the rate of convergence allowed us to analyze how quick they captured the optimal solution.

Discussion
To further strengthen the results mentioned in the previous section, Tables 10-21 represent the best centroids for D31, R15, Jain, Flame, Aggregation, Compound, Pathbased, Spiral, Glass, Iris, Wine, and Yeast datasets, respectively obtained by the suggested approach. In these tables, C1, C2, · · · , CK represent the best centroids obtained by the proposed approach, whereas F1, F2, · · · , FD represent the feature/attribute number of the dataset taken under consideration. Based on these centroids, the best value of the sum of fitness values of the respective benchmark dataset in Tables 3 and 4 can be validated. By assigning each data object of a benchmark dataset to the respective centroid in Tables 10-21, the best values in Tables 3 and 4 are ideally expected. This can be understood as, by putting all the 240 data objects of the Flame dataset to one of the centroids between the two centroids that are presented in Table 13, the best value of the fitness function found by the MFO algorithm for the Flame dataset should be 770.09978. If this value (770.09978) is not found, then either the best value (770.09978) reported in Table 3 or the best centroids reported in Table 13 or both allocations are wrong. This method can be applied to validate the best value of the remaining benchmark datasets.

Conclusions and Future Research Directions
Various optimization algorithms based on natural phenomena have been used to solve complex problems. Moth flame optimization algorithm is one of them that uses the navigation behavior of moths at night. In this paper, a moth flame optimizer guided approach is suggested for data clustering. The effectiveness of the suggested method is validated using twelve standard benchmark datasets. Various comparisons using the experimental values have proved the effectiveness and competitiveness of the suggested method. The proposed approach achieves better accuracy and complete coverage of search space in comparison to the existing methods.
In the future, research can be carried out to solve some other real-world problems of clustering such as optimal controller selection in wireless sensor networks, controller placement problems in a software-defined network, clustering in image segmentation by using the proposed approach. The suggested approach can also be extended to solve multiobjective optimization problems.